Wednesday, 17 June 2009

Research evaluation & bibliometrics

At my university there is a strong push, by... someone, to use bibliometric indicators to evaluate research productivity. My position is:
  • Bibliometrics is useful. It would be crazy not to use it.
  • Bibliometrics is not enough. It would be equally crazy to rely on bibliometrics only. A more general, scientometrics-based approach, should be used.
  • Expert peer review, informed by bibliometrics but not only, is the only reliable evaluation mechanism.
  • Evaluation needs resources (money, people, time, data, ...).
  • Evaluation strategies and approaches should be decided by evaluation experts, not by beginners.
Probably, someone, somewhere in the world, is fighting trying to convince they colleagues and/or administrators, that bibliometrics is useful. At my university, I'm in the opposite position: I'm fighting trying to convince people that bibliometrics can give some indication, but it's just one of the parameters to be measured, and that it is stupid to rely on bibliometrics only.

BTW, so far I did not get much success - they just pretend I'm not saying anything. Well, actually they've changed from using bibliometrics to evaluate single researchers (a position that demonstrated their level of... erm let's be polite and say "expertise"...) to using bibliometrics to evaluate groups of researchers. But I'm stubborn ;-)

Anyway, I'm writing this post because I'm happy that in the last REF report one can read:
There was a strong consensus that bibliometrics are not sufficiently mature to be used formulaically or to replace expert review, but there is considerable scope for citation indicators to inform expert review in the REF.
Let's see if bibliometrics enthusiast at my university will take this into account, or just pretend it doesn't exist.

And, yes, of course, let's see if bibliometrics-retractors will take this into account, or just pretend it doesn't exist.


No comments: