This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (18) Citing Articles via Google Scholar Google Scholar Articles by Warner, J. Search for Related Content Social Bookmarking                   What's this?

A critical review of the application of citation studies to the Research Assessment Exercises

The Queen’s University of Belfast, Northern Ireland, UK, j.warner{at}qub.ac.uk

The Research Assessment Exercises (RAE) conducted in the UK have attractedvarious types of published response. These include citation analyses and a review of the public reception of the RAE 1996, which included a brief critique of the citation studies.

This paper develops the critique. Largely unexplored issues in the theory or assumptions of bibliometrics, e.g. the level of citation which corresponds to a quantum of research quality, are found to emerge in the studies. A weak, and unsatisfactorily treated, correlation between citation aggregates and research quality for individual entities is revealed. The proposal to replace informed peer review by citation analysis is regarded as highly unrealistic.

Productive uses for citation analysis in research evaluation are suggested. A historically rare instance of correlation between rankings derived from citation aggregates and from real world peer review has been revealed by the studies. The future value of citation analysis could be to inform, but not to determine, judgements of research quality.

A combination of methods is advocated for future studies of the RAEs. Information science must attend to considerations of value, as well as using established techniques, if it is to avoid marginalisation.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				I. Aaltojarvi, I. Arminen, O. Auranen, and H.-M. Pasanen Scientific Productivity, Web Visibility and Citation Patterns in Sixteen Nordic Sociology Departments Acta Sociologica, March 1, 2008; 51(1): 5 - 22. [Abstract] [PDF]

				J. M. Diaz-Puente, A. Cazorla, and A. Dorrego Crossing National, Continental, and Linguistic Boundaries: Toward a Worldwide Evaluation Research Community in Journals of Evaluation American Journal of Evaluation, December 1, 2007; 28(4): 399 - 415. [Abstract] [PDF]

				V. Bence and C. Oppenheim The Influence of Peer Review on the Research Assessment Exercise Journal of Information Science, August 1, 2004; 30(4): 347 - 368. [Abstract] [PDF]