Press release - “Not everything that counts can be counted…”
06 June 2012
“… and not everything that can be counted counts”. Albert Einstein’s famous saying certainly resonates when it comes to assessing the fruits of university research. In its latest paper published today, the League of European Research Universities (LERU) opens a debate on why, how and for/by whom university research is evaluated.
Governments have by and large understood the need to invest in research because it is a vital engine driving innovative, knowledge-driven economies. It goes without saying that they - and funders of research in general - want to evaluate the fruits of their investment. But the growth in research evaluation regimes risks to create (has already done so, one may argue) a sometimes unhealthy obsession with monitoring and measuring research. This potentially has some undesirable consequences: demands on universities to produce excessive amounts of data straining finite human and financial resources, unhelpful or conflicting duplication of various assessment exercises, a short-sighted “bean counting” culture and other practices that detract from rather than support high quality research.
Of course universities themselves assess the research performed within their walls for a variety of reasons. Along with governments and research funders they want to gauge research output, quality and impact, improve performance and maximise return on investment. Universities are also interested in research assessment as a way to inform strategic planning and positioning of the university, to invest in areas of research strengths or new directions, to expose weaknesses, to identify and track individual accomplishments, to recruit, retain or reward top performers, to find and foster productive research collaborations, etc.
Even when it’s clear who wants to assess research for what purpose, further challenges stem from the fact that assessment can be performed in different ways. Peer review (basically asking other researchers to evaluate the research) is a wide-spread method, but costly, time-consuming and open to subjectivity or bias. Another method is to collect bibliometric data including number of publications, citation frequency etc. While it is less costly, the bibliometric approach also has drawbacks. Moreover, a new trend is to show research impact, i.e. evidence that a piece of research is in a broad sense useful to society. While impact has its place, it should be understood that it is not the driving force of research. Whatever the method, clearly there is a need for sophisticated tools, but it is equally necessary to understand their limits.
What can be done in practice? Universities for example need to have ample human expertise and sophisticated research assessment tools suited to the task of assessing universities’ research strengths and weaknesses. They should maintain central databases capable of producing fine-grained, accurate and up-to-date HR and research data. They can also support the emerging practice of using unique personal identifiers to avoid ambiguities about researchers’ correct names.
External agencies should avoid creating perverse incentives for universities and researchers and should ensure consistency for reliable comparisons locally and internationally. Above all, in assessing university research they need to appreciate that research often has a long term outlook rather than a concern with immediate return on investment.
Our main point is to call for a sensible approach to research assessment. Governments, universities and others should “assess assessment”, carefully looking at what works in different research environments and building on good practice where there is valid evidence that the process leads to demonstrable improvements in productivity and impact.
The paper will be presented during a breakfast launch event in Brussels on 19 June 2012.
---- ENDS ------
Notes for editors
This paper and all other LERU publications are freely available online at http://www.leru.org
The League of European Research Universities (LERU) is as an association of leading research-intensive universities that share the values of high-quality teaching within an environment of internationally competitive research. Founded in 2002, LERU advocates education through an awareness of the frontiers of human understanding; the creation of new knowledge through basic research, which is the ultimate source of innovation in society; and the promotion of research across a broad front in partnership with industry and society at large.
The LERU universities are:
- Universiteit van Amsterdam
- Universitat de Barcelona
- University of Cambridge
- University of Edinburgh
- Albert-Ludwigs-Universität Freiburg
- Université de Genève
- Universität Heidelberg
- Helsingin yliopisto (University of Helsinki)
- Universiteit Leiden
- KU Leuven
- Imperial College London
- University College London
- Lunds universitet
- Università degli Studi di Milano
- Ludwig-Maximilians-Universität München
- University of Oxford
- Université Pierre et Marie Curie, Paris 6
- Université Paris-Sud 11
- Université de Strasbourg
- Universiteit Utrecht