REF and bibliometrics
The Research Excellence Framework (REF) informs research funding allocations by the four UK HE funding bodies, including HEFCE. Bibliometrics such as citation counts, journal impact factors, and h indexes are not used directly, but they may feed into the assessment process for some units of assessment.
Brunel’s performance in the 2008 Research Assessment Exercise (RAE) performance heralded our arrival as a research-intensive university, and a university of international standing. REF 2014 replaced the RAE. See REF 2014 at Brunel for information about the University's submission and results.
About the REF
- It is a single system for assessing research quality across all disciplines
- There are 20 units of assessment, reduced from 24 in the RAE
- It generates profiles, like the RAE
- It determines quality-related research (QR) income
- The REF is based on output, environment and impact whereas the RAE was based on outputs, environment and esteem
- Weightings given to each element of assessment are: outputs (60%), environment (15%) and impact (25%)
- A maximum of four outputs per researcher can be submitted
- The evaluation of outputs will be done by peer review, with limited use of citation data in some units of assessment
- Outputs assessed in REF 2014 must have been published after 1 January 2008 and before 31 December 2013
- For REF 2020, HEFCE has announced that only journal articles and conference proceedings which have made open access upon acceptance for publication, and are deposited in BRAD and BURA will be eligible for assessment. Outputs which do not comply and are not covered by an exception will receive a zero rating.
Key events in the 2014 REF timetable:
- March to June 2009 – Bibliometrics pilot exercise involving 22 universities
- March 2011 – Publication of Decisions on accessing research impact
- July 2011 – Publication of Assessment framework and guidance on submissions
- Autumn 2012 – Pilot of the submissions systems
- 29 November 2013 – Closing date for submissions
- 18 December 2014 – Publication of outcomes
- March 2015 – Funding letters
After the bibliometric pilot exercise, it was concluded that citations were insufficiently robust to be used in a formulaic way or to replace peer review. Instead it was decided that the evaluation of outputs may be “informed by” bibliometrics, ie someone could consider an item’s citations and form a judgement in some units of assessment. Citation data used in REF 2014 was provided by Scopus®.
More information about the REF.
You can find citation counts for individual journal articles using the three multidisciplinary databases: Web of Science, Scopus and Google Scholar.
Using Web of Science:
- Open Web of Science
- Search for the article by title or DOI
- See the citation count towards the bottom of the entry
- Open Scopus
- Search for the article by title or DOI
- See the citation count to the far right of the entry
Using Google Scholar:
- Open Google Scholar
- Search for the article using by title or DOI
- See the citation count beneath the entry. If the citation count is missing, it means there are no citations.
These three databases don’t always give the same citation count for a given article, because they cover different data. Usually but not always Google Scholar will give the highest number. If you are comparing citation counts for different articles, you should use the same database.
The InCites database provides access to bibliometric reports based on Brunel publications indexed in the Web of Science. This data can be used in a variety of different ways, for example:
- To compare quantitative aspects of performance against other institutions and world and field benchmarks
- To identify influential and emerging researchers and research trends
- To showcase strengths and identify potential areas for growth
- To monitor collaboration activity and identify new collaboration opportunities
- To support accreditation activity, funding proposals, legislative agendas, alumni appeals, faculty and student recruitment
Various members of staff have been set up with personal accounts. All other staff and students: to access this e-resource you will require a unique username and password. If you are already registered for related Thomson Reuters products such as Web of Science™, ResearcherID, or EndNote® then your username and password will work for InCites, too.