CTWatch
August 2007
The Coming Revolution in Scholarly Communications & Cyberinfrastructure
Incentivizing the Open Access Research Web
Publication-Archiving, Data-Archiving and Scientometrics
Tim Brody, University of Southampton, UK
Les Carr, University of Southampton, UK
Yves Gingras, Université du Québec à Montréal (UQAM)
Chawki Hajjem, Université du Québec à Montréal (UQAM)
Stevan Harnad, University of Southampton, UK; Université du Québec à Montréal (UQAM)
Alma Swan, University of Southampton, UK; Key Perspectives

6
Validating Research Metrics

The one last parallel panel/metric RAE, in 2008, will provide a unique natural testbed for validating the rich new spectrum of Open Access metrics against the panel rankings. A statistical technique called multiple regression analysis can compute the contribution of each individual metric to the joint correlation of all the metrics with the RAE panel rankings. Once initialized by being validated against the panel rankings, the relative weight of each metric can then be adjusted and optimised according to the needs and criteria of each discipline, with the panels only serving as overseers and fine-tuners of the metric output, rather than having to try to re-review all the publications. This will allow research productivity and progress to be systematically monitored and rewarded.27

This is a natural, ‘horizontal’ extension of Citebase’s current functionality but it does not need to be restricted to the UK RAE: once validated, the metric equations, with the weights suitably adjusted to each field, can provide ‘continuous assessment’ of the growth and direction of scientific and scholarly research. Not only will the network of P-OA IRs do double duty by providing access to research for researchers as well as serving as the database for the field of scientometrics, but it will also provide an incentive for data-archiving (D-OA) alongside publication-archiving (P-OA) for other fields too, both by providing an example of the power and potential of such a worldwide database in scientometrics and by providing potential new impact metrics for research data-impact, alongside the more familiar metrics for research publication-impact.

The Open Access Impact Advantage

Citebase has already been able to demonstrate that, in physics, OA self-archiving dramatically enhances citation impact (Figure 4a) for articles deposited in Arxiv, compared to articles in the same journal and year that are not self-archived.28 Lawrence 29 had already shown this earlier for computer science. The advantage has since been confirmed in 10 further disciplines (Figure 4b) using the bibliographic metadata from the ISI Science and Social Science Citation Index (on CD-ROM leased to OST at UQAM) for millions of articles in thousands of journals, for which robots then trawled the web to see whether they could find a free online (OA) versions of the full text. An OA/non-OA citation advantage – OA articles are cited more than non-OA articles in the same journal and year - has been found in every discipline tested so far (and in every year except the two very first years of Arxiv).

Figure 4
Figure 4a & b. Open Access citation advantage: Although only a small proportion of articles is currently being made Open Access, those articles are cited much more than those articles (in the same journal and year) that are not.
4a. Particle physics, based on Arxiv.
Figure 4b

By discipline: total articles (OA+NOA), gray curve; percentage OA: (OA/(OA+NOA)) articles, green bars; percentage OA citation advantage: ((OA-NOA)/NOA) citation, red bars, averaged across 1992-2003 and ranked by total articles. All disciplines show an OA citation advantage (Hajjem et al. IEEE DEB 2005).
4b. Ten other fields, based on webwide robot searches.

Pages: 1 2 3 4 5 6 7

Reference this article
Harnad, S., Brody, T., Carr, L., Gingras, Y., Hajjem, C., Swan, A. "Incentivizing the Open Access Research Web," CTWatch Quarterly, Volume 3, Number 3, August 2007. http://www.ctwatch.org/quarterly/articles/2007/08/incentivizing-the-open-access-research-web/

Any opinions expressed on this site belong to their respective authors and are not necessarily shared by the sponsoring institutions or the National Science Foundation (NSF).

Any trademarks or trade names, registered or otherwise, that appear on this site are the property of their respective owners and, unless noted, do not represent endorsement by the editors, publishers, sponsoring institutions, the National Science Foundation, or any other member of the CTWatch team.

No guarantee is granted by CTWatch that information appearing in articles published by the Quarterly or appearing in the Blog is complete or accurate. Information on this site is not intended for commercial purposes.