Skip to main content
Log in

Publication types, citation rates and evaluation

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

In order to resolve questions frequently raised in the context of research evaluation about the citation rates of journal publications in relation to other types of publications, the total research output of substantial institutions or systems has to be brought under bibliographic control. That precondition has rarely been met: there are few published studies of the total range of publications of major research institutions, including books, book chapters, technical reports and published conference proceedings. The Research Evaluation and Policy Project (REPP) at the Australian National University (ANU) has established a database covering all the publications from the Institute of Advanced Studies (IAS), a fulltime research institution at the ANU, and has examined in detail citations in the journal literature accruing to all types of publications. The database contains a significant number of publications, nearly 30 000 items, and covers the sciences and the social sciences and humanities. This data enables us to examine whether the citation record of research publications appearing in journals indexed by the Institute for Scientific Information (ISI) is a useable surrogate for the citation record within ISI journals of all model of publication. We contend that, if certain preconditions are met, the choice of citation rate is not critical.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References and notes

  1. W. Shrum, Y. Shenhav, Science and technology in less developed countries, In:S. Jasanoff et al. (Eds),Handbook of Science and Technology Studies, Sage Publications Inc., California, 1995, pp. 627–651.

    Google Scholar 

  2. P. Bourke, L. Butler,Monitoring Research in the Periphery: Australia and the ISI indices. Research Evaluation and Policy Project Monograph Series No. 3, Research School of Social Sciences, The Australian National University, 1966.

  3. National Board of Employment, Education and Training (NBEET).Quantitative Indicators of Australian Academic Research, Commissioned Report No. 27, Australian Government Publishing Service, Canberra, 1994.

    Google Scholar 

  4. The CPR database includes information on research output in both published and unpublished modes, but our analysis is restricted to published output only.

  5. P. Bourke, L. Butler, 1996,op. cit. Monitoring Research in the Periphery: Australia and the ISI indices. Research Evaluation and Policy Project Monograph Series No. 3, Research School of Social Sciences, The Australian National University.

  6. The fields listed here are from the Australian Standard Research Classification which was prepared by the Australian Bureau of Statistics for use in the measurement and analysis of research and experimental development in Australia.

  7. P. Bourke, L. Butler, The use of bibliometric data in evaluating a research university: Issues and measures, In:M. Koenig, A. Bookstein (Eds),Fifth International Conference of the International Society for Scientometrics and Informetrics Proceedings — 1995. Learned Information Inc., NJ., 1995, pp. 93–102.

    Google Scholar 

  8. The Research School of Social Sciences and the Research School of Pacific and Asian Studies.

  9. S. Maricic, Impact factors of peripheral journals,Journal of Documentation, 45(4) (1989) 330–331.

    Google Scholar 

  10. L. J. Zmaic, S. Maricic, V. Simeon, Visibility of peripheral journals through the Science Citation Index,Information Processing & Management, (1989) 25.

  11. The six IAS research schools referred to in this table are the Research School of Chemistry (RSC), John Curtin School of Medical Research (JCSMR), Mount Stromlo and Siding Springs Observatories (MSSSO), Research School of Earth Sciences (RSES), Research School of Physical Sciences (RSPS) and Research School of Biological Sciences (RSBS).

  12. It is interesting to look at the two most highly cited non-SCI journal publications. One is from RSBS and appeared in theJournal of Molecular and Applied Genetics in 1981. This journal appears to have become a source journal forCurrent Contents in 1984 and 1985. The other article was from JCSMR and appeared in the journalComprehensive Virology. This journal was a source journal for the SCI in 1983 and 1984. The details of these two highly cited non-SCI journal articles (or, more accurately, non-SCI at the time of their publication) suggests that the common story for the more highly cited Australian publications of this type may be that they are in journals on the fringe of being accepted for indexing by ISI.

  13. In this particular instance the departments are not directly comparable as they undertake research in quite different fields; but if this analysis were of ‘like’ departments, it is easy to see the significance of the choice of citation rate.

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bourke, P., Butler, L. Publication types, citation rates and evaluation. Scientometrics 37, 473–494 (1996). https://doi.org/10.1007/BF02019259

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02019259

Keywords

Navigation