Skip to main content
SpringerLink
Log in

A note on reference publication year spectroscopy with incomplete information

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Reference publication year spectroscopy (RPYS) is a scientometrics method that consists in the study of the annual distribution of a corpus’ cited references. The method has already been used in many studies to investigate the historical roots of a research field, topic, or journal. While empirical applications are numerous, no study has yet investigated the statistical aspects of the method and especially how to deal with uncertainty or incompleteness in the data. In this methodological note, we focus on practical issues: the choice of the smoothing method for the running median at the beginning and at the end of the selected timespan, and the presence of reference publication years with zero cited references. The study is based on four datasets for influential journals in economics, management and finance, and some results are generalized using simulated data. We conclude that the “constant” smoothing method is preferable for the calculation of the running median and we discuss the implications of the different ways of dealing with years with zero citations, providing practical recommendations to researchers using RPYS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. Available at http://www.leydesdorff.net/software/rpys/. Accessed February 17th, 2021.

  2. CRExplorer procedure is detailed in the manual of the software (Thor et al. 2018). We could not find a description of this behavior in the official documentation of bibliometrix package, so we report what is our understanding of the source code.

  3. CRExplorer’s approach has been communicated to us by its developers. For bibliometrix package, we could not find a description of the process in the official documentation, so we report what is our understanding of the source code.

  4. The code for the two simulations is available upon request.

  5. For each journal, we collected as much data as our inhouse access to Web of Science permitted. The number of documents per journal are: QJE (1334), JF (3108), ECON (1946), JMS (2050). We uploaded the datasets in CRExplorer and exported the data on number of citations per publication year for further analysis.

  6. Thor et al. 2018 also suggest splitting the analysis in different consecutive and non-overlapping periods, although for simplifying the interpretation of the spectrogram.

References

  • Aria, M., & Cuccurullo, C. (2017). bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4), 959–975. https://doi.org/10.1016/j.joi.2017.08.007.

    Article  Google Scholar 

  • Barth, A., Marx, W., Bornmann, L., & Mutz, R. (2014). On the origins and the historical roots of the Higgs boson research from a bibliometric perspective. The European Physical Journal Plus, 129(6), 111. https://doi.org/10.1140/epjp/i2014-14111-6L.

    Article  Google Scholar 

  • Bornmann, L. (2018). PLOTRPYS: Stata module to plot a spectrogram using CSV export from CRExplorer. https://EconPapers.repec.org/RePEc:boc:bocode:s458378. Accessed 17 Feb 2021.

  • Bornmann, L., Haunschild, R., & Leydesdorff, L. (2018). Reference Publication Year Spectroscopy (RPYS) of Eugene Garfield’s publications. Scientometrics, 114(2), 439–448. https://doi.org/10.1007/s11192-017-2608-3.

    Article  Google Scholar 

  • Bornmann, L., & Marx, W. (2013). The proposal of a broadening of perspective in evaluative bibliometrics by complementing the times cited with a cited reference analysis. Journal of Informetrics, 7(1), 84–88. https://doi.org/10.1016/j.joi.2012.09.003.

    Article  Google Scholar 

  • Comins, J. A., & Leydesdorff, L. (2016). RPYS i/o: software demonstration of a web-based tool for the historiography and visualization of citation classics, sleeping beauties and research fronts. Scientometrics, 107(3), 1509–1517. https://doi.org/10.1007/s11192-016-1928-z.

    Article  Google Scholar 

  • Franceschini, F., Maisano, D., & Mastrogiacomo, L. (2014). Scientific journal publishers and omitted citations in bibliometric databases: Any relationship? Journal of Informetrics, 8(3), 751–765. https://doi.org/10.1016/j.joi.2014.07.003.

    Article  Google Scholar 

  • Franceschini, F., Maisano, D., & Mastrogiacomo, L. (2016). Empirical analysis and classification of database errors in Scopus and Web of Science. Journal of Informetrics, 10(4), 933–953. https://doi.org/10.1016/j.joi.2016.07.003.

    Article  Google Scholar 

  • Haunschild, R., Bauer, J., & Bornmann, L. (2019). Influential cited references in FEMS microbiology letters: Lessons from Reference Publication Year Spectroscopy (RPYS). FEMS Microbiology Letters. https://doi.org/10.1093/femsle/fnz139.

    Article  Google Scholar 

  • Marx, W. (2011). Special features of historical papers from the viewpoint of bibliometrics. Journal of the American Society for Information Science and Technology, 62(3), 433–439. https://doi.org/10.1002/asi.21479.

    Article  Google Scholar 

  • Marx, W., & Bornmann, L. (2010). How accurately does Thomas Kuhn’s model of paradigm change describe the transition from the static view of the universe to the big bang theory in cosmology? A historical reconstruction and citation analysis. Scientometrics, 84(2), 441–464. https://doi.org/10.1007/s11192-009-0107-x.

  • Marx, W., & Bornmann, L. (2016). Change of perspective: Bibliometrics from the point of view of cited references—a literature overview on approaches to the evaluation of cited references in bibliometrics. Scientometrics, 109(2), 1397–1415.

    Article  Google Scholar 

  • Marx, W., Bornmann, L., Barth, A., & Leydesdorff, L. (2014). Detecting the historical roots of research fields by Reference Publication Year Spectroscopy (RPYS). Journal of the Association for Information Science and Technology, 65(4), 751–764. https://doi.org/10.1002/asi.23089.

    Article  Google Scholar 

  • Marx, W., Haunschild, R., Thor, A., & Bornmann, L. (2017). Which early works are cited most frequently in climate change research literature? A bibliometric approach based on reference publication year spectroscopy. Scientometrics, 110(1), 335–353.

    Article  Google Scholar 

  • McLevey, J., & McIlroy-Young, R. (2017). Introducing metaknowledge: Software for computational research in information science, network analysis, and science of science. Journal of Informetrics, 11(1), 176–197. https://doi.org/10.1016/j.joi.2016.12.005.

    Article  Google Scholar 

  • Merton, R. K. (1949). Social theory and social structure. Chicago: Free Press of Glencoe.

  • R Core Team (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/. Accessed 17 Feb 2021.

  • Rhaiem, M., & Bornmann, L. (2018). Reference Publication Year Spectroscopy (RPYS) with publications in the area of academic efficiency studies: What are the historical roots of this research topic? Applied Economics, 50(13), 1442–1453. https://doi.org/10.1080/00036846.2017.1363865.

    Article  Google Scholar 

  • Thor, A., Marx, W., & Bornmann, L. (2018). CitedReferenceExplorer (CRExplorer) manual. (For version 1.9, July 16 2018). From: http://andreas-thor.github.io/cre/manual.pdf

  • Thor, A., Marx, W., Leydesdorff, L., & Bornmann, L. (2016a). Introducing CitedReferencesExplorer (CRExplorer): A program for reference publication year spectroscopy with cited references standardization. Journal of Informetrics, 10(2), 503–515. https://doi.org/10.1016/j.joi.2016.02.005.

    Article  Google Scholar 

  • Thor, A., Marx, W., Leydesdorff, L., & Bornmann, L. (2016b). New features of CitedReferencesExplorer (CRExplorer). Scientometrics, 109(3), 2049–2051. https://doi.org/10.1007/s11192-016-2082-3.

    Article  Google Scholar 

  • Tukey, J. W. (1977). Exploratory data analysis. London: Addison-Wesley Publishing Company.

  • Velleman, P. F., & Hoaglin, D. C. (1981). Applications, basics, and computing of exploratory data analysis. Boston, Mass: Duxbury Press.

  • Yeung, A. W. K., & Wong, N. S. M. (2019). The historical roots of visual analog scale in psychology as revealed by Reference Publication Year Spectroscopy. Frontiers in Human Neuroscience. https://doi.org/10.3389/fnhum.2019.00086.

    Article  Google Scholar 

Download references

Acknowledgments

We are grateful to Lutz Bornmann and Andreas Thor for discussions about the topic of this note. We are also indebted to Lutz Bornmann for his comments and suggestions on previous versions of the manuscript. We are also very grateful to the two anonymous reviewers for their helpful comments on previous drafts

Author information

Authors and Affiliations

  1. ESSCA School of Management, 1 rue Joseph Lakanal, 49003, Angers, France

    Matthieu Ballandonne & Igor Cersosimo

Authors
  1. Matthieu Ballandonne
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Igor Cersosimo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthieu Ballandonne.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ballandonne, M., Cersosimo, I. A note on reference publication year spectroscopy with incomplete information. Scientometrics 126, 4927–4939 (2021). https://doi.org/10.1007/s11192-021-03976-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-021-03976-1

Keywords

Search

Navigation