Skip to main content
Log in

The role of baseline granularity for benchmarking citation impact. The case of CSS profiles

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

In this paper we study the effect of granularity on Characteristic Scores and Scales (CSS). Unlike the traditional indicators that are mostly based on means and quantiles, CSS require the reduction of the citation distributions collaboration of the underlying reference population to four states (classes) and thus higher a different level of granularity. While the question of the choice of granularity is at higher levels of aggregation usually not critical since countries and university have rather multidisciplinary profiles, at lower aggregation levels specialisation becomes more typical. Inappropriate granularity might not warrant the depiction of the publication profiles at these levels in a correct and adequate manner and thus not add accurate citation profiles either. In order to be able to process one complete annual volume of the Web of Science, we decided to calculate CSS thresholds and classes for two levels of granularity, namely sub-fields and WoS Subject Categories. With about 5% deviation, we did not find a real significance. However, we identified journals with similar impact measures but different citation profiles, independently of the granularity. Finally, we have pointed to the limitations in the choice of granularity—in terms of both too broad and too narrow subjects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

(Data sourced from Clarivate Analytics Web of Science Core Collection)

Fig. 2

(Data sourced from Clarivate Analytics Web of Science Core Collection)

Similar content being viewed by others

References

  • Adams, J., Gurney, K., & Jackson, L. (2008). Calibrating the zoom—A test of Zitt’s hypothesis. Scientometrics, 75(1), 81–95.

    Article  Google Scholar 

  • Adams, J., & Testa, J. (2011). Thomson Reuters book citation index. In E. Noyons, P. Ngulube, J. Leta (Eds.), The 13th conference of the international society for scientometrics and informetrics (Vol. I, pp. 13–18). Durban, South Africa: ISSI, Leiden University and the University of Zululand.

  • Albarrán, P., & Ruiz-Castillo, J. (2011). References made and citations received by scientific articles. JASIST, 62(1), 40–49.

    Article  Google Scholar 

  • Bookstein, A. (1997). Informetric distributions. 3. Ambiguity and randomness. Journal of the American Society for Information Science, 48(1), 2–10.

    Article  Google Scholar 

  • Glänzel, W. (2007). Characteristic scores and scales. A bibliometric analysis of subject characteristics based on long-term citation observation. Journal of Informetrics, 1(1), 92–102.

    Article  Google Scholar 

  • Glänzel, W. (2009). The multi-dimensionality of journal impact. Scientometrics, 78(2), 355–374.

    Article  Google Scholar 

  • Glänzel, W. (2011). The application of characteristic scores and scales to the evaluation and ranking of scientific journals. Journal of Information Science, 37(1), 40–48.

    Article  MathSciNet  Google Scholar 

  • Glänzel, W., Meyer, M., Schlemmer, B., du Plessis, M., Thijs, B., Magerman, T., Debackere, K., Veugelers, R. (2003), NanotechnologyAnalysis of an emerging domain of scientific and technologic endeavour. Accessible via: https://www.ecoom.be/sites/ecoom.be/files/downloads/nanotech_domain_study.pdf.

  • Glänzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(2), 171–193.

    Article  Google Scholar 

  • Glänzel, W., & Schubert, A. (1988). Characteristic scores and scales in assessing citation impact. Journal of Information Science, 14(2), 123–127.

    Article  Google Scholar 

  • Glänzel, W., & Schubert, A. (2003). A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics, 56(3), 357–367.

    Article  Google Scholar 

  • Glänzel, W., Schubert, A., Thijs, B., & Debackere, K. (2009). Subfield-specific normalized relative indicators and a new generation of relational charts: Methodological foundations illustrated on the assessment of institutional research performance. Scientometrics, 78(1), 165–188.

    Article  Google Scholar 

  • Glänzel, W., & Thijs, B. (2017). The granularity of disciplinary structures for benchmarking citation impact. The case of CSS profiles. In Proceedings of ISSI 2015—The 15th international conference on scientometrics and informetrics (pp. 1190–1200). Wuhan, China.

  • Glänzel, W., Thijs, B., & Debackere, K. (2014). The application of citation-based performance classes to the disciplinary and multidisciplinary assessment in national comparison and institutional research assessment. Scientometrics, 101(2), 939–952.

    Article  Google Scholar 

  • Glänzel, W., Thijs, B., & Debackere, K. (2018). Citation classes: A distribution-based approach to profiling citation impact for evaluative purposes. In W. Glänzel et al. (Eds.), Springer handbook of science and technology indicators. Heidelberg: Springer. to be published.

    Google Scholar 

  • Glänzel, W., Verbeek, A., du Plessis, M., van Looy, B., Magerman, T., Thijs, B., Schlemmer, B., Debackere, K., Veugelers, R. (2004), Stem cellsAnalysis of an emerging domain of scientific and technological endeavour. Accessible via: https://www.ecoom.be/sites/ecoom.be/files/downloads/stemcells_domain_study.pdf.

  • Glänzel, W., & Zhou, P. (2011). Publication activity, citation impact and bi-directional links between publications and patents in biotechnology. Scientometrics, 86(2), 505–525.

    Article  Google Scholar 

  • Lehmann, F. (2013). Realität und imagination. Photographie in W. G. Sebalds Austerlitz und Michelangelo Antonionis blow up. Bamberg: University of Bamberg Press.

    Google Scholar 

  • Schubert, A., Glänzel, W., & Braun, T. (1989). Scientometric datafiles. A comprehensive set of indicators on 2649 journals and 96 countries in all major fields and subfields 1981–1985. Scientometrics, 16(1–6), 3–478.

    Article  Google Scholar 

  • Thijs, B., Debackere, K., & Glänzel, W. (2017). Improved author profiling through the use of citation classes. Scientometrics, 111(2), 829–839.

    Article  Google Scholar 

  • Vincze, I. (1974). Mathematical Statistics, 4th edn. University Script, Eötvös University Budapest (in Hungarian).

  • Zitt, M., Ramanana-Rahary, S., & Bassecoulard, E. (2005). Relativity of citation performance and excellence measures: From cross-field to cross-scale effects of field-normalization. Scientometrics, 63(2), 373–401.

    Article  Google Scholar 

Download references

Acknowledgements

The present study is an extended version of an article presented at the 16th International Conference on Scientometrics and Informetrics, Wuhan (China), 16–20 October 2017 (Glänzel and Thijs 2017). In particular, the paper has been extended by adding a new option based on new data and results, which are presented in a new section on topic-based granularity. Figure 1 has been reproduced from Glänzel et al. (2009) with permission of the publisher. Table 1 presents data on subfield-based journal indicators shared with figures of Table 9 in Glänzel et al. (2018).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Wolfgang Glänzel.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Glänzel, W., Thijs, B. The role of baseline granularity for benchmarking citation impact. The case of CSS profiles. Scientometrics 116, 521–536 (2018). https://doi.org/10.1007/s11192-018-2747-1

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-018-2747-1

Keywords

Navigation