Abstract
In this paper we study the effect of granularity on Characteristic Scores and Scales (CSS). Unlike the traditional indicators that are mostly based on means and quantiles, CSS require the reduction of the citation distributions collaboration of the underlying reference population to four states (classes) and thus higher a different level of granularity. While the question of the choice of granularity is at higher levels of aggregation usually not critical since countries and university have rather multidisciplinary profiles, at lower aggregation levels specialisation becomes more typical. Inappropriate granularity might not warrant the depiction of the publication profiles at these levels in a correct and adequate manner and thus not add accurate citation profiles either. In order to be able to process one complete annual volume of the Web of Science, we decided to calculate CSS thresholds and classes for two levels of granularity, namely sub-fields and WoS Subject Categories. With about 5% deviation, we did not find a real significance. However, we identified journals with similar impact measures but different citation profiles, independently of the granularity. Finally, we have pointed to the limitations in the choice of granularity—in terms of both too broad and too narrow subjects.
Similar content being viewed by others
References
Adams, J., Gurney, K., & Jackson, L. (2008). Calibrating the zoom—A test of Zitt’s hypothesis. Scientometrics, 75(1), 81–95.
Adams, J., & Testa, J. (2011). Thomson Reuters book citation index. In E. Noyons, P. Ngulube, J. Leta (Eds.), The 13th conference of the international society for scientometrics and informetrics (Vol. I, pp. 13–18). Durban, South Africa: ISSI, Leiden University and the University of Zululand.
Albarrán, P., & Ruiz-Castillo, J. (2011). References made and citations received by scientific articles. JASIST, 62(1), 40–49.
Bookstein, A. (1997). Informetric distributions. 3. Ambiguity and randomness. Journal of the American Society for Information Science, 48(1), 2–10.
Glänzel, W. (2007). Characteristic scores and scales. A bibliometric analysis of subject characteristics based on long-term citation observation. Journal of Informetrics, 1(1), 92–102.
Glänzel, W. (2009). The multi-dimensionality of journal impact. Scientometrics, 78(2), 355–374.
Glänzel, W. (2011). The application of characteristic scores and scales to the evaluation and ranking of scientific journals. Journal of Information Science, 37(1), 40–48.
Glänzel, W., Meyer, M., Schlemmer, B., du Plessis, M., Thijs, B., Magerman, T., Debackere, K., Veugelers, R. (2003), Nanotechnology—Analysis of an emerging domain of scientific and technologic endeavour. Accessible via: https://www.ecoom.be/sites/ecoom.be/files/downloads/nanotech_domain_study.pdf.
Glänzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(2), 171–193.
Glänzel, W., & Schubert, A. (1988). Characteristic scores and scales in assessing citation impact. Journal of Information Science, 14(2), 123–127.
Glänzel, W., & Schubert, A. (2003). A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics, 56(3), 357–367.
Glänzel, W., Schubert, A., Thijs, B., & Debackere, K. (2009). Subfield-specific normalized relative indicators and a new generation of relational charts: Methodological foundations illustrated on the assessment of institutional research performance. Scientometrics, 78(1), 165–188.
Glänzel, W., & Thijs, B. (2017). The granularity of disciplinary structures for benchmarking citation impact. The case of CSS profiles. In Proceedings of ISSI 2015—The 15th international conference on scientometrics and informetrics (pp. 1190–1200). Wuhan, China.
Glänzel, W., Thijs, B., & Debackere, K. (2014). The application of citation-based performance classes to the disciplinary and multidisciplinary assessment in national comparison and institutional research assessment. Scientometrics, 101(2), 939–952.
Glänzel, W., Thijs, B., & Debackere, K. (2018). Citation classes: A distribution-based approach to profiling citation impact for evaluative purposes. In W. Glänzel et al. (Eds.), Springer handbook of science and technology indicators. Heidelberg: Springer. to be published.
Glänzel, W., Verbeek, A., du Plessis, M., van Looy, B., Magerman, T., Thijs, B., Schlemmer, B., Debackere, K., Veugelers, R. (2004), Stem cells—Analysis of an emerging domain of scientific and technological endeavour. Accessible via: https://www.ecoom.be/sites/ecoom.be/files/downloads/stemcells_domain_study.pdf.
Glänzel, W., & Zhou, P. (2011). Publication activity, citation impact and bi-directional links between publications and patents in biotechnology. Scientometrics, 86(2), 505–525.
Lehmann, F. (2013). Realität und imagination. Photographie in W. G. Sebalds Austerlitz und Michelangelo Antonionis blow up. Bamberg: University of Bamberg Press.
Schubert, A., Glänzel, W., & Braun, T. (1989). Scientometric datafiles. A comprehensive set of indicators on 2649 journals and 96 countries in all major fields and subfields 1981–1985. Scientometrics, 16(1–6), 3–478.
Thijs, B., Debackere, K., & Glänzel, W. (2017). Improved author profiling through the use of citation classes. Scientometrics, 111(2), 829–839.
Vincze, I. (1974). Mathematical Statistics, 4th edn. University Script, Eötvös University Budapest (in Hungarian).
Zitt, M., Ramanana-Rahary, S., & Bassecoulard, E. (2005). Relativity of citation performance and excellence measures: From cross-field to cross-scale effects of field-normalization. Scientometrics, 63(2), 373–401.
Acknowledgements
The present study is an extended version of an article presented at the 16th International Conference on Scientometrics and Informetrics, Wuhan (China), 16–20 October 2017 (Glänzel and Thijs 2017). In particular, the paper has been extended by adding a new option based on new data and results, which are presented in a new section on topic-based granularity. Figure 1 has been reproduced from Glänzel et al. (2009) with permission of the publisher. Table 1 presents data on subfield-based journal indicators shared with figures of Table 9 in Glänzel et al. (2018).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Glänzel, W., Thijs, B. The role of baseline granularity for benchmarking citation impact. The case of CSS profiles. Scientometrics 116, 521–536 (2018). https://doi.org/10.1007/s11192-018-2747-1
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-018-2747-1