Skip to main content
SpringerLink
Log in

P-score: a reputation bibliographic index that complements citation counts

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

The notions of reputation and popularity in academia are critical for taking decisions on research grants, faculty position tenure, and research excellence awards. These notions are almost always associated with the publication track records of researchers. Thus, it is important to assess publication track records quantitatively. To quantify publication records, bibliographic indices are usually adopted and, among these, citation-based indices such as the H-index are frequently considered. In this paper we study the correlation between P-score, a publication record index and H-index, a very popular citation-based index, in the setting of conference ranking. While H-indices reflect the popularity of a given publication or researcher in academia, P-scores can reflect the reputation of a publication or researcher among its peers, considering a reference set of reputable researchers. Popularity and reputation are frequently considered to be equivalent properties in the formulation of citation based indices, however these properties are not identical. Indeed, we first show that H-indices and P-scores are correlated with a Kendall-Tau coefficient that exceeds 0.5. However, we also notice that they show important differences. Particularly, we identify publication venues with high H-indices and low P-scores, as well as venues with low H-indices and high P-scores. We provide interpretations for these findings and discuss how they can be used by research funding councils and committees to better support their funding decisions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://www.nap.edu/rdp

  2. According to Google Scholar, up to the end of 2017.

  3. http://www.capes.gov.br/

  4. While our data set is entirely composed of Computer Science conferences, nothing in our method is particular to this field. That is, our index is applicable to any field of knowledge.

  5. http://scholar.google.com.br

  6. http://dblp.uni-trier.de

  7. http://academic.microsoft.com

  8. http://academic.microsoft.com/authors

References

  • Babić, D., Kutlača, Đ., Živković, L., Štrbac, D., & Semenčenko, D. (2016). Evaluation of the quality of scientific performance of the selected countries of southeast europe. Scientometrics, 106(1), 405–434.

    Article  Google Scholar 

  • Baeza-Yates, R., & Ribeiro-Neto, B. (2011). Modern information retrieval: The concepts and technology behind search (2nd ed.). Boston: Addison-Wesley Publishing Company.

    Google Scholar 

  • Balaban, A. T. (2012). Positive and negative aspects of citation indices and journal impact factors. Scientometrics, 92(2), 241–247.

    Article  Google Scholar 

  • Bar-Ilan, J. (2008). Which h-index?—A comparison of WoS, Scopus and Google Scholar. Scientometrics, 74(2), 257–271.

    Article  Google Scholar 

  • Benevenuto, F., Laender, A. H., & Alves, B. L. (2016). The H-index paradox: Your coauthors have a higher H-index than you do. Scientometrics, 106(1), 469–474.

    Article  Google Scholar 

  • Bollen, J., Rodriquez, M. A., & Van de Sompel, H. (2006). Journal status. Scientometrics, 69(3), 669–687.

    Article  Google Scholar 

  • Bornmann, L., & Daniel, H. D. (2005). Does the h-index for ranking of scientists really work? Scientometrics, 65(3), 391–392.

    Article  Google Scholar 

  • Bornmann, L., & Marx, W. (2011). The h-index as a research performance indicator. European Science Editing, 37(3), 77–80.

    Google Scholar 

  • Braun, T., Glänzel, W., & Schubert, A. (2006). A Hirsch-type index for journals. Scientometrics, 69(1), 169–173.

    Article  Google Scholar 

  • Brin, S., & Page, L. (1998). The anatomy of a large scale hypertextual Web search engine. Computer Networks and ISDN Systems, 30(1–7), 107–117.

    Article  Google Scholar 

  • Chen, J., & Konstan, J. A. (2010). Conference paper selectivity and impact. Communications of the ACM, 53(6), 79–83.

    Article  Google Scholar 

  • Ding, Y., & Cronin, B. (2011). Popular and/or prestigious? Measures of scholarly esteem. Information Processing and Management, 47(1), 80–96.

    Article  Google Scholar 

  • Egghe, L. (2006). Theory and practise of the g-index. Scientometrics, 69(1), 131–152.

    Article  MathSciNet  Google Scholar 

  • Egghe, L. (2008). The influence of transformations on the h-index and the g-index. Journal of the American Society for Information Science and Technology, 59(8), 1304–1312.

    Article  Google Scholar 

  • Garfield, E. (1955). Citation indexes for science. Science, 122(3159), 108–111.

    Article  Google Scholar 

  • Gonçalves, G. D., Figueiredo, F., Almeida, J. M., & Gonçalves, M. A. (2014). Characterizing scholar popularity: A case study in the computer science research community. In Proceedings of the 14th ACM/IEEE-CS joint conference on digital libraries (pp. 57–66).

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102(46), 16569–16572.

    Article  Google Scholar 

  • Hirsch, J. E. (2007). Does the h index have predictive power? Proceedings of the National Academy of Sciences, 104(49), 19193–19198.

    Article  Google Scholar 

  • Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4), 422–446.

    Article  Google Scholar 

  • Kellner, A. W. A., & Ponciano, L. C. M. O. (2008). H-index in the Brazilian academy of sciences—Comments and concerns. Anais da Academia Brasileira de Ciências, 80(4), 771–781.

    Article  Google Scholar 

  • Kendall, M. G. (1955). Rank correlation methods (2nd ed.). New York: Hafner Publishing Company.

    MATH  Google Scholar 

  • Langville, A. N., & Meyer, C. D. (2006). Google’s pagerank and beyond: The science of search engine rankings. Princeton: Princeton University Press.

    Book  Google Scholar 

  • Lee, D. H. (2019). Predictive power of conference-related factors on citation rates of conference papers. Scientometrics, 118(1), 281–304. https://doi.org/10.1007/s11192-018-2943-z.

    Article  Google Scholar 

  • Leydesdorff, L. (2009). How are new citation-based journal indicators adding to the bibliometric toolbox? Journal of the American Society for Information Science and Technology, 60(7), 1327–1336.

    Article  Google Scholar 

  • Leydesdorff, L., Zhou, P., & Bornmann, L. (2013). How can journal impact factors be normalized across fields of science? An assessment in terms of percentile ranks and fractional counts. Journal of the American Society for Information Science and Technology, 64(1), 96–107.

    Article  Google Scholar 

  • Martins, W. S., Gonçalves, M. A., Laender, A. H., & Pappa, G. L. (2009). Learning to assess the quality of scientific conferences: A case study in computer science. In Proceedings of the 9th ACM/IEEE-CS joint conference on digital libraries (pp. 193–202).

  • Meyer, C. (1989). Stochastic complementation, uncoupling Markov chains, and the theory of nearly reducible systems. SIAM Review, 31(2), 240–272.

    Article  MathSciNet  Google Scholar 

  • Nane, T. (2015). Time to first citation estimation in the presence of additional information. In Proceedings of the 15th international society of scientometrics and informetrics conference (pp. 249–260).

  • Nature (2016). Time to remodel the journal impact factor. Nature  535(7613), 466.

  • Nelakuditi, S., Gray, C., & Choudhury, R. R. (2011). Snap judgement of publication quality: How to convince a dean that you are a good researcher. ACM SIGMOBILE Mobile Computing and Communications Review, 15(2), 20–23.

    Article  Google Scholar 

  • Newman, M. (2010). Networks: An introduction. Oxford: Oxford University Press.

    Book  Google Scholar 

  • Pinski, G., & Narin, F. (1976). Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics. Information Processing and Management, 12(5), 297–312.

    Article  Google Scholar 

  • Piwowar, H. A. (2013). Value all research products. Nature, 493(7431), 159.

    Article  Google Scholar 

  • Ribas, S., Ribeiro-Neto, B., de Souza e Silva, E., Ueda, A. H., & Ziviani, N. (2015a). Using reference groups to assess academic productivity in computer science. In Proceedings of the 24th international conference on world wide web, WWW ’15 companion (pp. 603–608). New York, NY, USA: ACM.

  • Ribas, S., Ribeiro-Neto, B., Santos, R. L., de Souza e Silva, E., Ueda, A., & Ziviani, N. (2015b). Random walks on the reputation graph. In Proceedings of the 2015 international conference on the theory of information retrieval, ICTIR ’15 (pp. 181–190). New York, NY, USA: ACM.

  • Riikonen, P., & Vihinen, M. (2008). National research contributions: A case study on Finnish biomedical research. Scientometrics, 77(2), 207–222.

    Article  Google Scholar 

  • Rossner, M., Van Epps, H., & Hill, E. (2007). Show me the data. The Journal of Cell Biology, 179(6), 1091–1092.

    Article  Google Scholar 

  • Saha, S., Saint, S., & Christakis, D. A. (2003). Impact factor: A valid measure of journal quality? Journal of the Medical Library Association, 91(1), 42–46.

    Google Scholar 

  • Sun, Y., & Giles, C. L. (2007). Popularity weighted ranking for academic digital libraries. In Proceedings of the 29th European conference on IR research, ECIR’07 (pp. 605–612). Berlin: Springer.

  • Wainer, J., Eckmann, M., Goldenstein, S., & Rocha, A. (2013). How productivity and impact differ across computer science subareas. Communications of the ACM, 56(8), 67–73.

    Article  Google Scholar 

  • Waltman, L., & Eck, N. (2013). Source normalized indicators of citation impact: An overview of different approaches and an empirical comparison. Scientometrics, 96(3), 699–716.

    Article  Google Scholar 

  • Wendl, M. C. (2007). H-index: However ranked, citations need context. Nature, 449(7161), 403.

    Article  Google Scholar 

  • Yan, E., Ding, Y., & Sugimoto, C. R. (2011). P-rank: An indicator measuring prestige in heterogeneous scholarly networks. Journal of the American Society for Information Science and Technology, 62(3), 467–477.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departamento de Ciência da Computação Sala 4304, Universidade Federal de Minas Gerais, Av. Pres. Antônio Carlos, 6627 - Pampulha, Belo Horizonte, MG, 31270-901, Brazil

    João Mateus de Freitas Veneroso, Marlon Dias, Alberto Ueda, Sabir Ribas, Berthier Ribeiro-Neto & Nivio Ziviani

  2. Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil

    Edmundo de Souza e Silva

Authors
  1. João Mateus de Freitas Veneroso
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marlon Dias
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alberto Ueda
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sabir Ribas
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Berthier Ribeiro-Neto
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Nivio Ziviani
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Edmundo de Souza e Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to João Mateus de Freitas Veneroso.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Freitas Veneroso, J.M., Dias, M., Ueda, A. et al. P-score: a reputation bibliographic index that complements citation counts. Scientometrics 121, 1269–1291 (2019). https://doi.org/10.1007/s11192-019-03247-0

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-019-03247-0

Keywords

Search

Navigation