Skip to main content
SpringerLink
Log in

Citation-based criteria of the significance of the research activity of scientific teams

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

The objective of this research is elaborating new criteria for evaluating the significance of the research results achieved by scientific teams. It is known, that the h-index (Hirsch index) is used to evaluate scientific organizations, as well as individual scientific workers. On the one hand, such a scientometric indicator as the “h-index of a scientific organization” reflects the organization’s scientific potential objectively. On the other hand, it does not always adequately reflect the significance that the results of a scientific team’s research activity have for the scientific megaenvironment (scientific community). The i-index has an even greater disadvantage, being principally limited by the size of a scientific team, although h-index is also dependent on the number of publications. Not trying to diminish the significance of the traditional parameters for monitoring the research activity of scientific organizations, including the institutions of higher education, the authors stress the necessity of using not only the traditional indicators, but also other parameters reflecting the significance of a scientific team’s research results for the scientific community. It should also not be forgotten that a scientific team is a social system whose functioning is not limited to the “sum” of individual scientific workers’ activities. The authors suggest new criteria of significance of research activity of scientific teams, which are suitable for the specific usage, hence they (the indicators) should be used with great caution; it is most appropriate to use the authors’ criteria for analyzing the dynamics of the research activity of scientific teams (following the principle “Compare yourself with yesterday’s yourself”). The authors’ proposed citation-based indicators make it possible to evaluate the true significance of research activity of a scientific team for the scientific community; while defining and justifying the new criteria, the authors also took into consideration the actuality of such a problem as the struggle with the self-citation effect (in a wider context—the problem of struggling with the artificial “improvement” of the scientometric indicators). The methodological basis of the research is formed by the system, metasystem, probability statistic, synergetic, sociological and qualimetric approaches. The research methods are the analysis of the problem situation, the analysis of the scientific literature and the best practices of research activity management at the institutions of higher education (benchmarking), the cognitive, structural–functional and mathematical modelling, the methods of graph, set and relation theory, the methods of qualimetry (the theory of latent variables), the methods of probability theory and mathematical statistics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Andrich, D. (2001). Advanced social and educational measurement. Perth: Murdoch University.

    Google Scholar 

  • Balykhin, M. G., & Generalova, A. V. (2015). Kraudfanding – platform podderzhki nauchnykh razrabotok [Crowdfunding as a platform for supporting research and development]. Sociological Studies, 9, 57–61 (in Russian).

    Google Scholar 

  • Bayburtyan, M. A. (2014). Korporativnyy kontrolling: podkhody k traktovke, sushchnost i funktsii [Corporate controlling: Definition, substance and tasks]. Polythematic Online Scientific Journal of Kuban State Agrarian University, 96, 397–409 (in Russian).

    Google Scholar 

  • Christiansen, J. A. (2000). Building the innovative organization: Management systems that encourage innovation. New York: St. Martin’s Press.

    Book  Google Scholar 

  • Covey, S. R. (2016). Sem navykov vysokoeffektivnykh lyudey: moshchnye instrument razvitiya lichnosti [The 7 habits of highly effective people: Restoring the character ethic]. Translation from English, 11th ed., add. Moscow: Alpina Publisher (in Russian).

  • Dudina, V. I. (2015). Sotsiologicheskoe znaniye v kontekste razvitiya informatsionnykh tekhnologiy [Sociological knowledge in the context of information technologies development]. Sociological Studies, 6, 13–22 (in Russian).

    Google Scholar 

  • Eck, N. V., & Waltman, L. (2008). Generalizing the h- and g-indices. Journal of Informetrics, 2(4), 263–267.

    Article  Google Scholar 

  • Efendiev, A. G., Sorokin, P. S., & Balabanova, E. S. (2015). Osnovnye problem razvitiya rossiyskikh i zarubezhnykh sotsiologicheskikh zhurnalov [The main problems of development of Russian and foreign sociological journals]. Sociological Studies, 9, 36–49 (in Russian).

    Google Scholar 

  • Egghe, L. (2008). Mathematical theory of the h- and g-indices. Journal of Informetrics., 2(4), 263–271.

    Article  Google Scholar 

  • Franceschini, F., Maisano, D., Perotti, A., & Proto, A. (2010). Analysis of the ch-index: An indicator to evaluate the diffusion of scientific research output by citers. Scientometrics, 85, 203–217.

    Article  Google Scholar 

  • Gavrilova, E. V., Ushakov, D. V., & Yurevich, A. V. (2015). Translyatsiya nauchnogo opyta i lichnostnoye znanie [Translation of scientific experience and tacit knowledge]. Sociological Studies, 9, 28–35 (in Russian).

    Google Scholar 

  • Geidarov, P Sh. (2015). Ballovaya sistema otsenki nauchnykh trudov i elektronnyy nauchnyy seminar [The scoring system evaluation of academic works and electronic academic seminar]. Sociological Studies, 4, 162–167 (in Russian).

    Google Scholar 

  • Guns, R., & Rousseau, R. (2009). Real and rational variants of the h-index and the g-index. Journal of Infometrics, 3(11), 64–71.

    Article  Google Scholar 

  • Guseltseva, M. S. (2014). Sistematizatsiya psikhologicheskogo znaniya posredstvom metateoreticheskogo analiza [Systematization of psychological knowledge by means of metatheoretical analysis]. Vopsosy psikhologii, 1, 156–159 (in Russian).

    Google Scholar 

  • Hirsch, J. E. (2005). An index to quantity an individual’s scientific research output. Proceedings of the National Academy of Sciences of the USA, 102(46), 16569–16572.

    Article  MATH  Google Scholar 

  • Holland, J. H. (1994). Adaptation in natural and artificial systems. An introductory analysis with application to biology, control, and artificial intelligence. London: Bradford Book.

    Google Scholar 

  • Jonash, R. S., & Sommerlatte, T. (1994). The innovation premium: How next generation companies are achieving peak performance and profitability. Cambridge: Massachusetts.

    Google Scholar 

  • Kincharova, A. V., & Sokolov, M. M. (2015). Issledovatelskie praktiki rossiyskikh sotsiologov [Research practices of Russian sociologists]. Sociological Studies, 6, 58–68 (in Russian).

    Google Scholar 

  • Kovshov, D. K., & Kovshova, E. S. (2015). Postroenie sotsialno-psikhologicheskoy karty kollektiva po dannym sociometricheskogo issledovaniya [Creating a social-psychological map of a team according to the data of sociometric research]. Sociological Studies, 10, 195–209 (in Russian).

    Google Scholar 

  • Koza, J. R. (1992). Genetic programming. Cambridge: The MIT Press.

    MATH  Google Scholar 

  • Kravchenko, S. A., & Salygin, V. I. (2015). Novyy sintez nauchnogo znaniya: stanovlenie mezhdistsiplinarnoy nauki [A new synthesis of scientific knowledge: The making of interdisciplinary science]. Sociological Studies, 10, 22–30 (in Russian).

    Google Scholar 

  • Lazarev, V. S., & Eliseeva, I. A. (2015). Issledovanie pedagogicheskogo kollektiva kak subyekta innovatsionnoy deyatelnosti [Research of pedagogical team as a subject of innovation activity]. Voprosy psikhologii, 1, 87–97 (in Russian).

    Google Scholar 

  • Lebedeva, I. P. (2015). Myagkie modeli kak forma matematizatsii sotsiologicheskogo znaniya [Soft models as a form of mathematizing sociological knowledge]. Sociological Studies, 1, 79–84 (in Russian).

    Google Scholar 

  • Lehmann, S., Jackson, A. D., & Lautrup, B. E. (2006). Measures for measures. Nature, 444(21), 1003–1004.

    Article  Google Scholar 

  • Loyko, V. I., Romanov, D. A., Kushnir, N. V., & Kushnir, A. V. (2015a). Diagnostika effektivnosti obrazovatelnykh sred (na primere kafedr i fakultetov) [Diagnosis of effective educational environment (on the example of departments and faculties)]. Polythematic Online Scientific Journal of Kuban State Agrarian University, 113, 1354–1378 (in Russian).

    Google Scholar 

  • Loyko, V. I., Romanov, D. A., & Popova, O. B. (2015b). Sovremennye metody otsenki znachimosti nauchnykh zhurnalov [Modern methods of scientific journals impact evaluation]. Polythematic Online Scientific Journal of Kuban State Agrarian University, 110(06), 836–852 (in Russian).

    Google Scholar 

  • Lutsenko, E. V. (2015). Hirschamaniya pri otsenke rezultatov nauchnoy deyatelnosti, eyo negativnye posledstviya i popytka ikh preodoleniya s primeneniyem mnogokriterialnogo podkhoda i teorii informatsii [Hirsch-mania when evaluating the results of scientific activity, its negative effects and an attempt to overcome them with the use of a multi-criteria approach and information theory]. Polythematic Online Scientific Journal of Kuban State Agrarian University, 108(04), 1–29 (in Russian).

    Google Scholar 

  • Maslak, A. A. (2006). Izmerenie latentnykh peremennykh v sotsialno-ekonomicheskikh sistemakh [Measuring latent variables in social-economic systems].Slavyansk-on-Kuban, SSPI.

  • Mukhin, V. V., & Orlov, A. I. (2014). O kontrollinge nauchnoy deyatelnosti [About controlling of scientific activity]. Polythematic Online Scientific Journal of Kuban State Agrarian University, 100, 1222–1237 (in Russian).

    Google Scholar 

  • Petkov, V. A., & Romanov, D. A. (2015). Metod formirovaniya pokazateley monitoring effektovnosti funktsionirovaniya sotsialnykh system [The method of formation of monitoring parameters of social systems’ effectiveness]. Society: Sociology, Psychology, Pedagogics, 5, 3–10 (in Russian).

    Google Scholar 

  • Popova, O. B., Popov, B. K., Karandei, V Yu., Romanov, D. A., Kobzeva, S. A., & Evseeva, M. A. (2015a). New methods and evaluation criteria of research efficiency. Mediterranean Journal of Social Sciences, 6(6), 212–217.

    Google Scholar 

  • Popova, O., Popov, B., & Karandey, V. (2015b). Intelligence amplification in distance learning through the binary tree of question-answer system. Procedia - Social and Behavioral Sciences, 214, 75–85. http://www.sciencedirect.com/science/article/pii/S1877042815059522.

  • Popova, O., Popov, B., Karandey, V., & Evseeva, M. (2015c). Intelligence amplification via language of choice description as a mathematical object (binary tree of question-answer system). Procedia - Social and Behavioral Sciences. 214, 897–905. http://www.sciencedirect.com/science/article/pii/S1877042815061030.

  • Popova, O. B., Romanov, D. A., Loyko, V. I., & Evseeva, M. A. (2017). Metodicheskaya znachimost rezultatov issledovatelskoy deyatelnosti [Methodical significance of activities research results]. Sociological Studies, 3, 79–87 (in Russian).

    Google Scholar 

  • Pshunetlev, A. A. (2014). Regionalnye aspekty vliyaniya nauchno-tekhnicheskogo progressa [Regional aspects of scientific-technical progress]. Polythematic Online Scientific Journal of Kuban State Agrarian University, 99, 1454–1463 (in Russian).

    Google Scholar 

  • Romanov, D. A., Popova, O. B., & Nosova, Yu S. (2015). Sovremennye metody otsenki produktivnosti issledovatelskoy deyatelnosti [Modern methods of evaluating the productivity of the research activity]. Polythematic Online Scientific Journal of Kuban State Agrarian University, 108(04), 725–739 (in Russian).

    Google Scholar 

  • Shevchenko, P. V. (2015). Modeli upravleniya obyedinennymi obrazovatelnymi organizatsiyami Moskvy [Models of administering unified education organizations in Moscow]. Sociological Studies, 2, 134–138 (in Russian).

    Google Scholar 

  • Tolstova, Yu N, & Voronina, N. D. (2015). O neobkhodimosti rasshireniya ponyatyiya sotsiologicheskogo izmereniya [On the necessity of broadening the notion of sociological measurement]. Sociological Studies, 7, 67–77 (in Russian).

    Google Scholar 

  • Tsyganov, A. V. (2013). Kratkoe opisanie naukometricheskikh pokazateley, osnovannykh na tsitiruemosti [Brief review of main scientometric indices based on citations]. Large systems management. Special issue 44: Scientometrics and expertise in science management, 248–278 (in Russian).

  • Yasvin, V. A. (2001). Obrazovatelnaya sreda: ot modelirovaniya k proektirovaniyu [Educational environment: From modelling to projecting]. Moscow: Smysl (in Russian).

    Google Scholar 

  • Yurkina, L. V. (2014). Integratsiya nauki i obrazovaniya: tendentsii i vozmozhnosti [Integration of science and education: Tendencies and opportunities]. Theory and Practice of Social Development, 2, 147–149 (in Russian).

    Google Scholar 

  • Zalibekova, D. Z. (2014). Aspekty povysheniya roli nauchnogo potentsiala Rossiyskoy Federatsii [The aspects of raising the role of the scientific potential of the Russian Federation]. Theory and Practice of Social Development, 3, 246–248 (in Russian).

    Google Scholar 

  • Zopounidis, C. (2002). Multi-group discrimination using multi-criteria analysis: Illustrations from the field of finance. European Journal of Operational Research, 139, 371–389.

    Article  MATH  Google Scholar 

  • Zyryanov, V. V., Temnova, L. V., & Sayko, E. A. (2014). Monitoring osnovnykh obrazovatelnykh programm po napravleniyu “Sotsiologiya”: problem i tendentsii [Monitoring basic education programs in “Sociology” specialization: Issues and trends]. Sociological Studies, 4, 139–143 (in Russian).

    Google Scholar 

Download references

Acknowledgements

The work was carried out with the financial assistance provided by the Russian Foundation for Humanities within the research Project No. 16-03-00382 within the theme “Monitoring the research activity of educational institutions in the conditions of information society” of 18.02.2016.

Author information

Authors and Affiliations

  1. Kuban State Technological University, 2 Moskovskaya Str., Krasnodar, Russia

    Olga Popova, Dmitry Romanov, Alexander Drozdov & Alexander Gerashchenko

Authors
  1. Olga Popova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dmitry Romanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander Drozdov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alexander Gerashchenko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Olga Popova.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Popova, O., Romanov, D., Drozdov, A. et al. Citation-based criteria of the significance of the research activity of scientific teams. Scientometrics 112, 1179–1202 (2017). https://doi.org/10.1007/s11192-017-2427-6

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-017-2427-6

Keywords

Search

Navigation