Skip to main content

Advertisement

Log in

Eigenfactor score and alternative bibliometrics surpass the impact factor in a 2-years ahead annual-citation calculation: a linear mixed design model analysis of Radiology, Nuclear Medicine and Medical Imaging journals

  • COMPUTER APPLICATIONS
  • Published:
La radiologia medica Aims and scope Submit manuscript

Abstract

Introduction

Because we believe the journal selection before a manuscript submission deserves further investigation in each medical specialty, we aimed to evaluate the predictive ability of seven bibliometrics in the Radiology, Nuclear Medicine and Medical Imaging category of the Web of Knowledge to calculate total citations over a 7-year period.

Methods

A linear mixed effects design using random slopes and intercepts were performed on bibliometrics corresponding to 124 journals from 2007 to 2011, with their corresponding citations from 2009 to 2013, which appeared in the Journal Citations Report Science Edition.

Results

The Eigenfactor Score, Article Influence Score, Cited Half-life, 5-years impact factor and Number of Articles are significant predictors of 2-year-ahead total citations (p ≤ 0.010 for all variables). The impact factor and Immediacy Index are not significant predictors. There was a significant global effect size (R2 = 0.934; p < 0.001), which yielded a total variance of 93.4%.

Conclusions

Our findings support researchers’ decision to stop the misuse of IF alone to evaluate journals. Radiologists and other researchers should review journal’s bibliometrics for their decision-making during the manuscript submission phase. A re-ranking of journals using Eigenfactor Score, Article Influence Score, and Cited Half-life provides a better assessment of their significance and importance in particular disciplines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Sharma M, Sarin A, Gupta P, Sachdeva S, Desai AV (2014) Journal impact factor: its use, significance and limitations. World J Nucl Med 13(2):146. https://doi.org/10.4103/1450-1147.139151

    Article  PubMed  PubMed Central  Google Scholar 

  2. Kumar V, Upadhyay S, Medhi B (2009) Impact of the impact factor in biomedical research: its use and misuse. Singapore Med J 50(8):752–755

    PubMed  CAS  Google Scholar 

  3. Baethge C (2012) Impact factor—a useful tool, but not for all purposes. Deutsches Arzteblatt Int 109(15):267–269. https://doi.org/10.3238/arztebl.2012.0267

    Article  Google Scholar 

  4. Rawat S (2014) How is impact factor impacting our research? Biomed J. https://doi.org/10.4103/2319-4170.131388

    Article  PubMed  Google Scholar 

  5. Elliott DB (2014) The impact factor: a useful indicator of journal quality or fatally flawed? Ophthalmic Physiol Optics J Br Coll Ophthalmic Opt 34(1):4–7. https://doi.org/10.1111/opo.12107

    Article  Google Scholar 

  6. Callaway E (2016) Beat it, impact factor! Publishing elite turns against controversial metric. Nature 535(7611):210–211. https://doi.org/10.1038/nature.2016.20224

    Article  PubMed  CAS  Google Scholar 

  7. Jackson A (2010) The impact factor game: the rising impact factor of the British Journal of Radiology—a success story? Br J Radiol 83(986):93–98. https://doi.org/10.1259/bjr/18689409

    Article  PubMed  PubMed Central  Google Scholar 

  8. Ending the tyranny of the impact factor (2014) Nat Cell Biol 16(1):1. https://doi.org/10.1038/ncb2905

  9. Diamandis EP (2017) The Journal Impact Factor is under attack—use the CAPCI factor instead. BMC Med 15(1):9. https://doi.org/10.1186/s12916-016-0773-5

    Article  PubMed  PubMed Central  Google Scholar 

  10. Choudhri AF, Siddiqui A, Khan NR, Cohen HL (2015) Understanding bibliometric parameters and analysis. Radiographics 35(3):736–746. https://doi.org/10.1148/rg.2015140036

    Article  PubMed  Google Scholar 

  11. Sardanelli F, Sconfienza LM (2013) Declining impact factor of radiologic journals: a matter for debate. AJR Am J Roentgenol 201(3):W391–W393. https://doi.org/10.2214/AJR.12.10256

    Article  PubMed  Google Scholar 

  12. Shanta A, Pradhan AS, Sharma SD (2013) Impact factor of a scientific journal: is it a measure of quality of research? J Med Phys Assoc Med Phys India 38(4):155–157. https://doi.org/10.4103/0971-6203.121191

    Article  CAS  Google Scholar 

  13. Thomson_Reuters (2014) Web of knowledge. http://wokinfo.com. Accessed 19 May 19 2014

  14. Roldan-Valadez E, Rios C (2015) Alternative bibliometrics from impact factor improved the esteem of a journal in a 2-year-ahead annual-citation calculation: multivariate analysis of gastroenterology and hepatology journals. Eur J Gastroenterol Hepatol 27(2):115–122. https://doi.org/10.1097/MEG.0000000000000253

    Article  PubMed  Google Scholar 

  15. Diaz-Ruiz A, Orbe-Arteaga U, Rios C, Roldan-Valadez E (2018) Alternative bibliometrics from the web of knowledge surpasses the impact factor in a 2-year ahead annual citation calculation: linear mixed-design models’ analysis of neuroscience journals. Neurol India 66(1):96–104. https://doi.org/10.4103/0028-3886.222880

    Article  PubMed  Google Scholar 

  16. Garfield E (1996) How can impact factors be improved? BMJ 313(7054):411–413

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  17. Mathur VP, Sharma A (2009) Impact factor and other standardized measures of journal citation: a perspective. Indian J Dent Res Off Publ Indian Soc Dent Res 20(1):81–85

    Article  Google Scholar 

  18. Bergstrom CT, West JD (2008) Assessing citations with the Eigenfactor metrics. Neurology 71(23):1850–1851. https://doi.org/10.1212/01.wnl.0000338904.37585.66

    Article  PubMed  Google Scholar 

  19. Bergstrom CT, West JD, Wiseman MA (2008) The Eigenfactor metrics. J Neurosci Off J Soc Neurosci 28(45):11433–11434. https://doi.org/10.1523/JNEUROSCI.0003-08.2008

    Article  CAS  Google Scholar 

  20. Ascaso FJ (2011) Impact factor, eigenfactor and article influence. Archivos de la Sociedad Espanola de Oftalmologia 86(1):1–2. https://doi.org/10.1016/j.oftal.2010.12.005

    Article  PubMed  Google Scholar 

  21. Tabachnick BG, Fidell LS (2013) Multivariate normality. In: Tabachnick BG, Fidell LS (eds) Using multivariate statistics, 6th edn. p 253

  22. Peugh JL (2010) A practical guide to multilevel modeling. J Sch Psychol 48(1):85–112. https://doi.org/10.1016/j.jsp.2009.09.002

    Article  PubMed  Google Scholar 

  23. Shek DT, Ma CM (2011) Longitudinal data analyses using linear mixed models in SPSS: concepts, procedures and illustrations. Sci World J 11:42–76. https://doi.org/10.1100/tsw.2011.2

    Article  Google Scholar 

  24. Field A (2013) Multilevel Linear Models. In: Field A (ed) Discovering Statistics using SPSS, 4th edn. SAGE Publications Inc., London, pp 734–776

    Google Scholar 

  25. Pallant J (2011) Multiple regression. SPSS survival manual. Allen & Unwin, Crows Nest, pp 148–167

    Google Scholar 

  26. Cohen JW (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum Associates, Hillsdale

    Google Scholar 

  27. Singer JD (1998) Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. J Educ Behav Stat 23(4):323–355. https://doi.org/10.3102/10769986023004323

    Article  Google Scholar 

  28. Butler JS, Sebastian AS, Kaye ID, Wagner SC, Morrissey PB, Schroeder GD, Kepler CK, Vaccaro AR (2017) Understanding traditional research impact metrics. Clin Spine Surg 30(4):164–166. https://doi.org/10.1097/BSD.0000000000000530

    Article  PubMed  Google Scholar 

  29. Reider B (2017) Brace for impact. Am J Sports Med 45(10):2213–2216. https://doi.org/10.1177/0363546517721707

    Article  PubMed  Google Scholar 

  30. Gutierrez FR, Beall J, Forero DA (2015) Spurious alternative impact factors: the scale of the problem from an academic perspective. BioEssays 37(5):474–476. https://doi.org/10.1002/bies.201500011

    Article  PubMed  Google Scholar 

  31. Citrome L (2013) How we rate: is impact factor the most important measure? Int J Clin Pract 67(9):819–820. https://doi.org/10.1111/ijcp.12266

    Article  PubMed  CAS  Google Scholar 

  32. Zupanc GK (2014) Impact beyond the impact factor. J Compar Physiol A Neuroethol Sens Neural Behav Physiol 200(2):113–116. https://doi.org/10.1007/s00359-013-0863-1

    Article  Google Scholar 

  33. Kodumuri P, Ollivere B, Holley J, Moran CG (2014) The impact factor of a journal is a poor measure of the clinical relevance of its papers. Bone Joint J 96-B(3):414–419. https://doi.org/10.1302/0301-620x.96b3.32279

  34. Sillet A, Katsahian S, Range H, Czernichow S, Bouchard P (2012) The Eigenfactor Score in highly specific medical fields: the dental model. J Dent Res 91(4):329–333. https://doi.org/10.1177/0022034512437374

    Article  PubMed  CAS  Google Scholar 

  35. Rizkallah J, Sin DD (2010) Integrative approach to quality assessment of medical journals using impact factor, eigenfactor, and article influence scores. PLoS One 5(4):e10204. https://doi.org/10.1371/journal.pone.0010204

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  36. Oosthuizen JC, Fenton JE (2014) Alternatives to the impact factor. Surgeon 12(5):239–243. https://doi.org/10.1016/j.surge.2013.08.002

    Article  PubMed  CAS  Google Scholar 

  37. Yao L, Wei T, Zeng A, Fan Y, Di Z (2014) Ranking scientific publications: the effect of nonlinearity. Sci Rep 4:6663. https://doi.org/10.1038/srep06663

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  38. Fersht A (2009) The most influential journals: impact Factor and Eigenfactor. Proc Natl Acad Sci USA 106(17):6883–6884. https://doi.org/10.1073/pnas.0903307106

    Article  PubMed  PubMed Central  Google Scholar 

  39. Weale AR, Bailey M, Lear PA (2004) The level of non-citation of articles within a journal as a measure of quality: a comparison to the impact factor. BMC Med Res Methodol 4:14. https://doi.org/10.1186/1471-2288-4-14

    Article  PubMed  PubMed Central  Google Scholar 

  40. Casadevall A, Fang FC (2014) Causes for the persistence of impact factor mania. mBio 5(2):e00064–e00014. https://doi.org/10.1128/mbio.00064-14

  41. Chen C (2018) Opportunities and pitfalls in clinical proof-of-concept: principles and examples. Drug Discov Today. https://doi.org/10.1016/j.drudis.2018.01.045

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Ulises Orbe-Arteaga, MSc, was a research fellow at the MRI unit of Medica Sur Clinic and Foundation during 2014–2015. No IRB approval was required for this manuscript.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ernesto Roldan-Valadez.

Ethics declarations

Conflict of interest

The authors have no conflict of interests to disclose.

Ethical standards

This article does not contain any studies with human participants or animals performed by any of the authors.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 260 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Roldan-Valadez, E., Orbe-Arteaga, U. & Rios, C. Eigenfactor score and alternative bibliometrics surpass the impact factor in a 2-years ahead annual-citation calculation: a linear mixed design model analysis of Radiology, Nuclear Medicine and Medical Imaging journals. Radiol med 123, 524–534 (2018). https://doi.org/10.1007/s11547-018-0870-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11547-018-0870-y

Keywords

Navigation