Skip to main content

Advertisement

SpringerLink
Log in

Characteristics of Drug Intervention Clinical Trials and Scientific Impact of the Trial Outcome: A Bibliometric Analysis Using the Relative Citation Ratio in Non-small Cell Lung Cancer from 2007 to 2016

  • Original Research
  • Published:
Therapeutic Innovation & Regulatory Science Aims and scope Submit manuscript

Abstract

Background

Although a large number of clinical trials have been conducted, the types of clinical trials that are scientifically influential, frequently utilized by society, and contribute to the progress of evidence-based medicine (EBM) have not been studied. Thus, we aimed to investigate the relationship between the characteristics of clinical trials and the scientific impact of the outcome in non-small cell lung cancer (NSCLC) by performing a bibliometric analysis using relative citation ratio (RCR), a newly developed bibliometric index by the National Institutes of Health (NIH).

Methods

Primary publications of drug intervention clinical trials for NSCLC between 2007 and 2016 were included in the study. The characteristics of clinical trials were compared among four RCR categories with 50 trials in each [LOW50, 50 NIH percentile (50NIH%ile), 95 NIH percentile (95NIH%ile), and TOP50], totaling to 200 trials.

Results

Median RCRs of LOW50, 50NIH%ile, 95NIH%ile, and TOP50 were 0.03, 1.00, 5.76, and 26.89, respectively. Publications of Phase 3, randomized, blinded, for-profit-company supported/sponsored, multi-center trials, and trials with a larger number of subjects were shown to have a higher scientific impact. Publications of clinical trials of newly developed molecular target drugs, including epidermal growth factor receptor-tyrosine kinase inhibitors, anaplastic lymphoma kinase inhibitors, and immune checkpoint inhibitors demonstrated a higher scientific impact than those of traditional chemotherapies.

Conclusion

Clinical trials designed to have a high evidence level would improve the scientific impact of the outcome, and novel interventions would be another factor to improve the clinical trials’ influence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1.
Figure 2.
Figure 3.
Figure 4.
Figure 5.
Figure 6.

Similar content being viewed by others

References

  1. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68:394–424.

    Google Scholar 

  2. Noone AM, Howlader N, Krapcho M, Miller D, Brest A, Yu M, et al. SEER Cancer Statistics Review, 1975–2017, National Cancer Institute. https://seer.cancer.gov/csr/1975_2017/results_merged/topic_survival.pdf. Accessed 4 May 2020.

  3. American Cancer Society. Key Statistics for Lung Cancer. https://www.cancer.org/cancer/lung-cancer/about/key-statistics.html. Accessed 7 Mar 2020.

  4. Iafolla MAJ, Juergens RA. Update on programmed death-1 and programmed death-ligand 1 inhibition in the treatment of advanced or metastatic non-small cell lung cancer. Front Oncol. 2017;7:67.

    Article  Google Scholar 

  5. NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines®). Non-Small Cell Lung Cancer. Version 3.2020—February 11, 2020. https://www.nccn.org/professionals/physician_gls/pdf/nscl.pdf. Accessed 7 Mar 2020.

  6. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312(7023):71–2.

    Article  CAS  Google Scholar 

  7. Burns PB, Rohrich RJ, Chung KC. The levels of evidence and their role in evidence-based medicine. Plast Reconstr Surg. 2011;128(1):305–10.

    Article  CAS  Google Scholar 

  8. Merlin T, Weston A, Tooher R. Extending an evidence hierarchy to include topics other than treatment: revising the Australian ‘levels of evidence. BMC Med Res Methodol. 2009;9:34.

    Article  Google Scholar 

  9. ClinicalTrials.gov. Trends, Charts, and Maps. https://clinicaltrials.gov/ct2/resources/trends. Accessed 7 Mar 2020.

  10. OCEBM Levels of Evidence Working Group. “The Oxford Levels of Evidence 2”. Oxford Centre for Evidence-Based Medicine. https://www.cebm.net/index.aspx?o=5653. Accessed 7 Mar 2020.

  11. Moed HF. New developments in the use of citation analysis in research evaluation. Arch Immunol Ther Exp. 2009;57(1):13–8.

    Article  Google Scholar 

  12. Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci USA. 2005;102(46):16569–72.

    Article  CAS  Google Scholar 

  13. Santangelo GM. Article-level assessment of influence and translation in biomedical research. Mol Biol Cell. 2017;28(11):1401–8.

    Article  CAS  Google Scholar 

  14. Editorial, Deciphering impact factors. Nat Neurosci. 2003;6:783.

  15. Hutchins BI, Yuan X, Anderson JM, Santangelo GM. Relative citation ratio (RCR): a new metric that uses citation rates to measure influence at the article level. PLoS Biol. 2016;14(9):e1002541.

    Article  Google Scholar 

  16. National Institutes of Health, Office of Portfolio Analysis. iCite. https://icite.od.nih.gov/. Accessed 6 May 2019.

  17. Rock CB, Prabhu AV, Fuller CD, Thomas CR Jr, Holliday EB. Evaluation of the relative citation ratio, a new National Institutes of Health-supported bibliometric measure of research productivity, among academic radiation oncologists. J Am Coll Radiol. 2018;15:469–74.

    Article  Google Scholar 

  18. Reddy V, Gupta A, White MD, Gupta R, Agarwal P, Prabhu AV. Assessment of the NIH-supported relative citation ratio as a measure of research productivity among 1687 academic neurological surgeons. J Neurosurg. 2020;31:1–8.

    Google Scholar 

  19. Surkis A, Spore S. The relative citation ratio: what is it and why should medical librarians care? J Med Libr Assoc. 2018;106(4):508–13.

    Article  Google Scholar 

  20. Rifkin WJ, Yang JH, DeMitchell-Rodriguez E, Kantar RS, Diaz-Siso JR, Rodriguez ED. Levels of evidence in plastic surgery research: a 10-year bibliometric analysis of 18,889 publications from 4 major journals. Aesthet Surg J. 2020;40(2):220–7.

    PubMed  Google Scholar 

  21. Wang Z, He X, Qiao H, Chen P. Global trends of organoid and organ-on-a-chip in the past decade: a bibliometric and comparative study. Tissue Eng Part A. 2020. https://doi.org/10.1089/ten.tea.2019.0251.

    Article  PubMed  Google Scholar 

  22. Cai X, Zhou C, Zhou L, Xu Q. A bibliometric analysis of IL-35 research from 2009 to 2018. PeerJ. 2019;7:e7992.

    Article  Google Scholar 

  23. Chen R, Desai NR, Ross JS, Zhang W, Chau KH, Wayda B, et al. Publication and reporting of clinical trial results: cross sectional analysis across academic medical centers. BMJ. 2016;352:i637.

    Article  Google Scholar 

  24. Flacco ME, Manzoli L, Boccia S, Capasso L, Aleksovska K, Rosso A, et al. Head-to-head randomized trials are mostly industry sponsored and almost always favor the industry sponsor. J Clin Epidemiol. 2015;68(7):811–20.

    Article  Google Scholar 

  25. Guyatt G, Cairns J, Churchill D, Cook D, Haynes B, Hirsh J, et al. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992;268(17):2420–2425.

    Article  Google Scholar 

  26. Canadian Task Force on the Periodic Health Examination. Task force report: the periodic health examination. Can Med Assoc J. 1979;121:1193.

    Google Scholar 

  27. Bellomo R, Warrillow SJ, Reade MC. Why we should be wary of single-center trials. Crit Care Med. 2009;37:3114–9.

    Article  Google Scholar 

  28. Bafeta A, Dechartres A, Trinquart L, Yavchitz A, Boutron I, Ravaud P. Impact of single centre status on estimates of intervention effects in trials with continuous outcomes: meta-epidemiological study. BMJ. 2012;344:e813.

    Article  Google Scholar 

  29. Chan JK, Ueda SM, Sugiyama VE, Stave CD, Shin JY, Monk BJ, et al. Analysis of phase II studies on targeted agents and subsequent phase III trials: what are the predictors for success? J Clin Oncol. 2008;26(9):1511–8.

    Article  Google Scholar 

  30. Djulbegovic B, Kumar A, Miladinovic B, Reljic T, Galeb S, Mhaskar A, et al. Treatment success in cancer: industry compared to publicly sponsored randomized controlled trials. PLoS ONE. 2013;8(3):e58711.

    Article  CAS  Google Scholar 

  31. Unger JM, Barlow WE, Ramsey SD, LeBlanc M, Blanke CD, Hershman DL. The scientific impact of positive and negative phase 3 cancer clinical trials. JAMA Oncol. 2016;2(7):875–81.

    Article  Google Scholar 

  32. Bates SE, Berry DA, Balasubramaniam S, Bailey S, LoRusso PM, Rubin EH. Advancing clinical trials to streamline drug development. Clin Cancer Res. 2015;21(20):4527–35.

    Article  CAS  Google Scholar 

  33. Laterre PF, François B. Strengths and limitations of industry vs. academic randomized controlled trials. Clin Microbiol Infect. 2015;21(10):906–9.

    Article  Google Scholar 

  34. Malik AT, Jain N, Yu E, Khan SN. The top 50 most-cited articles on cervical spondylotic myelopathy. World Neurosurg. 2018;116:e1168–80.

    Article  Google Scholar 

  35. Alan N, Cohen J, Ozpinar A, Agarwal N, Kanter AS, Okonkwo DO, et al. Top 50 most cited articles on primary tumors of the spine. J Clin Neurosci. 2017;42:19–27.

    Article  Google Scholar 

  36. Ankomah F, Ikpeze T, Mesfin A. The top 50 most-cited articles on thoracolumbar fractures. World Neurosurg. 2018;118:e699–706.

    Article  Google Scholar 

  37. Bala MM, Akl EA, Sun X, Bassler D, Mertz D, Mejza F, et al. Randomized trials published in higher vs. lower impact journals differ in design, conduct, and analysis. J Clin Epidemiol. 2013. https://doi.org/10.1016/j.jclinepi.2012.10.005.

    Article  PubMed  Google Scholar 

  38. Hakala A, Kimmelman J, Carlisle B, Freeman G, Fergusson D. Accessibility of trial reports for drugs stalling in development: a systematic assessment of registered trials. BMJ. 2015;350:h1116.

    Article  Google Scholar 

  39. Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ. 2012;344:d7292.

    Article  Google Scholar 

  40. Zarin DA, Tse T, Sheehan J. The proposed rule for U.S. clinical trial registration and results submission. N Engl J Med. 2015;372(2):174–80.

    Article  CAS  Google Scholar 

  41. IFPMA, EFPIA, JPMA & PhRMA Joint Position on the Publication of Clinical Trial Results in the Scientific Literature, 10 June 2010. https://www.ifpma.org/wp-content/uploads/2010/11/Joint-Position-on-Publication-of-CT-Results-in-Literature-Revised-Oct2017vF.pdf. Accessed 7 Mar 2020.

Download references

Acknowledgements

The author acknowledges Mamoru Narukawa (Kitasato University) and Masayuki Kaneko (Kitasato University) for their useful advice on this study. We would like to thank Editage (www.editage.com) for English language editing.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author information

Authors and Affiliations

  1. Department of Clinical Medicine (Pharmaceutical Medicine), Graduate School of Pharmaceutical Sciences, Kitasato University, 5-9-1 Shirokane, Minato-Ku, Tokyo, 108-8641, Japan

    Yutaka Noguchi MS, Masayuki Kaneko PhD & Mamoru Narukawa PhD

  2. DaiichiSankyo Co., Ltd, Tokyo, Japan

    Yutaka Noguchi MS

Authors
  1. Yutaka Noguchi MS
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Masayuki Kaneko PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mamoru Narukawa PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the discussions and data interpretation contained in this paper and provided input on the manuscript and approved its final version. YN conducted data collection and data analysis. MK contributed to the data analysis. MN contributed to manuscript writing and finalizing.

Corresponding author

Correspondence to Yutaka Noguchi MS.

Ethics declarations

Conflict of interests

None of the authors have any conflicts of interest that are directly relevant to this research. Yutaka Noguchi is an employee of Daiichi Sankyo Co., LTD.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 14 kb)

Supplementary material 2 (DOCX 19 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Noguchi, Y., Kaneko, M. & Narukawa, M. Characteristics of Drug Intervention Clinical Trials and Scientific Impact of the Trial Outcome: A Bibliometric Analysis Using the Relative Citation Ratio in Non-small Cell Lung Cancer from 2007 to 2016. Ther Innov Regul Sci 54, 1501–1511 (2020). https://doi.org/10.1007/s43441-020-00177-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s43441-020-00177-5

Keywords

Search

Navigation