Abstract
Purpose Researchers often analyze cancer registry data to assess for differences in survival among cancer treatments. However, the retrospective non-random design of these analyses raise questions about study validity. The purpose of this study was to determine the extent to which comparative effectiveness analyses using cancer registry data produces results concordant with randomized clinical trials.
Methods We identified 141 randomized clinical trials referenced in the National Comprehensive Cancer Network Clinical Practice Guidelines for 8 common solid tumor types. For each trial we identified subjects from the National Cancer Database (NCDB) matching the eligibility criteria of the randomized trial. With each trial we used three Cox regression models to determine hazard ratios (HRs) for overall survival including univariable, multivariable, and propensity score adjusted models. Multivariable and propensity score analyses controlled for potential confounders including demographic, comorbidity, clinical, treatment, and tumor-related variables. Each NCDB survival analysis was defined as discordant if the HR for the NCDB analysis fell outside the 95% confidence interval of the corresponding randomized trial.
Results NCDB analyses produced HRs for survival discordant with randomized trials in 62 (44%) univariable analyses, 43 (30%) multivariable analyses, and 51 (36%) propensity score models. NCDB analyses produced p-values discordant with randomized trials in 83 (59%) univariable analyses, 76 (54%) multivariable analyses, and 78 (55%) propensity score models. We did not identify any clinical trial characteristic associated with discordance between NCDB analyses and randomized trials including disease site, type of clinical intervention, or severity of cancer.
Conclusion Comparative effectiveness research with cancer registry data often produces survival outcomes discordant to randomized clinical data. These findings help provide context for providers interpreting observational comparative effectiveness research in oncology.
Competing Interest Statement
AK receives compensation for consulting from Sympto Health. JDM receives compensation for consulting from Boston Consulting Group. RRS receives compensation for consulting from Boston Consulting Group.
Funding Statement
NIH TL1 TR001443 (AK, PTC, RRS)
Author Declarations
All relevant ethical guidelines have been followed; any necessary IRB and/or ethics committee approvals have been obtained and details of the IRB/oversight body are included in the manuscript.
Yes
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
All data referred to in the manuscript is publicly available through the National Cancer Database (NCDB).