https://orcid.org/0000-0003-2411-7177
Figures
Abstract
Background
Increasing costs and complexity in clinical trials requires recruitment of more narrowly defined patient populations. However, recruitment for clinical trials remains a considerable challenge.
Aim
Our overall aim was to quantify recruitment performance in industry-sponsored phase III clinical trials conducted globally during 2008–2019 with primary aim to examine development of overall clinical trial measures (number of trials completed, number of participants enrolled, trial duration in months) and key recruitment metrics (recruitment rate, number of sites, number of patients enrolled per site).
Methods
The publicly available AACT database containing data on all trials registered at ClinicalTrials.gov since 2008 was used. The analysis was completed during three time periods from 2008–2019 of 4 years each.
Results and conclusion
Recruitment duration for industry-sponsored phase III clinical trials have increased significantly during the last 12 years from an average recruitment period of 13 months (IQR 7–23) in 2008–2011 to 18 months (IQR 11–28) in 2016–2019 (p = 0.0068). Further, phase III clinical trials have increased the number of registered sites per clinical trial by more than 30% during the last 12 years from a median number 43 sites (IQR 17–84) in 2012–2015 to 64 sites (IQR 30–118) in 2016–2019 (p = 0.025), and concurrently, the number of participants enrolled in clinical research has decreased significantly from 2012–2015 and 2016–2019 (p = 0.046). We believe that these findings indicate that recruitment for phase III clinical trials is less effective today compared to 12 years ago.
Citation: Brøgger-Mikkelsen M, Zibert JR, Andersen AD, Lassen U, Hædersdal M, Ali Z, et al. (2022) Changes in key recruitment performance metrics from 2008–2019 in industry-sponsored phase III clinical trials registered at ClinicalTrials.gov. PLoS ONE 17(7): e0271819. https://doi.org/10.1371/journal.pone.0271819
Editor: Sathishkumar V. E., Hanyang University, REPUBLIC OF KOREA
Received: April 23, 2022; Accepted: July 7, 2022; Published: July 26, 2022
Copyright: © 2022 Brøgger-Mikkelsen et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data are available. The link to all raw data is https://aact.ctti-clinicaltrials.org/connect.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The complexity and costs of clinical trials have increased dramatically over the past decade, especially in the area of new drug development [1]. Increasing prevalence of chronic diseases, and increasing needs for personalised medicine and advanced technologies are just some of the demands that the clinical trial industry is currently facing [2]. New investigational drugs that target chronic, difficult-to-treat and rare diseases require recruitment of a more narrowly defined patient subpopulation, increasing the scope of clinical trials and the burden to execute them [3]. However, recruitment of participants into clinical trials remains a considerable challenge. In approximately 80% of clinical trials, enrollment timelines are not met [2, 4] and for each day of trial delays, pharmaceutical companies stand to lose from $600,000 to $8 mio. per day [5].
Recruiting patients for clinical research is a difficult task requiring sponsors to dedicate significant resources. The recruitment process not only includes identifying and screening potential participants by eligibility criteria before enrolling patients in the trial. During the process, potential participants should be thoroughly informed about all aspects of the trial by educated and relevant personnel, ensuring full comprehension and voluntariness and obtaining informed consent for participation [5]. Unfortunately, 6–29% of clinical trials terminate due to insufficient recruitment [6–8].
When recruitment is slow and inefficient it may have serious scientific, financial and ethical consequences for both patients and the clinical trial stakeholders involved [9]. Not only will study delays result in higher costs of the clinical trial, lengthy trials also delay the availability of potentially beneficial new treatments to the public. Moreover, participants who are occupied in ongoing trials are rarely eligible for other clinical studies, diminishing the pool of patients available for clinical trials. The integrity and validity of clinical trials is highly dependent on adequate sample sizes, as the risk of type II error (e.g., false negative discoveries) increases when target sample size is not achieved [10]. Ultimately, trials may be terminated due to low data quality wasting considerable human and material resources without a compensatory gain in research knowledge.
From an operational standpoint, the goal is to complete clinical trials as quickly as possible without delays and with the highest data quality [11]. However, such goals require efficient and high-performing clinical trial sites. Unfortunately, low- and non-performing sites that only enroll a single subject or no subjects at all, remain a significant challenge for recruitment and trial efficiency. According to multiple sources, the overall industry average of low or non-performing sites is approximately 33% on any given trial at a cost of approximately $50,000 per low or non-performing site [12, 13].
Previous poor recruitment performance, busy site staff, lack of research experience within site staff and slow approval processes are just some of the factors that may influence the performance of clinical trial sites [14]. In the literature, few studies have assessed clinical trial site performance. In a study evaluating 105 industry-sponsored phase II-IV clinical trials, Lamberti et al. found that 22% of the clinical trial sites did not enrol any participants [15]. Another study, investigating recruitment performance of clinical trial sites on a regional level, found that recruitment performance was lower than forecasted in Europe and North America, whereas recruitment was 104% of predicted in Latin America despite ethics, regulatory and contractual delays [16]. These findings underline the complexity of site selection for clinical trials and the importance of rapid recruitment in order to meet enrollment timelines.
Phase III trials account for most of the resources spent during clinical testing with the top cost drivers of trial expenditures being clinical procedure costs (15–22% of total), administrative staff costs (11–29% of total), and site monitoring costs (9–14% total) [17, 18]. Phase III clinical trials are pivotal in proving long term safety and efficacy, however phase III trials are highly expensive with a median cost around $21 million for a single trial [19]. Consequently, successful recruitment in phase III clinical trials is particularly important. In this study, we investigated changes in key recruitment metrics and overall clinical trial development measures in industry-sponsored phase III clinical trials with a drug as an intervention conducted globally during 2008–2019.
Methods
Our primary aim was to investigate whether there has been a significant change in key recruitment performance metrics in industry-sponsored phase III clinical trials with a drug as an intervention conducted globally when comparing three time intervals during 2008–2019. Recruitment performance was evaluated by three key recruitment metrics: recruitment rate (number of participants enrolled per month of recruitment), the number of sites registered per clinical trial, and the number of participants enrolled per site. Our second aim was to examine clinical trial development measures (trial duration, number of participants, number of clinical trials) over time from 2008–2019, and to examine the distribution of these trials by geographic location of trial sites, number of geographic regions registered and therapeutic area.
Study design and data extraction
We analysed all industry-sponsored phase III clinical trials registered in the clinical trials database ClinicalTrials.gov as completed during 2008–2019. Data were extracted on the 25th of January 2021 from a cloud-based access through R-studio to the Aggregate Analysis of ClinicalTrials.gov database (AACT), which is a publicly available database that contains all information about every study registered in ClinicalTrials.gov [20]. We identified all completed interventional trials that were classified as phase III or phase II/III, and were funded by an industry-sponsor. In September 2007, the Food and Drug Administration (FDA) Amendments Act was signed into law. All clinical trials of FDA-regulated drugs or biological products that are started or ongoing after 26th December 2007 must be registered in ClinicalTrials.gov (except phase I trials) [21]. Therefore, we included all trials completed from 2008. To investigate changes in key recruitment metrics and clinical trial development measures over time, we compared trials completed during three time intervals of 4 years each; from 2008–2011, from 2012–2015, and from 2016–2019.
Classification of geographic region and therapeutic area
Geographic region was classified according to the United Nations’ classification that includes five main regions [22]. We included the subdivision of the Americas and Europe resulting in nine regions: Africa, Asia, Latin America and the Caribbean, Northern America, Oceania, Eastern Europe, Northern Europe, Southern Europe, and Western Europe. As illustrated by Fig 1, not all trials had registered data on the location of trial sites. This data was found within “Recruitment Details” in the AACT database, resulting in data on 1117 trials. For each of the 1117 trials, we mapped data on trial sites’ location by region. Trials were excluded if one of the following conditions were met: 1) no trial sites were registered, 2) a geographic location of trial sites including overall continents such as “Asia, Europe and Australia” making it impossible to map the location within the 9 geographical regions.
Therapeutic area was classified according to the International Statistical Classification of Diseases and Related Health Problems (ICD-10) [23]. In total, 12 therapeutic areas were included: Certain infectious diseases, Circulatory system, Digestive system, Endocrine and metabolic system, Genitourinary system, Mental disorders, Musculoskeletal system, Neoplasms, Nervous system, Respiratory system, Skin, and Other therapeutic areas. In ClinicalTrials.gov, the therapeutic area of a clinical trial is listed in a medical condition description field. For each trial, we manually reviewed the medical condition description.
Data analysis
All data analysis was conducted using R-language, and p<0.05 was considered statistically significant. As the data on overall clinical trial development and key recruitment metrics collected from the AACT database were non-parametric data, Kruskal-Wallis tests were used to compare overall differences of means between the three groups. Further, Wilcoxon tests were used for pairwise tests for significance across the three groups. For geographical data, the distribution of trial sites and the mean number of geographic regions included in a trial were compared across the three time periods using the parametric Anova test as data was normally distributed with no extreme outliers. Further, the distribution of therapeutic areas across the three time periods was compared using parametric Anova tests.
Results
As illustrated by the CONSORT flow diagram in Fig 1, 5672 clinical trials fulfilled the search criteria. Trials with invalid date registration, or invalid participant enrollment registration were excluded. For each of the 5672 trials we extracted data on the number of participants enrolled and trial duration in months. Out of the 5672 trials, 3652 trials had “drug” registered as intervention. For each of the 3652 trials we extracted data on the therapeutic area. Of these 3652 clinical trials, 1672 trials had valid data on the number of sites and 606 trials had valid data on the length of recruitment in months. Further, 1117 trials had data on geographic location of trial sites.
Overall clinical trial development measures from 2008–2019
Of the 5,672 trials, 1,965 trials were completed from 2008–2011, 2,032 trials were completed from 2012–2015, and 1,676 trials were completed from 2016–2019. We found a decrease in the number of industry-sponsored phase III clinical trials conducted globally during 2008–2019 (see appendices, Table 1). From 2008–2011, 1965 trials were completed, and from 2016–2019, 1,676 trials were completed, corresponding to a decrease of 15% over time. The highest number of trials (2,031) was completed during 2012–2015. Similarly, we found an overall decrease in the number of participants enrolled in clinical trials during 2008–2019, again with the highest number of enrolled participants (1,249,809) registered between 2012–2015. From 2016–2019, 1,156,515 participants were enrolled, corresponding to a decrease of 7% compared to 2012–2015. However, as shown in Fig 2A, this decrease was only significant between the two time intervals from 2012–2015 and 2016–2019 (p = 0.046). Overall, we found no significant difference in the number of participants enrolled during 2008–2019 (p = 0.1024). For trial duration in months, a significant increase was found between all three time intervals from 2008–2019 (p = <2.2e-16). As seen in Fig 2B, trial duration increased significantly pairwise between all three time intervals. The overall median trial duration in months increased from 22 (InterQuartile Range (IQR) 13–35) in 2008–2011, to 24 (IQR 14–40, p = 1.7e-05) in 2012–2015, to a median trial duration of 28 months (IQR 17–47, p = 9.3e-11) in 2016–2019.
Comparison of clinical trial development metrics across the three time periods from 2008–2019 (A: Comparison of the mean number of participants enrolled in clinical trials from 2008–2019. B: Comparison of mean trial duration in months from 2009–2019).
Over time development of number of trials, number of participants enrolled and trial duration in months are presented in the table together with corresponding p-values.
Clinical trial key recruitment metrics from 2008–2019
From the 3,652 industry-sponsored phase III clinical trials with “drug” as intervention, 606 clinical trials had valid registration on length of recruitment. We found a significant increase in recruitment duration in months from an average recruitment period of 13 months (IQR 7–23) in 2008–2011 to 18 months (IQR 11–28) in 2016–2019 (p = 0.0068) (see appendices, Table 2). As shown in Fig 3A, no change in median recruitment duration in months was found between 2008–2011 and 2012–2015, however a significant increase in length of recruitment was found from 2012–2015 to 2016–2019 (p = 0.0048). For recruitment rate, we found a non-significant decrease in the number of participants enrolled per month of recruitment from an average of 26 participants (IQR 10–60) in 2008–2011 to 20 participants (IQR 10–46) in 2012–2016 (p = 0.195). As illustrated in Fig 3B, no significant difference in recruitment rate was found pairwise across the three groups. From the 3652 phase III clinical trials with “drug” as intervention, 1672 clinical trials had registered valid data on the number of sites per clinical trial in ClinicalTrials.gov. As shown in Fig 3C, we found a significant increase in the median number of sites registered per clinical trial from 43 sites (IQR 17–84) in 2012–2015 to 64 sites (IQR 30–118) in 2016–2019 (p = 0.025). In 2008–2011 the median number of sites per clinical trial was 51 (IQR 24–93), and consequently, the increase in the average number of sites across all three time intervals was non-significant (p = 0.077).
Comparison of key recruitment metrics across the three time periods from 2008–2019 (A: Comparison of recruitment rate (mean, IQR), B: Comparison of recruitment duration in months (mean, IQR), C: Comparison of the number of sites (mean, IQR), D: Comparison of the number of participants enrolled per site (mean, IQR), E: Comparison of the number of participants enrolled per site per month (mean, IQR)).
Over time development of recruitment duration, recruitment rate, number of sites and site effectiveness are presented in the table together with corresponding p-values.
For the development of site effectiveness over time from 2008–2019, we investigated two metrics; 1) number of participants enrolled per site, and 2) the number of participants enrolled per site per month. As illustrated in Fig 3D, across all three time intervals from 2008–2019, we found a significant decrease in the overall median number of participants enrolled per site (p = 2.8e-06). In 2008–2011, sites enrolled 7 participants (IQR 5–11) on average, while this number decreased to an average of 6 participants (IQR 4–10) in 2016–2019. In 2012–2015, the median number of participants enrolled per site was also 7 (IQR 5–13). For the median number of participants enrolled per site per month, we found a non-significant decrease from 0.8 participants (IQR 0.3–1.9) in 2012–2015 to 0.4 (IQR 0.2–0.6) in 2016–2019. As shown in Fig 3E, no significant differences in the number of participants enrolled per site per month was found pairwise across all three groups.
Geographic region and therapeutic area
For the development of clinical trial site location over time during 2008–2019, we found a significant decrease in the number of trial sites located in North America (p = 0.03), as illustrated in Fig 4A. Further, we found a significant increase in the number of trial sites located in Northern Europe (p = 0.006), Western Europe (p = 0.001), Southern Europe (p = 6.03e-06), Eastern Europe (p = 3.02e-08), Asia/Pacific (p = 4.2e-15), and Oceania (p = 0.0007). We found non-siginicant increases across the three time periods for the remaining geographic regions, including Latin America and Africa (see appendices, Table 3). For the mean number of geographic regions included in a clinical trial, we found an overall significant increase (p = 7.66e-07) when comparing the three time periods from 2008–2019. As shown in Fig 5, a significant increase in the mean number of regions was also found between the two time periods from 2012–2019 (p = 0.0018).
Each clinical trial may have registered several geographical regions.
Fig 6 illustrates the distribution of therapeutic areas over time. We found overall significant decreases throughout the three time periods for trials examining digestive system (p = 0.0001), endocrine/metabolic system (p = 0.004), genitourinary system (p = 0.02), mental disorders (p = 2.2e-06), and nervous system (p = 0.01) (see appendices, Table 4). Overall significant increases were found for trials examining neoplasms (p = 1.03e-12) and skin (p = 8.3e-16). For infections, circulatory system and musculoskeletal system, we found no significant changes from 2008–2019.
Discussion
Our study shows a decline in several key recruitment performance metrics for industry-sponsored phase III clinical trials with a drug as an intervention conducted globally from 2008–2019. From the trials included in this study, we found that recruitment duration as well as overall clinical trial duration for phase III clinical trials have increased significantly during the last 12 years. Further, we found that industry-sponsored phase III clinical trials have increased the number of registered sites per clinical trial by more than 30% during the last 12 years, and concurrently, the number of participants enrolled in clinical research has decreased significantly. These findings indicate that recruitment for phase III clinical trials is less effective today compared to 12 years ago. The median number of participants enrolled per site in our study ranged from 7 (IQR 5–11) in 2008–2011 to 6 (IQR 4–10) in 2016–2019, reflecting the findings from The European Federation of Pharmaceutical Industries and Associations (EFPIA), who found that 8.8 participants were enrolled on average per active site in 931 industry-sponsored phase II-III clinical trials [24]. For the number of participants enrolled per month of recruitment (recruitment rate) and the number of participants enrolled per site (or per site per month), we found non-significant decreases.
The combination of increasing number of sites in clinical trials and decreasing participant enrollment could be due to more sites being low- or non-performing, as more sites are needed to comply with timelines and sample sizes. As previously mentioned, clinical trials are facing challenges recruiting more narrowly defined patient subpopulations, as new technologies and diagnostic tools have paved the way for diagnosing and treating chronic, difficult-to-treat and rare diseases [3, 25, 26]. This might be one explanation to the seemingly decrease of recruitment and site effectiveness found in this study, especially during the last 8 years. However, an increase in the number of sites per clinical trial could also be an expression of an increased globalisation. In this study, we found an overall significant increase in the mean number of regions included in clinical trials from 2008–2019. Further, our findings indicate a shift in location of clinical trial sites towards emerging regions such as Eastern Europe and Asia/Pacific with a concurrent decrease in the number of trial sites located in North America from 2008–2019. This reflects the findings from several studies investigating the increasing globalisation of clinical trials [27–29], reporting that the number of clinical trials conducted outside of the United States and Europe have doubled over the past 10 years [30]. There are many factors influencing the selection of clinical trial sites, such as operational costs and the regulatory landscape in different markets. However, the opportunity for increased exposure towards large, untested research populations could also be an important factor contributing to the increased globalisation [27].
Recruitment
For clinical trial recruitment performance, increasing complexity in clinical trials [26] and patients’ willingness to participate in clinical research [31, 32] are two important factors that seem to reinforce each other.
Designing protocols that balance measuring efficacy and safety with minimized patient participation burden and advanced levels of individually targeted therapies, is a difficult task [33]. In 2017, Getz. et al found an increasing trend in clinical trial protocol complexity across phase I, II and III clinical trials based on 9373 protocols of which 76% were protocols provided by large companies [3]. For phase III trials, they found a relative growth of 70% in total procedures carried out as well as an increase of 25% in planned study visits. Further, Getz KA et al. found a growth of 34% in mean cost per visit over the last 10 years. The mean total cost per study volunteer per study increased substantially even though the cost for many study procedures, such as blood tests, has decreased during the past decade. These findings could pose a threat to the effectiveness of patient recruitment and retention and could lead to increasing numbers of protocol amendments, adding to recruitment and thereby timeline delays. As the number of procedures increases, the work required to support these increases similarly [34]. Furthermore, demanding protocols could also negatively influence clinical trial site performance, increasing the level of stress for trial site personnel [35]. High complexity and demanding trial site tasks may impact the performance of clinical trial sites, where low- and non-performing sites are already a challenge for the clinical trial industry [11]. In an analysis from 2018 conducted by the Boston Consulting Group and KMR Group, data from more than 75000 sites were used to examine site performance and underlying drivers of success. The analysis concluded that companies that reduce the number of non-enrolling sites by approximately 50% save around $10–13 million on an average phase III trial [13].
For trial participants, the willingness to enter and complete clinical trials is negatively correlated to a high number of study visits and performed procedures as well as more comprehensive and advanced consent forms [32, 36]. Patients participate in clinical trials for a number of reasons, ranging from receiving new medication not yet available for the general population, to making a contribution that could help another person [37]. Patient-centricity is an accelerating trend in clinical trials, and transparent patient reimbursement is an important part of this. The increasing trial complexity with clinical trials often being carried out at specialist centers, requiring patients to travel longer and perhaps also more often, has put a new focus on patient reimbursement [38]. Compensating patients in clinical trials may be seen as an award for a risk taken by the patient, however reimbursement rates range from approximately 400$ to 2,000$ per patient in clinical trials, significantly impacting the costs of clinical research [39]. Increasing patient reimbursement rates may be necessary due to the increase in trial complexity, however it may also pose a threat to patient willingness to participate in clinical trials, nudging patients towards trials with higher levels of compensation.
Increasing cost of clinical research may have significant implications for public health, as it affects drug companies’ willingness to undertake clinical trials, which in turn limits patient access to novel treatments [17]. Armed with such knowledge, a growing number of pharmaceutical companies and contract research organizations (CRO’s) are working to optimize recruitment for clinical trials through optimizing study designs in order to improve feasibility and ease site and subject participation burden [40]. The declining productivity of drug development has intensified the interest in digital tools and other technologies that may improve trial efficiency [11]. In a systematic review and meta analysis from 2019, we found that targeting participants using online remedies was an effective tool for patient recruitment in clinical trials, as online recruitment was superior to traditional in-clinic recrutiment in regards to both time-efficiency and cost-effectiveness [41]. However, our study also concluded that for online recruitment of participants to be an effective tool, both effort and money should be invested in recruitment ads and campaigns as well as in training trial staff. With fewer trial sites, logistical complexity could be limited and effective recruitment, either through high-performing sites or through digital tools, are a key factor in minimizing recruitment delays. This will in turn impact cost-per-patient and time-to-market, increasing the sponsor’s return on investment, increasing value from the drugs patent lifespan and the societal benefit of bringing new therapies to patients in need [11].
Risk of bias
The results reported in this study rely on voluntary reporting from trial sponsors, and therefore could be biased. Further, not all clinical trials are registered in ClinicalTrials.gov and the registration compliance probably varies across countries. However, it is mandatory to register all phase III clinical trials of FDA-related drugs and biological products in ClinicalTrials.gov [21] and hence, we expect that the majority of industry-sponsored phase III clinical trials are registered. Moreover, since 2005, the International Committee of Medical Journal Editors (ICMJE) has required prospective registration of clinical trials in ClinicalTrials.gov or the WHO International Clinical Trials Registry Platform (ICTRP) as a precondition for publication [42]. The norms and rules for reporting trials in ClinicalTrials.gov change over time. The overall clinical trial metrics included in this study, such as the number of participants enrolled, is mandatory upon trial registration and reporting of results. However, for recruitment metrics such as recruitment duration and geographic location of trial sites, manual reporting is required for trial sponsors. Accordingly, the risk of bias mostly relates to non-mandatory metrics where manual reporting without clear guidelines is required. Further, the geographic distribution of trials could be biased due to unequal registration of trials in different geographic regions.
Study limitations
This study on changes in recruitment performance over time has several limitations. Notably, key recruitment metrics including recruitment duration in months and the number of sites per clinical trial are registered manually as a free text by trial sponsors under “recruitment details’’ in the AACT database. Hereby, far from every trial had data on recruitment and site metrics, reducing the scope of the dataset in this study. There is a risk that the sponsors are more likely to fill in the free text under “recruitment details’’ making the results less representative from a global perspective. For the assessment of site effectiveness, it was impossible to estimate the number of sites that did not enroll any participants as these data are not registered in ClinicalTrials.gov. Hence, no associations between geographic location of sites and site performance could be investigated in this study. Further, we only collected data from clinical trials that were completed within the three defined time periods and consequently, no data on recruitment from all ongoing and not yet completed clinical trials is included in this study.
Lastly, neoplasms are the largest therapeutic area in the last period from 2016–2019. In the field of cancer there has been a change in phase 3 trials due to targeted treatment and companion diagnostics. Phase 3 trials are more biomarker-driven which means that large patient groups no longer exist, and large phase 3 trials are impossible to complete. Even frequently occurring cancers are divided into many rare subtypes. This may possibly affect the outcome. Further, the results show a trend with an overall reduction in multicenter trials. However, we have not investigated the trends for all industry sponsored trials.
Conclusion
We assessed changes in recruitment performance over time in industry-sponsored phase III clinical trials and found a decline in key recruitment metrics. Recruitment duration have increased significantly during the last 12 years from an average recruitment period of 13 months in 2008–2011 to 18 months in 2016–2019. Further, the number of registered sites per clinical trial has increased by more than 30% during the last 12 years, and the number of participants enrolled has decreased significantly.
Clinical trials have become larger measured on the number of sites registered per clinical trials, and recruitment for clinical trials takes more time and less participants are enrolled, indicating that recruitment for phase III clinical trials is less effective today compared to 12 years ago.
For future research, more and deeper knowledge of which factors are strongly associated with unsuccessful recruitment is essential to identify suitable strategies to improve recruitment.
References
- 1. Oba K. Statistical challenges for central monitoring in clinical trials: a review. Int J Clin Oncol. 2016 Feb;21(1):28–37. pmid:26499195
- 2. Johnson O. An evidence-based approach to conducting clinical trial feasibility assessments. Clin Investig. 2015 May;5(5):491–9.
- 3. Getz KA, Campo RA. Trial watch: Trends in clinical trial design complexity. Nat Rev Drug Discov. 2017 May;16(5):307. pmid:28417986
- 4.
Recruitment Roles [Internet]. [cited 2021 Apr 28]. Available from: https://www.appliedclinicaltrialsonline.com/view/recruitment-roles
- 5. Chaudhari N, Ravi R, Gogtay NJ, Thatte UM. Recruitment and retention of the participants in clinical trials: Challenges and solutions. Perspect Clin Res. 2020 Apr;11(2):64–9. pmid:32670830
- 6. Schroen AT, Petroni GR, Wang H, Thielen MJ, Gray R, Benedetti J, et al. Achieving sufficient accrual to address the primary endpoint in phase III clinical trials from U.S. Cooperative Oncology Groups. Clin Cancer Res. 2012 Jan 1;18(1):256–62. pmid:21976533
- 7. Kasenda B, von Elm E, You J, Blümle A, Tomonaga Y, Saccilotto R, et al. Prevalence, characteristics, and publication of discontinued randomized trials. JAMA. 2014 Mar 12;311(10):1045–51. pmid:24618966
- 8. Carlisle B, Kimmelman J, Ramsay T, MacKinnon N. Unsuccessful trial accrual and human subjects protections: an empirical analysis of recently closed trials. Clin Trials. 2015 Feb;12(1):77–83. pmid:25475878
- 9. Roberts J, Waddy S, Kaufmann P. Recruitment and retention monitoring: facilitating the mission of the National Institute of Neurological Disorders and Stroke (NINDS). J Vasc Interv Neurol. 2012 Aug;5(supp):14–9. pmid:23230460
- 10. Thoma A, Farrokhyar F, McKnight L, Bhandari M. Practical tips for surgical research: how to optimize patient recruitment. Can J Surg. 2010 Jun;53(3):205–10. pmid:20507795
- 11. Yang E, O’Donovan C, Phillips J, Atkinson L, Ghosh K, Agrafiotis DK. Quantifying and visualizing site performance in clinical trials. Contemp Clin Trials Commun. 2018 Mar;9:108–14. pmid:29696232
- 12. Fogel DB. Factors associated with clinical trials that fail and opportunities for improving the likelihood of success: A review. Contemp Clin Trials Commun. 2018 Sep;11:156–64. pmid:30112460
- 13. Smith B, Martin L, Martin S, Denslow M, Hutchens M, Hawkins C, et al. What drives site performance in clinical trials? Nat Rev Drug Discov. 2018 Jun;17(6):389–90. pmid:29674700
- 14. Bruhn H, Treweek S, Duncan A, Shearer K, Cameron S, Campbell K, et al. Estimating Site Performance (ESP): can trial managers predict recruitment success at trial sites? An exploratory study. Trials. 2019 Apr 3;20(1):192. pmid:30944022
- 15. Lamberti MJ, Brothers C, Manak D, Getz K. Benchmarking the Study Initiation Process. Ther Innov Regul Sci. 2013 Jan;47(1):101–9. pmid:30227494
- 16. Berthon-Jones N, Courtney-Vega K, Donaldson A, Haskelberg H, Emery S, Puls R. Assessing site performance in the Altair study, a multinational clinical trial. Trials. 2015 Apr 8;16:138. pmid:25872747
- 17. Sertkaya A, Wong H-H, Jessup A, Beleche T. Key cost drivers of pharmaceutical clinical trials in the United States. Clin Trials. 2016 Apr;13(2):117–26. pmid:26908540
- 18.
Facts and figures 2014: The pharmaceutical industry and global health [Internet]. 2014 [cited 2021 Apr 28]. Available from: https://www.ifpma.org/resource-centre/facts-and-figures-2014-the-pharmaceutical-industry-and-global-health/
- 19. Mullard A. How much do phase III trials cost? Nat Rev Drug Discov. 2018 Oct 30;17(11):777. pmid:30374183
- 20.
AACT Database [Internet]. [cited 2021 Apr 28]. Available from: https://aact.ctti-clinicaltrials.org/
- 21.
FDAAA 801 and the Final Rule [Internet]. [cited 2021 Apr 28]. Available from: https://clinicaltrials.gov/ct2/manage-recs/fdaaa
- 22.
United Nations Statistics Division. Methodology [Internet]. [cited 2021 Apr 28]. Available from: https://unstats.un.org/unsd/methodology/m49/
- 23.
Classification of Diseases (ICD) [Internet]. [cited 2021 May 5]. Available from: https://www.who.int/standards/classifications/classification-of-diseases
- 24.
Upholding competitive clinical trial authorisation timelines ahead of trialogue agreement [Internet]. [cited 2021 May 5]. Available from: https://www.efpia.eu/news-events/the-efpia-view/statements-press-releases/131127-industry-researchers-and-patients-warn-of-the-importance-of-upholding-competitive-clinical-trial-authorisation-timelines-ahead-of-trialogue-agreement/
- 25. Dal-Ré R, Moher D, Gluud C, Treweek S, Demotes-Mainard J, Carné X. Disclosure of investigators’ recruitment performance in multicenter clinical trials: a further step for research transparency. PLoS Med. 2011 Dec;8(12):e1001149. pmid:22215987
- 26. Mentz RJ, Hernandez AF, Berdan LG, Rorick T, O’Brien EC, Ibarra JC, et al. Good Clinical Practice Guidance and Pragmatic Clinical Trials: Balancing the Best of Both Worlds. Circulation. 2016 Mar 1;133(9):872–80. pmid:26927005
- 27. Jeong S, Sohn M, Kim JH, Ko M, Seo H-W, Song Y-K, et al. Current globalization of drug interventional clinical trials: characteristics and associated factors, 2011–2013. Trials. 2017 Jun 21;18(1):288. pmid:28637515
- 28. Atal I, Trinquart L, Porcher R, Ravaud P. Differential Globalization of Industry- and Non-Industry-Sponsored Clinical Trials. PLoS One. 2015 Dec 14;10(12):e0145122. pmid:26658791
- 29. Qiao Y, Alexander GC, Moore TJ. Globalization of clinical trials: Variation in estimated regional costs of pivotal trials, 2015–2016. Clin Trials. 2019 Jun;16(3):329–33. pmid:30922113
- 30. Morse BL. On the Blog. Nurse Educ. 2021;46(2):81. pmid:33755383
- 31. Gayet-Ageron A, Rudaz S, Perneger T. Study design factors influencing patients’ willingness to participate in clinical research: a randomised vignette-based study. BMC Med Res Methodol. 2020 Apr 26;20(1):93. pmid:32336266
- 32. Madsen S, Holm S, Riis P. Ethical aspects of clinical trials: the attitudes of the public and out-patients. J Intern Med. 1999 Jun;245(6):571–9. pmid:10395186
- 33.
Trial Complexity Is Increasing Across Trial Planning and Management [Internet]. 2018 [cited 2021 May 26]. Available from: https://www.medidata.com/en/life-science-resources/medidata-blog/trial-complexity-increasing-across-trial-planning-and-management
- 34.
Conroy L. How protocol complexity is impacting patient recruitment [Internet]. [cited 2021 May 26]. Available from: https://www.antidote.me/blog/how-protocol-complexity-is-impacting-patient-recruitment
- 35.
Stergiopoulos S, Getz KA. Diagnosing protocol amendment experience to drive clinical trial performance [Internet]. 2017 [cited 2021 May 5]. Available from: https://www.pharmoutsourcing.com/Featured-Articles/333834-Diagnosing-Protocol-Amendment-Experience-to-Drive-Clinical-Trial-Performance/
- 36.
Tufts Center for the Study of Drug Development. Rising protocol design complexity is driving rapid growth in clinical trial data volume, according to Tufts Center for the Study of drug development [Internet]. Tufts Center for the Study of Drug Development. 2021 [cited 2021 May 5]. Available from: http://www.globenewswire.com/en/news-release/2021/01/12/2157143/0/en/Rising-Protocol-Design-Complexity-Is-Driving-Rapid-Growth-in-Clinical-Trial-Data-Volume-According-to-Tufts-Center-for-the-Study-of-Drug-Development.html
- 37.
Clinical research [Internet]. 2017 [cited 2021 May 20]. Available from: https://neabaptistclinic.com/clinical-trials/
- 38.
Weyman K. Clinical Trial Reimbursement for Patients: Current Challenges and future improvements [Internet]. [cited 2021 May 20]. Available from: https://www.anjusoftware.com/about/all-news/insights/clinical-trial-reimbursement-for-patients-current-challenges-and-future-improvements
- 39.
Kantor E. How to find and take part in paid clinical trials [Internet]. [cited 2021 May 20]. Available from: https://www.antidote.me/blog/how-to-find-and-take-part-in-paid-clinical-trials
- 40. Getz K. Improving protocol design feasibility to drive drug development economics and performance. Int J Environ Res Public Health. 2014 May 12;11(5):5069–80. pmid:24823665
- 41. Brøgger-Mikkelsen M, Ali Z, Zibert JR, Andersen AD, Thomsen SF. Online Patient Recruitment in Clinical Trials: Systematic Review and Meta-Analysis. J Med Internet Res. 2020 Nov 4;22(11):e22179. pmid:33146627
- 42.
ICMJE [Internet]. [cited 2021 May 5]. Available from: http://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/clinical-trial-registration.html