home > ict > summer 2010 > planned to perfection
International Clinical Trials

Planned to Perfection

Kim Boericke at i3 Research and Bill Gwinn at i3 Pharma Informatics explain how site selection and patient recruitment are interdependent

Do you hate to be late? Most people do, but many clinical trials run late, year after year. This article will address the major causes by demonstrating new techniques to mitigate the risk with protocol implementation, country and site selection, and patient recruitment. Better planning leads to faster trials.

In 2004, McKinsey and Company conducted an analysis on site performance (1). The results found that 30 per cent of sites enrolled 70 per cent of the eligible subjects in a trial, 50 per cent enrolled zero to two subjects, and 20 per cent enrolled more than two subjects, but well below the contracted total. When 70 per cent of the sites did not perform to contract, overall delays in meeting enrolment milestones resulted. In most cases, additional funding was needed to open additional sites or to conduct on-site visits to try and impact the overall timeline.

Six years later improvements have taken place, but a significant number of clinical studies do not complete enrolment on time. In 2009, 55 per cent of trials were able to finish enrolment early or on time, but the remaining 45 per cent completed enrolment late. Most of the late trials took up to twice the original timeline, but 10 per cent took even more. Figure 1 depicts the average delay in enrolment across 1,300 clinical trials conducted in 2009.

Over time, many project teams have retrospectively tried to understand the root cause of the delays, and several common factors have emerged (2):

  • Protocol development – entry criteria
  • Country/site selection
  • Engaged, experienced investigators
  • Experienced site staff with appropriate time allocated
  • Patient recruitment


The first major impact on the ability to recruit subjects is the protocol design. Since the cost of conducting clinical trials is rising, most drug developers are trying to maximise their R&D dollars by decreasing the total number of trials needed to demonstrate safety and efficacy, rapidly cutting drug development programmes that fail, and shortening the overall drug development cycle. As a result, protocols are becoming more complex due to strictly defined patient populations and increasing numbers of endpoints (3). The result is a significant decrease in the potential patient population, which increases the enrolment timeline, the number of recruiting sites, and the total cost. Figure 2 presents the average number of eligibility criteria by phase for studies completed from 2005 to 2009. The ones with the most patients, Phases II-IV, are going up.

The impact of complexity on trial timing is profound. Delphi Pharma performed a rigorous statistical analysis of over 1,300 US trials in 2009. The results demonstrated a direct correlation between increased inclusion/exclusion criteria and months of delay. This correlation is significant statistically at the 99 per cent level of confidence. More importantly, as the criteria continue to grow, drug developers can expect more delays unless new techniques are implemented.

Drug developers can achieve better results with better planning and access to objective information. Access to data assets, through organisations that conduct sophisticated analyses, can help the drug developer to finalise the entry criteria to maximise the potential patient pool. Specifically, analysts can mine insurance claims data to see how many potential patients really exist. The analyst starts with age criteria and diagnosis codes, and then works closely with the drug developer to create a more robust result. There are many insurance claim databases for sale, but the best analysis uses the largest one to extract the detailed data. The output presents the number of patients who match each entry criteria specification. These results clearly demonstrate the effect the criteria have on the potential patient pool, which is usually negative.

An illustration for a recent project to find bipolar patients is presented below in Table 1. A full protocol feasibility analysis, based on insurance claims, was conducted. The results define the initial potential patient counts based on basic ICD-9 diagnosis codes. The output demonstrates how the potential patient pool drops when additional entry criteria are added to the analysis. If there is a specification that eliminates too many patients, it shows. The drug developer can decide whether to revise a criterion or retain it. In either case, the output provides information that can help the drug developer plan and execute the trial.

The same data assets can be used to profile the potential patients and where they live, either by geographical zone or by their physician. The output is statistical profiles that depict high populations of potential patients or physicians, and not a description of any individual. This data is used to help support the site selection process. 

Table 1: Bipolar I disorder protocol feasibility analysis
   Bipolar patient counts from US insurance claims
 Description Patient count
Percentage of patients
 Initial patient set with bipolar diagnosis (age 18 and over, ICD9 diagnosis code 296x, which is a range of codes to cover all forms
 Exclude epilepsy – history of seizures  10,094  10.4
 Exclude patients with suicidal thoughts
 1,358  1.4
 Exclude patients who may become pregnant (on fertility drugs, pregnancy counselling and so on)
 1,067  1.1
Source: UnitedHealth Group, Ingenix subsidiary, national claims database of over 40 million lives  


Once the protocol has been assessed and the patient population defined, the next major question can be addressed – site selection. Where in the world should the study be conducted? What countries have similar standard-of-care practices? Who has access to enough potential patients to meet the enrolment timeline for the study? Is there any competition?

Most organisations conducting clinical trials have access to an investigator database. Over time, these databases have evolved. They started with a home-grown electronic phone book of investigative sites by therapeutic speciality; today they are complex relational databases, storing previous performance data, feasibility information, quality measures, and the basic investigator site information. These databases provide a starting point for the development of the site list of experienced investigators, but are not always the best place to find investigative sites that will enrol enough subjects to meet the recruitment timelines. Investigators may drop out of research, or not have appropriate patients.

Over 50 per cent of the investigators will only conduct one trial and then will return to private practice, according to the data found in the Bioresearch Monitoring Information System (BMIS) database over the past 26 years (4). The BMIS contains information submitted by clinical investigators to the US FDA. Figure 3 (page 28) presents the data analysed from the BMIS database between 1980 and 2006, assessing the number of studies in which 152,073 investigators have participated during that period.

The investigator database is one place to start. However, there are many other sources that can be used to gather, and objectively and analytically develop a qualified investigator site list for a trial. The trend is moving from the traditional method of collecting self-reported data through the feasibility questionnaire to a more analytical approach utilising privately and publicly available data. The result is a more objective assessment of screening and recruitment rates. No longer do drug developers have to rely solely on the self-reported data which require an internal correction factor to estimate the potential recruitment rates by site.

Many companies are developing processes for pulling data from internal and external sources. Some have access to a unique database, housing over 115,000 clinical trial investigators identified from the practicing physician population of 688,000. The source is insurance claims and other proprietary data. The profile within this database includes full contact information and, more importantly, these evaluation metrics:

  • Experience – this is the number of trials conducted in the past five or 10 years. A statistical analysis shows that higher experience is strongly correlated with recruitment success, measured by the number of patients randomised in past trials.
  • Date of last trial – as stated above, most investigators conduct only one trial. Concentrating on identifying physicians who have worked recently, there is a strong correlation with successful performance.
  • Patient count from insurance claims – this data is the most critical for ranking sites. Derived from insurance claims, it shows the number of potential patients at the site who exactly match the criteria of the protocol. The top-ranking investigators are shown to randomise patients more than twice as fast.

In addition to the proprietary data, most companies are mining published papers on trials completed in similar indications, accessing publicly available information from BMIS,, CRISP, AERS, as well as extracting incidence and prevalence data. This information can then be combined with the proprietary data assets to create a data set that can be analysed to produce a robust potential site list.

After the analyses are completed, the final report provides the proposed country selection and the potential site list with recruitment rates. The output utilises a modelling method that provides different scenarios for successfully completing the trial, which allows the drug developer to make the final call on where the trial will be conducted.

The potential site list is used for the final phase of assessment – the qualification visit. During site selection, it is imperative that time is spent on site to determine the investigator’s ability to be successful. Factors on top of the analytical information are:

  • Investigator engagement: who is the investigator and will this study be a priority at the site?
  • Staff experience: does the clinical team have the expertise and the time needed to conduct this trial? Is the facility appropriate for the trial?
  • Patient population: do they have access to the patient population? Do they have a patient recruitment strategy? Is there an internal network within the facility? Do they have the capability to reach out to other investigators or patient support groups?

By utilising the data assets and the site qualification visit, the major root causes for enrolment delays can be mitigated. By placing the clinical trial in the right sites around the world, with a solid protocol clearly defining the patient population, half the battle towards completing enrolment on time is done. The final step involves the actual recruitment of the patient.


Assuming the protocol has been finalised allowing for the largest potential patient pool, and that the investigative sites with the highest potential for meeting their enrolment goals have been selected, the next focus is on defining and implementing the strategy for recruiting patients. A trial-level strategy should be developed, pulling the information gathered at the site-qualification phase and combining the techniques with other strategies that can be implemented based on study design, indication and regional regulatory requirements.

Many vendors that can help define patient recruitment strategies and can implement specific plans around use of media, local advertising, outreach programmes, use of the internet and many other concepts. In all cases, the return on investment should be defined upfront in order to measure the effectiveness the strategy has on the actual enrolment. Plans should be monitored at the trial level and adjusted in real time to maximise the positive effect on recruitment.

Once the trial strategy is defined, the plan needs to be customised for each investigative site. Certain topics should be addressed in addition to the plan and discussed with the staff: 

  • The expected workload at the site level to identify potential patients, expected number of medical charts to pull, review, and then time to approach the patient
  • A screening rate and recruitment rate should be developed to define the performance expectations for the clinical staff to allow ongoing measurement of that performance
  • A value statement for the patient should be generated at the project level so a consistent message can be presented to outline the trial to the potential patients

Numerous vendors have varying strategies available to drug developers; however a unique recruitment technique is focused specifically on the investigative site. Accessing the insurance claims data, provides another unique application – to aid investigator sites in finding undiscovered patients within their practice.

Investigators commonly invite their own patients into a trial, but finding appropriate patients can require time-consuming chart reviews. Many patients may remain undiscovered, especially in large group practices comprised of many physicians. A thorough review of the charts could take weeks. A company with insurance claims access can solve this problem with automated reviews of the site’s claims, to identify the patients who exactly match all the criteria. The process protects patient privacy by providing the information only after receiving an IRB or Privacy Board waiver of authorisation, and by providing the information only to his or her own physician. That physician has to consent to be in the program and can decide whether to present the trial to the patient.


Utilising the output from the bipolar protocol and site selection feasibility analysis, the protocol was finalised with a patient population that could be recruited within the timeframe required. More than 60 sites were opened across the following countries: Brazil, Croatia, Czech Republic, France, India, Russia, South Africa and the US. The site selection process validated the feasibility information and sites were activated on time. Investigator engagement was good and resourcing at the sites allowed for sites to screen and recruit subjects according to the patient recruitment plans. The recruitment timelines for the study remained constant, however the protocol changed three times. In each case the protocol added procedures and increased the total number of patients to be randomised. Due to the high performing sites selected for the trial, not only did the recruitment timeline stay the same, the revised trial actually completed recruitment one month early. Figure 4 shows that actual recruitment was faster than planned recruitment for the trial.


Redefining the protocol to include the largest potential patient population, recruiting investigative sites with the high potential of exceeding enrolment targets, and developing a customised patient recruitment strategy are three key areas of focus during trial planning phase. The drug developer needs to understand the expected workload of the site to implement performance measures and goals to ensure successful enrolment of the trial. As presented in this article, the application of these techniques has worked well and has been implemented as a standard practice.


1. McKinsey and Company, Private study, 2004

2. McDonald A, Knight A, Campbell M, Entwistle V, Grant A, Cook J, Elbourne D, Francis D, Garcia J, Roberts I and Snowdon C, What Influences Recruitment to Randomized Controlled Trials? A Review of Trials Funded by Two UK Funding Agencies, Trials 7:7, 2006

3. Dugas M, Lange M, Berdel W and Müller-Tidow C, Workflow to Improve Patient Recruitment for Clinical Trials with Hospital Information Systems – A Case Study, Trials 9:2, 2008

4. Bioresearch Monitoring Information System, US Food and Drug Administration, InformationOnDrugs/ucm135162.htm

Read full article from PDF >>

Rate this article You must be a member of the site to make a vote.  
Average rating:

There are no comments in regards to this article.

 You must be a member of the site to make a comment.

In 1989, Kim McLean Boericke started conducting basic preclinical research at Duke University, before moving to Becton Dickinson Research Center in 1992, leading the biological models team conducting preclinical research for the transdermal drug delivery division. In 1996, Kim began managing global drug development programmes for ClinTrials. Over the next 13 years and across various CROs, she worked her way up from a Project Manager to become Vice President, i3 Research, the Americas. In her current role within i3, as Global Vice President, Start Up, Regulatory Compliance and Strategic Outsourcing, she is responsible for expediting start up from site/country feasibility through drug release, and for developing, managing and expanding i3’s strategic outsourcing relationships through functional service providers.

Bill Gwinn is Vice President, Clinical Informatic Solutions, at i3 Pharma Informatics. He supports the clinical trials of new drugs, and has developed medical statistics to find new patients and trial sites. Earlier, Bill was Director of Clinical Trial Solutions at Thomson Reuters Healthcare, and has held roles of leadership with Procter & Gamble, Schering-Plough, IMS Health and Inclinix. He has been speaker this year at industry conferences, including the Association of Clinical Research Professionals and Partnerships in Clinical Trials. Bill has an MBA from the University of Chicago.

Kim McLean Boericke
Bill Gwinn
Print this page
Send to a friend
Privacy statement
News and Press Releases

Luina Bio Announces New Bio-Manufacturing GMP And Development Facility Expansion

1st September 2020 Highlights • Small scale GMP manufacturing suites open in Q3 2020 • 500L tech transfer and process development site aim for open Q4 2020 • This precedes the next phase with Luina planning to commission a 10,000m2 late phase clinical and commercial production facility targeting late 2021 • New facilities capture more of the global value chain supporting technically advanced microbiome projects • New expanded capacity for bacterial and yeast recombinant projects including vaccine manufacturing including COVID-19 candidates
More info >>

White Papers

Inhaler Adherence: Exploring Key Factors

Aptar Pharma

Aptar Pharma recently hosted an expert panel roundtable on “Inhaler Adherence”. This international scientific forum was held in Paris, France on November 29th 2013. The roundtable was organized to explore and exchange views on the issues which may influence inhaler adherence/ compliance with a particular focus on asthma and C.O.P.D. therapies and their corresponding inhaler devices (MDIs, DPIs).
More info >>




©2000-2011 Samedan Ltd.
Add to favourites

Print this page

Send to a friend
Privacy statement