home > ict > spring 2010 > determining data
International Clinical Trials

Determining Data

John Shillingford at Averion discusses the role of study metrics in planning for clinical studies

The recent worldwide financial crisis has had a significant effect on the investment community, resulting in the need for industry to further reduce costs and tighten spending activities. This is accompanied by an ongoing requirement of management to reduce the R&D time and control costs of potential new drugs and devices – all taking place in a regulatory environment that is ever more complex.

The use of the global research programme is now standard, with industry looking for cheaper and faster routes to obtain the data they need for submissions to the authorities. The pharmaceutical industry has significant requirements to satisfy the demands of worldwide regulations; in this they need to employ the skills of international thought leaders and at the same time find adequate numbers of suitable patients who are willing and able to be part of such programmes. This often means that programmes need to be undertaken with countries with significant therapeutic opinion leaders and additionally in countries where patient acquisition into projects is optimal. The decisions as to which investigational sites, and thus in which countries to place the work, are often the crucial points to be considered for the planning of such pivotal studies. Detailed feasibility assessments and projections are vitally important, but this information is improved with a close look at recent site performance metrics on both a country and regional basis. These factors can greatly aid the overall creation of a good project strategy. To this end, analysis of performance data can be a vital tool in the determination of such a strategy.

This article describes analyses of two large international projects looking at aspects of patient accrual rates, quality of data and comparative costs in projects across Europe and the Americas. The aim was to analyse real-life situations that can give clear indicators of global project cost efficiencies from solid data collected over the last three years.

The analysis is geared to look at study metrics in terms of:

  • Recruitment rates – the number of patients recruited per site per month. The ability to monitor recruitment rates to attain study timelines is a key project management function.
  • Quality of data – the number of queries per completed CRF
  • Time spent on site – by clinical research associates to obtain one completed case report form. This is a good measure of the cost effectiveness of the investigator site and project monitoring team.


The first study analysed in this way was a renal carcinoma project which ran from 2006 through to 2009 and was undertaken in a total of 11 European countries. A total of 109 investigational sites were initiated; in turn, these sites recruited 518 patients. The results shown in Table 1 give a breakdown of average overall metrics for the study on a country by country basis.

The recruitment rates for western European countries (Germany, France and the UK) were broadly similar to those seen in Nordic countries (Sweden, Norway and Finland). However, those rates seen in central and eastern European countries – that is Russia, Ukraine, the Czech Republic and Poland – had a recruitment rate factor twice those of western Europe.

Data quality was largely similar across the project for western European and Nordic countries. However it can be seen that Russian, Czech and Polish sites had reduced query rates per CRF. Also, it must be commented that the quality of data coming from Ukraine was high, producing on average half the amount of queries seen in many western European sites. This factor may well come from the fact that the attention that these studies are given in eastern European research centres is certainly higher than that seen in western sites – a factor which is possibly aided by the higher staffing of clinic doctors to patients ratios that we see in these regions.

As can be seen in Table 1, a further interesting factor is the actual monitoring time metric. On average, the time a CRA in Russia, Ukraine, the Czech Republic or Poland has had to spend on site to attain a complete clean CRF was about half the time spent by CRAs in either western Europe or Nordic countries. Normally, an average visit is considered to be in the order of 20 hours (travel, preparation, on site and report times). Given the differentials in costs of the CRAs in CEE regions in terms of salaries and other associated factors, which can be more than 40 per cent less than their western European counterparts, we should be able to show considerable cost savings. The reasons for this efficiency in CRA time are probably rooted in the fact that not only are there generally more CRFs on site, which in itself makes the processes more efficient, but also that the investigators make an effort to be available, as well as the fact that the CRFs are generally ‘cleaner’ requiring fewer queries.

The general increase of required site time in western Europe may be a reflection of the need to work with the sites to look at their screening failures and develop a recruitment plan, as well as spend time motivating the investigator team to recruit for this particular study. Patient recruitment is a highly competitive arena, where each study is competing not only for the patients to be allocated, but also for the actual time of the investigating team. This required support and ‘tender loving care’ is very apparent with western European sites and, as can be seen from the numbers in Table 1, becoming true for central European sites as these become even busier with projects.


The second project presented here is a cardiovascular study in patients with chronic heart failure. A total of 160 investigator sites in 17 countries took part in the study recruiting 2,034 patients. The details of the study metrics are presented in Table 2, page 24.

The study was carried out under an IND where the FDA had specified that 50 per cent of the patients must come from countries that were classified as ‘USlike’ in their medical practices for heart failure. This effectively meant that the US, Canada, western Europe and Israel were classified as ‘US-like’, with other countries being included for purposes of rapid patient accrual to bring the study in on a tight timetable. As can be seen, a total of 92 sites in the ‘US-like’ countries were needed to bring in 1,020 patients, as compared to 68 sites required in the rest of the world, such as central and eastern Europe and Argentina, to recruit 1,014 patients.

As can be seen from Table 2, patient recruitment rates were similar in the US-like countries, with the exception of Israel, the UK and Italy, which were a factor close to three times those rates seen in the US. The Italian data is skewed by one very high recruiter who was an active member of the steering committee. The UK and Israel, with more centralised National Health Systems, showed good comparable recruitment rates.

Recruitment rates in eastern Europe were clearly remarkable, with Russia, Romania and Ukraine showing rates with a factor of six times those seen in the US. Russia was actually stopped from recruiting prematurely as, while the patients recruited were within terms of the inclusion and exclusion criteria, on blind analysis of baseline data the patients were seen to have a differing profile – being younger, better renal function and with a prospect of staying in hospital longer. Therefore they may have had better outcomes than the norms of patients from other regions. A point here that needs to be made for large international studies is observation of patient profiles during the patient accrual phase.

General observation of data quality in this project demonstrated that, while the rates of queries per CRF are comparable across the Americas, western and central Europe, those seen in eastern Europe are some 30 per cent lower, and with the amount of data coming from this region, this became a significant factor in cost efficiency terms. It must be commented further that Argentina, Ukraine and Romania were bought into the project at a late stage, hence while their recruitment rates were good, some aspects of quality of data and cost, in terms of CRA visits to site, suffered somewhat due to the haste in which they were initiated and followed up.

However, in general, the factor of CRA time spent to attain a completed CRF in eastern Europe represents an average of 60 per cent of that required in other regions that took part in this study. The reasons for centres in the US and western Europe requiring more CRA site time have already been discussed. However, with some 40 per cent of the study patients coming from central and eastern Europe – where there are significantly lower CRA and investigator costs – this region did give the study significant efficiency.

The metrics shown by the investigator sites in Israel clearly demonstrate the advantage of working there. The fact that the FDA regards this region as ‘US-like’ also makes this an attractive country for many projects.


In the planning process for any large pivotal study, a number of factors need to be considered in allowing the project team to derive a focused strategy to ensure project success. The major factors are highlighted as follows:

  • Project aim – the exact purpose for the projects needs to be examined. Is it a standalone pivotal study or will there be other projects as confirmatory studies? What are you trying to achieve?
  • Which key opinion leaders will be involved and where are they based?
  • What regulatory strategy is being employed – such as submission strategies to worldwide authorities?
  • Where is the patient source for the project?
  • Considerations of timelines and cost constraints will always be crucial?

The analyses of study metrics over the two projects presented here give remarkably similar profiles and are in line with other published data. It is clear that countries and regions outside of the traditional centres of North America and western Europe are able to recruit valid patients quickly and produce quality data in a cost effective manner.

However, while such investigational centres can aid project teams to successfully bring in studies in a timely manner, you do need to be mindful of the requirements of key regulatory agencies such as the FDA and EMA. The agencies are becoming more mindful of the comparability of treatment procedures and patient profiles from differing regions. Experience has shown that the FDA and EMA have accepted new drug applications with high levels of ‘non USlike’ data, dependent upon indication of course. A usual reasonable level of between 50 and 70 per cent has been accepted in cardiovascular, oncological and CNS indications. However, it is important that an international project team is involved at all stages of planning, feasibility assessment, project initiation and development, taking into account the numbers of patients included in certain regions, and constantly ensuring patient standard of care is consistent.

The challenge as we move forward is integrating the work and expertise of the key opinion leaders and the centres of clinical excellence that are recognised by the major regulatory agencies of the world with centres that are able to recruit patients that are in the emerging nations of drug development rapidly. These nations, which are building experience and expertise on a year by year basis, are becoming a major part of the engine of global drug development, which is important in our cost- and time-restrained world.

Read full article from PDF >>

Rate this article You must be a member of the site to make a vote.  
Average rating:

There are no comments in regards to this article.

John Shillingford is Senior VP in Clinical Operations of Averion International, with responsibility for new projects initiations in North America and Europe. John is located in Averion’s German office close to Frankfurt. John has had the opportunity to live and work in a number of European countries and the US. A major focus of the last 10 years has been to successfully commence and manage operations in the countries of central and eastern Europe. John has over 30 years of pharmaceutical and contract research industry experience. He has worked in all phases of drug development and has managed clinical research projects in eastern and western Europe, and in North America on behalf of clients from Japan, the US and Europe. Prior to Averion, he has held senior positions in management, clinical operations and project management for PRA International and Imform GmbH.
John Shillingford
Print this page
Send to a friend
Privacy statement
News and Press Releases

Takeda Provides Updates on Its Pharmacokinetic-guided Prophylaxis Studies at ISTH 2020, Reinforcing Its Commitment to Advancing Personalized Care for People with Bleeding Disorders

July 12, 2020 - The Data Demonstrating Benefits of Pharmacokinetic-Guided Prophylaxis Therapy in Hemophilia and von Willebrand Disease are Among 13 Presentations from Takeda’s Hematology Portfolio and Pipeline
More info >>

White Papers

R&D to Product: How to Reduce Costly Scale-Up Problems

Natoli Engineering Company, Inc.

When scaling a new drug formulation from the development side of R&D into manufacturing, there is often a reality check with respect to process scaleup and performance of the formulation at production levels. Problems with capping/lamination and sticking/ picking are often first realized when a formulation is introduced to a level of compression required to supply market demand.
More info >>

Industry Events

World Vaccine Congress Washington

27-29 September 2020, Walter E Washington Convention Center, Washington, US

The World Vaccine Congress is an award-winning series of conferences and exhibitions that have grown to become the largest and most established vaccine meeting of its kind across the globe. Our credibility is show through the prestigious scientific advisory board that spend months of hard work creating a new and topical agenda, year on year.
More info >>



©2000-2011 Samedan Ltd.
Add to favourites

Print this page

Send to a friend
Privacy statement