home > ict > winter 2014 > vital statistics
International Clinical Trials

Vital Statistics

Quality and integrity of data in clinical trials is indisputably the overriding factor in ensuring patient safety and, ultimately, determining study success. This argument was backed by the US Food and Drug Administration (FDA) in August 2013, when it released new guidance on risk-based monitoring (RBM) approaches. The guidance encouraged sponsors in the biopharmaceutical and medical device industries to diversify their monitoring methods to not only enhance the conduct of clinical investigations, but also to improve the quality of data collected (1).

The guidance describes the importance of using centralised monitoring tools, such as central statistical monitoring (CSM), to objectively and independently interrogate study data. In a landscape that is becoming ever-more challenging – with both the size and cost of studies increasing and making certifying data quality more difficult – CSM approaches represent a significant step forward, helping to guarantee the quality of the data, while mitigating risk and containing costs.

Monitoring Challenges

Recent years have seen clinical trial processes become increasingly difficult to manage, with multiple fragmented practices incorporated into every trial without questioning the value they contribute to the overall conduct of a study. To add to this, over the last decade the number of drug approvals has declined steadily, while the cost of clinical research has risen. If this continues, sufficiently sized trials will become unfeasible.

This has led many in the industry to question some of the custom practices that stretch both budgets and resources, along with a call for more effi cient means of carrying out essential procedures. Eisenstein et al and Califf have conducted research into where costs could be reduced without compromising scientific validity (2,3). Their research has indicated that significant savings could be made by reducing labour-intensive tasks such as on-site monitoring, which typically accounts for 30 per cent of sponsors’ overall costs in global clinical trials, but results in less than three per cent of any data changes (3).

Practical Thinking

Historically, the standard model for clinical site monitoring was a prescribed schedule of site visits every four to eight weeks. Aimed at providing an exhaustive quality control, these methods usually rely on 100 per cent source data verifi cation (SDV) to help ensure subject safety and generate reliable data. This is fundamentally a reactive approach that is limited in its ability to efficiently identify data issues and allow measures to be put in place to prevent them from recurring.

When combined with the current system of regulatory bureaucracy in clinical trials, traditional thinking has created an expensive research paradigm that, in spite of complex systems of oversight and exhaustive data collection, cannot be shown to adequately ensure the integrity of the research process (3).

With budgets becoming tighter and data issues growing, regulatory authorities no longer believe these methods are the best solutions. As a result, the industry is witnessing a move towards more practical trial designs that offer improved cost efficiency without compromising the integrity of results.

Regulatory Landscape

The FDA guidance for industry encourages greater reliance on centralised monitoring practices than has traditionally been the case, with less emphasis on on-site monitoring. The European Medicines Agency (EMA), in a 2011 reflection paper, has adopted a similar viewpoint by stating: “Adaptations to conventional Good Clinical Practice methods, for example, adaptation of on-site monitoring visits, sample/focused SDV, new central monitoring processes etc, subject to appropriate metrics being captured to determine when/if escalation in monitoring, would be appropriate” (4).

Both the FDA guidance and EMA reflection paper support the use of RBM tools, as well as increased efficiency through reduced SDV and greater focus on targeting on-site monitoring activities. They recommend that a monitoring plan should only be developed once the particular risks associated with a study have been assessed, and that data is analysed on an ongoing basis to review and adapt the monitoring strategy.

Risk Assessment

As no single approach will be appropriate for every study, an RBM plan should be tailored to the unique risks associated with a specifi c trial. The plan needs to identify the methods to be used and the rationale for using them, with a focus on preventing important and likely sources of error in the conduct, collection and reporting of critical data.

Sponsors need to prospectively identify critical data and processes necessary for trial integrity, before performing a risk assessment to identify the risks that could impact the collection of data or the performance of processes. From here a plan can be developed that highlights the important and possible risks (1).

The publishing of the new regulatory advice has highlighted that risk assessment should be the guiding principle in determining monitoring plans. This shift in thinking calls for RBM processes that use centralised and off-site mechanisms to monitor important study parameters globally and use adaptive on-site monitoring to support site processes.

Though current methods do deliver some control, advances in risk-based approaches and the emergence of supporting technologies offer the opportunity for sponsors to adopt a more holistic and proactive strategy. By incorporating quality and risk management techniques into the scientific design of trials, risks can be mitigated and issues prevented or detected earlier, enhancing data quality.

Current Approaches

RBM strategies have tended to focus on key risk indicators – for example, number of queries and number of adverse events – all of which are predefined factors. The use of these methods enables metrics to be computed for each study site, which allows monitoring frequency to be adapted according to site performance.

However, although effective to a certain extent, these approaches are limited by a number of implementation challenges. They need to be predefined, programmed, tested and validated, and only make use of certain parts of the massive volumes of data generated. Furthermore, problems are created by the capability to only pinpoint errors once a study has reached a certain stage, meaning that a trial may have progressed signifi cantly before any issues are identified.

New regulatory guidance has pointed to the use of statistical theory to help optimise RBM. Statistical theory is already heavily embedded in the design and analysis of clinical studies, and the potential of statistics to uncover fraud in multicentre trials has received significant academic attention (5,6). CSM is generating interest in the industry for its potential to take a holistic view of data, enabling anomalies to be identified efficiently and, as a result, focusing monitoring activities on centres that are deemed to be higher risk (7).

Introducing CSM

CSM is an agnostic approach that is based on actual clinical data, rather than predefined criteria. All variables are deemed to be indicative of quality, whether it is lab data, clinical data, baseline data or treatment outcomes, and counted as equally important.

The method compares the distribution of all variables in each study site with other sites, meaning that abnormal patterns can be identifi ed. Using complex and proven statistical algorithms to ‘drill down’ into individual patient data, it can detect signals to better manage risk and determine variations between sites in order to identify statistical outliers. This allows sponsors to identify errant sites quickly, and immediately send monitors to those sites, consequently improving data quality.

By analysing all available information, CSM is able to detect issues, such as lack of variability in or implausible values, that are likely to go undetected using other methods. It offers the potential to determine where issues might lie during study conduct and before significant problems occur, helping to avoid any issues that could put a trial at risk or jeopardise successful regulatory submissions.

The approach requires minimal work for the study team in gaining objective information, and the protocol is implemented in the same manner using the same case report form (CRF) at all sites.

Multivariate Structure

Throughout a study, variables are grouped by CRF section. These are then grouped by visits, visits are grouped by patient, patients are grouped by investigator, investigators grouped by centre, centres grouped by country, and countries grouped by geographical region.

The method works on the basis that the multivariate structure and time dependence of variables in statistical checks are highly sensitive to deviation and extremely difficult to mimic. Comparisons can be performed with either one variable at a time in a univariate fashion, or with several variables, taking into account the multivariate structure of the data or using longitudinal data when the variable is repeatedly measured over time.

Fabricated data will exhibit abnormal multivariate patterns that are detectable statistically. In addition, humans are known to be poor random number generators, meaning that tests on randomness can be used to detect falsified data (8).

Industry Expectations

During a recent webinar, 300 industry professionals from clinical operations, data management and statistics were surveyed about the benefits they would expect from an RBM approach (9). Furthermore, CSM can help define more effective RBM monitoring strategies by efficiently detecting errors, sloppiness, tampering and even fraud.

Sponsors can take corrective action in a timely manner to prevent issues becoming deep-rooted and reanalyse data regularly as part of an iterative process. This considerably reduces the probability for safety studies to be performed further down the line, should data issues cast any doubts on trial results.

An additional benefit of CSM is that sponsors which strategically outsource to contract research organisations (CROs) are using the solution as an oversight tool to check the quality of their data. As a result, the technique is proving an essential instrument in the selection of sites and CRO partners for future trials.

Objective Focus

There is agreement across the industry, from both sponsors and regulatory agencies, that extensive data checks during on-site monitoring visits are neither able to identify errant sites quickly, nor adequately ensure the quality of trial data. In contrast, CSM approaches are able to identify anomalies in data earlier, offering the opportunity to eradicate any issues as they are uncovered – yielding large cost savings while increasing the reliability of trial results.

While traditional methods are able to hint at possible issues, CSM looks at all the information available to focus on the issue itself, providing a more objective and meaningful interpretation of data. This represents a major paradigm shift. Sponsors that put themselves at the forefront of the initiative and embrace new regulatory guidance and methodologies early will reap the benefi ts of an approach that puts quality data at the very core of a clinical trial.

1. FDA Guidance for Industry: Oversight of clinical investigations – a risk-based approach to monitoring. Visit: downloads/drugs/guidancecomplianceregulatoryinformation/ guidances/UCM269919.pdf
2. Eisenstein E, Collins R, Cracknell BS et al, Sensible approaches for reducing clinical trial costs, Clin Trials 5: pp75-84, 2008
3. Califf RM, ACS and acute heart failure models, Speaker presentation at the Institute of Medicine Workshop on Transforming Clinical Research, October 2009. Visit: transforming-clinical-research-in-the-united-states.aspx
4. EMA reflection paper on risk based quality management in clinical trials, EMA/INS/GCP/394194/2011. Visit: docs/en_GB/document_library/scientific_guideline/2011/08/ WC500110059.pdf
5. Buyse M, George S, Evans S et al, The role of biostatistics in detection and treatment of fraud in clinical trials, Statist Med 18: pp3,435-3,451, 1999
6. Evans S, Statistical aspects of the detection of fraud; in Lock S, Wells F and Farthing M, Fraud and misconduct in biomedical research, BMJ Publishing Group, 2001 (third edition)
7. Baigent C, Harrel F, Buyse M et al, Ensuring trial validity by data quality assurance and diversification of monitoring methods, Clin Trials 5: pp49-55, 2008
8. Venet D, Doffagne E, Burzykowski T et al, A statistical approach to central monitoring of data quality in clinical trials, Clin Trials
9: pp705-713, 2012 9. Webinar: Removing the risk in risk-based monitoring: a practical and proven approach to implementing central statistical monitoring. Visit: xto612cluepoints/reg1.html

Read full article from PDF >>

Rate this article You must be a member of the site to make a vote.  
Average rating:

There are no comments in regards to this article.

Marc Buyse is the Founder of the International Drug Development Institute (IDDI) and CluePoints Inc. He holds Masters’ degrees from Brussels University and Cranfield School of Management, as well as a doctorate degree from the Harvard School of Public Health. Prior to setting up the IDDI in 1991, Mark worked at the European Organisation for Research and Treatment of Cancer in Brussels and the Dana Farber Cancer Institute in Boston. He is also an Associate Professor of Biostatistics at Limburgs Universitair Centrum in Belgium.
Marc Buyse
Print this page
Send to a friend
Privacy statement
News and Press Releases

First cellularized collagen membrane implant to regenerate heart tissue in patients with ischemic heart disease

• 3P Biopharmaceuticals and Viscofan go ahead with this innovative project of regenerative medicine for cardiac use • Beginning of Phase I clinical trial in a group of 10 patients
More info >>

White Papers

Role of Comparators in Clinical Trial Supply

Clinical Services International

CSI analyses the role of comparators in different trial designs and offers expert advice on robust methodology for choosing the most appropriate comparators. Case studies provide real insight on how to choose comparators for rare diseases and oncology studies.
More info >>




©2000-2011 Samedan Ltd.
Add to favourites

Print this page

Send to a friend
Privacy statement