home > ict > spring 2014 > quality control
International Clinical Trials

Quality Control

Drug developers are now moving from the traditional high-cost model of clinical trial monitoring to data-driven, risk-based monitoring (RBM), with the potential to improve data quality and reduce research costs. However, while both the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA) have published guidance supporting risk-based approaches, they have not provided specific direction for the design and implementation of RBM plans (1,2).

A 2013 survey by the Avoca Group found that 47 per cent of top 20 pharmaceutical companies and 39 per cent of contract research organisations (CROs) use some type of RBM approach in more than half of the clinical trials they conduct (3). Several initiatives, including the Clinical Trials Transformation Initiative and TransCelerate, have proposed RBM methodologies for industry adoption (4,5).

This article demonstrates how quality risk management (QRM) principles can be applied, prior to clinical trial execution, to optimise the design and execution of RBM plans. The authors discuss current experience using a risk management tool to systematically assess risks inherent in study design and execution, and present a case study to show how findings can be used to design an effective RBM plan.

Systematic Risk Assessment

The ability to predict and mitigate risks is at the heart of improving data quality, patient safety and operational efficiencies. The adoption of a systematic risk assessment process is essential to realise the full benefits of RBM.

Identification and prioritisation of potential study risks is a significant challenge for drug sponsors due to compartmentalised research functions that make data sharing difficult. In the Avoca survey, 58 per cent of top 20 pharmas and 39 per cent of CROs reported using some form of systematic risk assessment in 75 per cent of their trials. The most frequent types of risks assessed were patient enrolment, vendor performance, data quality and timelines (3). In addition, the Metrics Champion Consortium lists patient enrolment/retention, protocol compliance, data quality, patient safety and sample quality problems among the eight most common considerations for RBM plans (6).

Integrating QRM and RBM

QRM offers the means to improve on current industry practice, but its applications in drug development have been limited. The International Conference on Harmonisation (ICH) 2006 guidance, Q9 Quality Risk Management, defines QRM as the “application of systematic assessment, control, communication, and review of risks to the quality of a drug product across the product lifecycle” (7). It includes three components, based on scientific knowledge and conducted systematically across multi-disciplinary teams: risk assessment, risk control and risk review. Risk assessment addresses three central questions: What might go wrong? What is the probability it will go wrong? What are the consequences?

Applying QRM principles, a systematic risk assessment approach is being developed by PPD to evaluate risks inherent in study design and execution. This two-step process uses information from the study protocol to conduct an initial risk assessment that drives the design of a tailored, adaptive RBM plan. Initial assessment is followed by a comprehensive, cross-functional risk evaluation of potential risks across the study, with the results used to further define the monitoring plan.

Two-Step Process

Two risk assessment questionnaires were developed, aimed at systematising a rigorous risk assessment process to be applied across studies and project teams.

Initial Protocol Risk Assessment
The company’s risk management process begins with a preliminary risk evaluation to support bid submission. This initial protocol risk assessment (IPRA) uses the 15 selected risk categories shown in Table 1 to gauge the appropriate effort and intensity of monitoring needed to oversee the quality and compliance of the sites.

To complete the initial assessment, protocol requirements are detailed for each of the categories. A level of risk is assigned for each category on a scale of one (no risk) to five (very high risk), and a mitigation action is assigned. For example, in a study involving patients with metastatic pancreatic cancer, eligibility might be complex. The eligibility category might be scored ‘moderate risk’ and monitoring might call for remote review of eligibility as soon as it is entered into the electronic data collection (EDC) system (3).

The IPRA drives the majority – perhaps 95 per cent – of the strategy used to define the monitoring plan. It determines which subject visits are the most critical or susceptible to site error and guides mitigation plans. Results are used to identify and prioritise essential training and specific monitoring activities. Sites, visits and data that pose the most significant risks are assigned the highest levels of scrutiny, with 100 per cent source data verification and on-site monitoring.

Lower levels of risk require lower levels of monitoring – for example, 20 per cent on-site and 80 per cent remote review. As the trial moves forward, the RBM plan is adapted based on continuous quality assessment data: poor source quality drives increased review; poor case report form quality drives increased case report form review; risk factors and poor overall quality drive higher levels of on-site monitoring.

Project Risk Assessment Questionnaire
Once a project is contracted, a project risk assessment questionnaire (PRAQ) is used to conduct a comprehensive, cross-functional risk evaluation to identify risks that may occur in any study function, as well as in the hand-off of data and information between functions. The PRAQ assesses the nine risk categories listed in Table 2 to identify risks in four areas critical to study outcome: complexity of design, eligibility criteria, treatment administration and study end-points.

Based on the nine-category output, the project manager (PM) consults with the cross-functional team to determine which risks warrant mitigation plans. Each question is scored on a scale of one to five, with five being the highest risk. The nine-category traffic light output will show which questions scored a four or a five, denoting the questions in that category that scored the highest risk. Risk mitigation plans are based on the score and the PM’s agreement that the item does, in fact, pose a risk.

Options include ‘disagree’ that it is a risk; ‘agree and accept’ the risk; and ‘agree and mitigate’ the risk. The output will show those questions that scored a four or a five, so the highest risk questions in that category and the mitigation plans only are written for those risks that are classified by the PM as ‘agree and mitigate’, in consideration of client internal risk management plan and client clinical team point of view.

The PRAQ results may prompt refinements to the monitoring plan. For example, a study requirement for monthly pharmacokinetics samples might be identified as a risk for site compliance; a direction for contract research associates (CRAs) to review and confirm monthly samples could be added to the monitoring plan. PRAQ evaluation is repeated throughout the duration of the project to drive ongoing adjustments.

Fostering Best Practice

The IPRA has been applied to RBM design in 30 clinical trials since October 2013, while the PRAQ was recently rolled out over some 20 studies. Most of those studies are not currently RBM trials; however, the use of PRAQ for RBM studies going forward is expected to grow.

As these studies are completed, project teams plan to compare risks identified in the assessment with actual risks experienced in each study, and to evaluate the effectiveness of mitigation strategies. Findings will be used to improve the risk assessment process and foster best practices across projects, as depicted in Figure 1. Results also may provide insight for risk profiles for therapeutic categories, patient populations, geographic regions and other study parameters.

Phase 3 Case Study

This section describes how key findings from an IPRA and PRAQ were used to design the RBM plan for a Phase 3, randomised, crossover, multicentre study to evaluate a chemotherapeutic agent in subjects with non-small cell lung cancer. The study enrolled about 700 subjects across 80 sites.

IPRA Findings
The initial assessment indicated that subjectivity would be a factor in a number of critical aspects of the study. To reduce risks related to subjective determinations at the sites, the RBM plan called for robust site training and strong oversight, to ensure accurate documentation and assessment of disease progression, clinical significance and adverse events (AEs).

Risks Related to Interpretation of Disease Progression
The study required stratification using electrocochleography (ECOG) disease progression measures, tumour response and disease state. To ensure accuracy and consistency, the RBM plan called for assessment of investigator familiarity with ECOG; training and reiteration of the purpose of ECOG during the initial site visit; and real-time remote review of EDC data for randomisation.

To ensure that patients met qualifications for specific disease progression, the study required up to five years of past patient records. The monitoring plan stipulated real-time review of eligibility, scans and laboratory data, and eligibility question management between the site and medical monitor using an electronic protocol enquiry process.

Monitoring of efficacy end-points included regular remote review of disease assessments; checklists confirming progression assessment were submitted to CRAs during remote monitoring visits. To address potential risks arising from lack of integration between EDC and electronic patient-reported outcomes (ePROs), CRAs were required to incorporate ePRO review into remote monitoring visits to identify AEs and ensure their documentation in EDC.

Risks Related to Patient Safety
Based on the safety profile of the investigational product, the protocol clearly defined dose reduction steps in the case of neurologic or haematologic toxicity. Ensuring site compliance with these steps was critical. The monitoring plan therefore called for remote review of laboratory results and AEs in EDC, and for comparison against dose reduction notifications from interactive voice response systems, to ensure that sites took appropriate steps.

PRAQ Refinements
Completion of the full PRAQ identified additional project-level risks. The RBM plan was refined to address additional issues, including subjects’ use of electronic tools, subject discontinuation and just-in-time clinical supplies.

Risks Posed by Subjects’ Use of Electronic Tools
The protocol specified subject use of electronic tools (eTools), including patient diaries and individual electronic heart monitors. The monitoring plan provided a back-up plan for data collection if an eTool was unavailable and detailed instructions for CRA assessment of eTool data at specific times, including AEs, missing data and transmission errors.

Risks Related to Subject Discontinuation
The anticipated rate of subject discontinuation was 15 per cent or more. The monitoring plan outlined requirements for appropriate management of discontinuations: what information needed to be collected; how to document discontinuation of a drug or discontinuation of all study activity, including post-study, follow-up treatment; the procedure for follow-up monitoring after discontinuation; and the procedure to document the continuation of medical care.

Risks Related to Clinical Supplies

Investigational, comparator and add-on drugs and devices were supplied to sites using just-in-time supply services. Sites purchased some supplies locally; study drugs were also pooled across different protocols. Monitoring included clinical supply assessments during site management calls and remote interim monitoring visits.

In addition to guiding RBM plans, formal, systematic risk assessment could inform risk surveillance plans and audit plans. Results of the IPRA could be used to develop additional edit checks or listings and key risk indicators at the project level or for a therapeutic area, and perhaps to identify trends earlier. Quality assurance teams might use results to fi ne-tune audit strategies. PRAQ project scores could be risk-ranked, which might help quality assurance teams target studies for audit plans.

Case-By-Case Evaluation

A rigorous, systematic approach to risk assessment – applied across study execution – improves the likelihood of identifying risks to subject safety and data quality, and preventing errors that impact study outcomes.

The IPRA facilitates a detailed monitoring approach for customised risk mitigation; it helps research teams capture and apply their combined experience to ensure that potential risks are not overlooked, rather than depending on the knowledge of a single source. Findings of the PRAQ further refine RBM plans. Indeed, the PRAQ is an evolving risk assessment tool aimed at ensuring consistent risk assessment practice across studies and research teams.

As a process, conduct of the IPRA and PRAQ systematises case-by-case risk evaluation and design of monitoring activities, and encourages collaboration across functions to improve overall study quality.

1. FDA, Guidance for industry: Oversight of clinical investigations – a risk-based approach to monitoring, 2013. Visit: downloads/drugs/.../guidances/UCM269919.pdf
2. EMA, Reflection paper on risk-based quality management in clinical trials, 2013. Visit: jsp?curl=pages/regulation/document_listing/document_ listing_000136.jsp
3. The 2013 Avoca report: Sponsor and provider perceptions on managing clinical development risk, 2013. Visit: summary
4. Clinical Trials Transformation Initiative, Quality objectives of monitoring: Workstream 2 final report. Project: Effective and efficient monitoring as a component of quality in the conduct of clinical trials, 2009. Visit: documents/monitoringWS2fi nalreport.pdf
5. TransCelerate position paper: Risk-based monitoring methodology, 2013. Visit: uploads/2013/09/risk-based-monitoring-methodology-positionpaper.pdf
6. Price J, Risk-based monitoring: What is your plan? BioClinica, 3rd September 2013. Visit: what-your-plan
7. ICH Guidance for industry: Q9 quality risk management, 2006. Visit:

Read full article from PDF >>

Rate this article You must be a member of the site to make a vote.  
Average rating:

There are no comments in regards to this article.

Stephannie Perrin is Director of Quality Risk Management at PPD, responsible for establishing a quality risk management and governance framework, and maintaining close client relationships. She joined PPD in 2004 and has held positions in both the clinical and quality areas. Stephannie is a licensed psychologist, and holds a Bachelor’s degree in Psychology from the University of North Carolina at Chapel Hill and a Master’s degree in Clinical Psychology from the University of North Carolina at Wilmington.

Nicole Stansbury is PPD’s Senior Director of Remote Site Management, a group created to focus solely on implementation of the company’s RBM approach and the people, tools and processes associated with it. She previously served as Senior Director of Global Clinical Performance. Nicole joined PPD in 1997 as a Clinical Research Associate. She holds a Bachelor’s degree in Animal Science from Texas A&M University.
Stephannie Perrin
Nicole Stansbury
Print this page
Send to a friend
Privacy statement
News and Press Releases

Nemera launches UniSpray, its unidose system primarily for systemic-acting drug administration

Nemera has a proven track record in developing and manufacturing complex Ear Nose and Throat solutions. In line with clear growing market interest in unidose system coupled with promising advantages that nasal route could offer, Nemera is thrilled to extend its portfolio by launching its unidose system, UniSpray, for systemic-acting drug administration to face emergency and crisis.
More info >>

White Papers

Clinical Trial Labelling – More Than Just Labels

Faubel & Co. Nachfolger GmbH

Deciding which label is best for a particular trial project is not always easy. Labels have become multifunctional tools which are able to convey variable data in different languages, indicate first opening, product originality, support ease of use or blind study drugs. They are no longer used as mere carriers of specified contents.
More info >>




©2000-2011 Samedan Ltd.
Add to favourites

Print this page

Send to a friend
Privacy statement