home > ict > winter 2012 > time for change
International Clinical Trials

Time for Change

Jules T Mitchel and Judith M Schloss Markowitz at Target Health Inc. explain how risk-based monitoring and direct data entry will transform clinical trials

Risk-based monitoring (RBM) within a clinical trial involves, at a minimum, performing a risk assessment of how to assure that the protocol will be followed; assessing which types of data should be collected and how to gather and monitor the data for accuracy; and evaluating and monitoring potential safety concerns. When RBM programmes are initiated, monitoring activities can focus more effi ciently on compliance with the critical protocol elements and how to prevent errors in data collection and reporting.

Direct data entry (DDE) allows for electronic collection of data at the time a study subject outcome is being evaluated in a clinical trial. DDE also allows for the acceptance by regulators of eSource data as original data, rather than requiring subject data to be fi rst transcribed on a piece of paper. Basically, eSource then becomes the eClinical trial record (eCTR), which can include, for example: clinical laboratory diagnostic data; the electronic medical record (EMR); and creative technology solutions by EDC companies. This ensures that eSource records are original records controlled by the clinical research sites, and that a writeprotected copy of the eCRF for the study archives are available following review and sign-off.

There will be some resistance by the pharmaceutical industry to adopt RBM and DDE, as there was with the adoption of electronic data capture (EDC), but once adopted the transition will result in a signifi cant increase in the effi ciency of clinical trial operations, especially in protocol compliance and monitoring of data integrity and data quality.

RBM and DDE should also reduce the overall costs of monitoring clinical trials and will expedite the delivery of safe and effective new products to the market. Over the last fi ve years electronic submission standards and technology have become well-established, and this fact may also ease the transition to RBM and DDE. RBM and DDE have the potential to modernise and reduce time spent in the processes where regulatory dossiers are authored, compiled, inspected, checked for completeness and reviewed by regulatory authorities. DDE could enable regulatory authorities to perform risk based pre-approval inspections of high-enrolling clinical trial sites even during conduct of the pivotal trials, which saves time and gives a more realistic picture of how well the trial is being carried out.

The fundamental logic to assure a rational approach to RBM will be coordinated with clinical research, biostatistics, data management, quality assurance and regulatory affairs. Regulators in both the US and Europe have opened doors with several new guidance documents (see below). As a result, the barriers to full scale adoption of the paperless clinical trial are the same barriers that are systemic with paradigm shifts within any industry.


In 2011, Karyn Korieth wrote a thoughtful article on the high cost and questionable impact of 100 per cent source document verification (SDV) (1). Korieth made it very clear that FDA regulations do not require study monitors to check every data point within a clinical trial. Nevertheless, 100 per cent SDV has become the norm. Korieth also noted that as budget pressures intensify, the practice of 100 per cent SDV will be questioned, and cited evidence that pharmaceutical companies could save up to 23 per cent on a Phase 3 study that combined RBM with fewer monitoring visits (2).

DDE is not new. Back in 1984, Jay Herson highlighted remote data entry (RDE) as a new technology that may have a profound impact on the conduct of clinical trials (3). Under RDE, the clinic site has the ability to enter data into a personal computer at the clinic, followed promptly by data transmission to a mainframe computer at the data coordinating centre. In contrast, the conventional data entry process at that time involved mailing completed case report forms (CRFs) to the data coordinating centre, processing the CRFs by the data management department and, finally, entering data directly into a centralised mainframe computer. Herson also addressed regulatory concerns, stating that “it is not clear what position the Food and Drug Administration will take on the acceptability RDE in the regulatory process of new drug applications. RDE would have to provide complete audit trails of all changes and modifications of patient data and provide for an equivalent of an investigator’s signature on all data entered.” In terms of costs, the paper stated that RDE would lower costs for producing and printing the CRFs as well as executing and managing the entire data entry process. Much of these savings were to come from the fact that clinical research site employees would perform data entry under RDE as opposed to company employees. Another advantage was that the cost for ineligible patients (protocol violators) was also less under RDE, since fewer ineligible patients were going to be enrolled into studies.

In 1998, Andrew Hyde wrote that since 1988, RDE has been the “great white hope” of the pharmaceutical industry in achieving three main aims: cutting clinical trial duration time, saving resources and improving data quality (4). However, Hyde concluded that RDE had failed to meet those three aims. Hyde stated that the time of the RDE paradigm was past and that the future would be shaped by new study site technologies which will increasingly provide much of the required clinical data directly, and without the need for transcription to paper and then re-entry to another system. Hyde added that direct data capture (DDC) from machines such as patient record systems, MRI machines and an increasing range of other previously manual data providers would enable errorfree and resource-efficient data capture. Hyde concluded that with DDC, substantial reduction and possible elimination of errors will allow early locking of the database and, therefore, potentially earlier product launch.

In 1999, Roberto Scognamillo et al noted that, despite the widespread use of information technology and communication advances in the pharmaceutical industry, there were still some steps within clinical operations, such as clinical data collection, that did not take advantage of the current technological revolution (5). Therefore, in order to verify whether a new technology such as the internet could both speed up data capture and improve the overall quality of data, the paper reported that the Italian Glaxo Wellcome Company and IBM had co-sponsored a pilot project called CLINical Trial & Research Management via InterNET (CLINT&RNET). Within this joint project, IBM developed the data capture internet application and Glaxo Wellcome handled the clinical and organisational aspects. The topline results of the study were recently communicated to the authors by Dr Giuseppe Recchia, Vice President, Medical & Scientific Director, GlaxoSmithKline, Spa Pharmaceuticals (personal communication). Recchia indicated that, while the project was stopped with the merger GlaxoWellcome and SmithKline Beecham, there was an enthusiastic result for centres with broadband connections in the hospital, but it was a failure for centres with low connections. Electronic data capture (EDC) is the current manifestation of RDE (6-11). While some EDC systems, like those used in paperless Phase 1 units, rely on electronic source data rather than paper records, it is still the exception for Phase 2 or Phase 3 studies to collect original electronic source data at the time of the study subject visit.


In terms of data standards, in 2006 the CDISC Electronic Source Data Interchange (eSDI) Group issued a document entitled ‘Leveraging the CDISC Standards to Facilitate the Use of Electronic Source Data within Clinical Trials’ (12). The objective of the working group was to produce a document that aligned multiple factors in the current regulatory environment and to encourage the use of eSource collection and industry data standards. The document provided a set of base regulatory requirements to assist those conducting trials using eSource data collection in their planning and execution, as well as potential scenarios that exemplified the use of CDISC data standards and appropriate processes for eSource data collection and interchange.


In 2009, Bruce Dye and Jules Mitchel co-authored a book chapter on data management in oral health research (13). The authors indicated that technology now provides numerous opportunities for DDE in clinical trials and that data can now be entered directly into the collection database at the time observations are made and responses are given. Electronic forms can be created to resemble paper forms, therefore a hard copy readable form can also be produced when needed for a study audit. The authors concluded that DDE process eliminates the need for doublekeyed entry and related tasks associated with collecting data on paper forms.

In 2010, using data from the NIH global HIV/AIDS clinical trials networks, Jonathan Kagan et al analysed the absolute and relative times required to span defined phases associated with specific activities within the clinical protocol lifecycle (14). Using simple median duration and Kaplan-Meyer survival analysis, it was demonstrated how time-based analyses can provide a rationale for the prioritisation of research process analysis and re-engineering, as well as a means for statistically assessing the impact of policy modifications, resource utilisation, re-engineered processes and best practices. According to the authors, when successfully applied, this approach can help researchers be more efficient in capitalising on new science to speed the development of improved interventions for human disease.

Recently, Vadim Tantsyura et al evaluated approaches to risk-based source data verification (SDV) (2), and Mitchel et al evaluated the clinical and statistical significance of changes to clinical trial data as a result of SDV (15). Both of these articles support RBM as well as DDE. Tantsyura’s article provided a theoretical framework for RBM while the article by Mitchel confirmed that most of the monitoring of clinical trials involves the management of transcription errors from paper source records to EDC systems. The latter article concluded that if paper records could be minimised, there could be a much greater focus placed on protocol compliance, patient protection/safety and data integrity.


Two critical EMA Reflection documents and three critical FDA Guidances have recently opened the doors to RBM and DDE (16-20). Both EMA and FDA have acknowledged that there is an increasing use of computerised systems in clinical trials to generate and maintain source data. Even the regulators have acknowledged that it should be cost-effective to replace paper quality systems which constitute a major proportion of the cost of drug and device development, and which are generally acknowledged to be inefficient and time-consuming.

As early as 2007, the FDA acknowledged that they would accept original data recorded by direct entry into a computerised system as well as accept certified copies of original data, defined as a copy of original information that has been verified, as indicated by a dated signature, as an exact copy having all of the same attributes and information as the original.

In terms of RBM, the EMA has suggested that sponsors are expected to move towards a more systematic and risk-sure that limited resources are used to address the most important issues and priorities – especially those associated with predictable or identifiable risks to the wellbeing of trial subjects and the quality of trial data. The FDA has noted that their risk-based guidance is intended to assist sponsors of clinical investigations to develop risk-based monitoring strategies and plans for investigational studies of medical products. The FDA guidance has made it clear that sponsors can use a variety of approaches to fulfil their responsibilities related to monitoring investigator conduct and the progress of investigational new drug (IND) or investigational device exemption (IDE) studies. For example, the guidance specifically encouraged greater use of centralised monitoring methods where appropriate.

In terms of DDE, the FDA has acknowledged advantages of electronic platforms including:

  • Elimination of unnecessary duplication of data
  • Dramatic reduction of transcription errors
  • Promotion of real-time entry of data during visits
  • The prompting for missing data and data errors at time of data entry and not weeks later
  • Potential interfaces with medical devices, electronic health records, laboratory data, imaging systems
  • The availability of rigorous audit trails


A survey conducted by the Clinical Trials Transformation Initiative (CTTI) indicated that a range of practices has been used to monitor the conduct of clinical trials (21). For major efficacy trials, companies typically conduct on-site monitoring visits at approximately four- to eight-week intervals, at least partly because of the perception that the frequent on-site monitoring visit model, with 100 per cent SDV, is FDA’s preferred way for sponsors to meet their monitoring obligations. In contrast, academic coordinating centres, cooperative groups, and government organisations often use on-site monitoring less extensively. FDA concluded that these examples demonstrate that the use of alternative monitoring approaches should be considered by all sponsors of clinical trials when developing risk-based monitoring strategies and planning.


In 2011, the first Phase 2 clinical trial under a US IND that used both DDE and RBM was initiated and completed. The study followed 20 subjects over approximately 12 weeks at a single centre. Data entry occurred at the time of the subject’s office visit and centralised monitoring of eCRF forms occurred within one hour of data entry. A risk-based monitoring plan was included in the clinical data monitoring plan that identified:

  • The requirement to perform a daily review of each entered eCRF
  • Which paper source document forms were to be monitored
  • The frequency of onsite monitoring
  • Actions to be taken based on the result of central and onsite monitoring

The frequency of Quality by Design (QbD) meetings to discuss findings and initiate corrective actions For daily monitoring, there were standard online EDC reports that identified:

  • The number of forms entered and monitored
  • The number and specific edit checks that fired
  • The number and specific forms with edit checks
  • Drug supply compliance

For centralised monitoring, there were online monitoring reports that:

  • Imported data directly from the EDC database, including the number of forms monitored over the monitoring period, the number and total of subjects who signed informed consent, and the subjects for whom forms were monitored
  • Identified all corrective actions and their status
  • Allowed for online monitoring reports with eSignatures for the CRA and project manager

As a result:

  • Screening errors were picked up early
  • There was no need to do additional subject recruitment
  • EDC edit checks were modified early in the clinical trial
  • Compliance issues were identified in real time
  • There was transparency of safety issues
  • There was just one on-site interim monitoring visit which took just two hours
  • The clinical site saved 70 hours of data entry time


In the not-too-distant future, with the advent of the integration of RBM and DDE, the role of data monitoring within a clinical trial will change dramatically. DDE will be an enabler of RBM since it will eliminate the need to verify the data transcription process. When data are entered directly at the time of the subject’s visit, online and realtime business logic and range checks will identify any potential data entry errors or inconsistencies. Audit trails will identify those who entered and modified data, the time and date of the modification and the reason. EDC-based query systems will be used to ask questions about the data. Online data management reports and batch edit checks will play a major role, while traditional monitoring, as we know it, will disappear. The focus of monitoring will switch from checking data transcription to a more intelligent approach. There is also the potential role of the CRA to focus on spreading best practices from trial sites with high data quality to trial sites with suboptimal performance.

We as an industry will need to decide what additional skills CRAs will need in an environment where DDE and RBM are common. Job descriptions will and must change. CRAs will become more like data managers and auditors, and will assure that the protocol is being followed and sites are properly trained. Utilising user-friendly web-based document management systems will also allow for all protocol-related study documents (eTMF) to be posted and signed online. ‘Paper trails’ as we know them will disappear, in turn being replaced by ‘electronic trails’. Furthermore, financial incentives for centralised data monitoring will include the near-elimination of costly and timely on-site monitoring visits.

In the new setting, it is important to establish where the technology enables a new way of performing oversight of CRO performance by sponsors. Risked-based online oversight activities and audits offer a promising development when there will be direct access to real-time online clinical data and study-related documents. And fi nally, another aspect to be addressed in the new setting is harmonisation of DDE and RBM among the regulatory authorities in the different ICH and ICH-like countries. Acknowledgement The authors would like to thank Joyce Hays of Target Health Inc for her insight and review of the manuscript.


  1. Korieth K, The high cost and questionable impact of 100 per cent SDV, CenterWatch Monthly, November pp1, 15-17, 2011
  2. Tantsyura V, Grimes I, Mitchel J et al, Risk-based source data verifi cation approaches: pros and cons, Drug Information Journal 44: pp745-756, 2010
  3. Herson JA, Technology assessment of remote data entry in clinical trials, proceedings of the annual symposium on computer application in medical care 3: pp837-840, 1984
  4. Hyde AW, The changing face of electronic data capture: from remote data entry to direct data capture, Drug Information Journal 323: pp1,089-1,092, 1998
  5. Scognamillo R, Strozzi C, Vincenzi B and Recchia G, Clinical trial management and remote data entry on the internet, Drug Information Journal 33: pp1,061-1,065, 1999
  6. Mitchel J, You J, Lau A et al, Paper Versus Web; A tale of three trials, Applied Clinical Trials, August, pp34-35, 2000
  7. Chadwick B and Gisanti S, EDC 2001: A (pharma) space odyssey, Innovations in Pharmaceutical Technology, pp92-95, 2001
  8. Mitchel J, You J, Kim YJ et al, Internet-based clinical trials – practical considerations, Pharmaceutical Development and Regulations 1: pp29-39, 2003
  9. Mitchel J, You J and Lau A, Clinical trial data integrity. Using internetbased remote data entry to collect reliable data, Applied Clinical Trials, March 2003 pp6-8, 2003
  10. Mitchel J, Ernst C, Cappi S et al, Meeting the challenges of internet-based clinical trials, Applied Clinical Trials, June, 2004
  11. Garvey A, EDC: The State of the art, Innovations in Pharmaceutical Technology, pp116-118, February 2005
  12. Leveraging the CDISC Standards to facilitate the use of electronic source data within clinical trials clinical data interchange standards consortium, 2006 les /0/2f6eca8f0df7caac5bbd4fadfd76d575/miscdocs/esdi.pdf
  13. Dye BA and Mitchel J, in Giannobile W, Burt B and Genco R (eds), Data Management in Oral Health Research, chapter 6: pp103-121, 2009
  14. Kagan, JM, Rosas, S and Trochim WMK. Integrating utilizationfocused evaluation with business process modeling for clinical research omprovement, Research Evaluation 19: pp239-250, October 2010
  15. Mitchel J, Kim YJ, Choi JH et al, Evaluation of data entry errors and data changes to an electronic data capture (EDC) clinical trial database, Drug Information Journal 45: pp421-430, 2011
  16. Reflection Paper On Expectations for Electronic Source Data and Data Transcribed to Electronic Data Collection Tools in Clinical Trials (EMA/INS/GCP/454280/2010), 2010, media//dokumente/Refl ection_Paper_on_e_source_data.pdf
  17. Reflection Paper on Risk Based Quality Management in Clinical Trials (EMA/INS/GCP/394194/2011), 2011, docs/en_GB/document_library/Scientifi c_guideline/2011/08/ WC500110059.pdf
  18. Guidance for Industry, Computerized Systems Used in Clinical Investigations, US Department of Health and Human Services, Food and Drug Administration (FDA), Office of the Commissioner (OC), May 2007, gdl0002.pdf
  19. Guidance for Industry Electronic Source Documentation in Clinical Investigations, December 2010, Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ UCM239052.pdf
  20. Guidance for Industry Oversight of Clinical Investigations – A Risk-Based Approach to Monitoring, August 2011 downloads/Drugs/GuidanceComplianceRegulatoryInformation/ Guidances/UCM269919.pdf
  21. Morrison B, Cochran C, Giangrandec J et al, Monitoring the quality of conduct of clinical trials: a survey of current practices, Clinical Trials 8: 342-349, 2011

Read full article from PDF >>

Rate this article You must be a member of the site to make a vote.  
Average rating:

There are no comments in regards to this article.

Jules T Mitchel is President of Target Health Inc, a full service eCRO with staff dedicated to all aspects of regulatory affairs, CMC, clinical research, biostatistics, data management and strategic planning. Jules brings over 25 years of experience in the pharmaceutical industry. He leads the team at Target Health to develop software tools to enable the ‘paperless clinical trial’. Jules has held positions at Wyeth, Pfi zer Laboratories and Pfi zer Consumer Health Care, and is currently Adjunct Professor at the Ernest Mario School of Pharmacy of Rutgers University and Adjunct Clinical Professor at SUNY Stony Brook School of Medicine. Email:

Judith M Schloss Markowitz holds a BA and an MS from Queens College of CUNY. Judith currently serves as Senior Project Manager at Target Health Inc. She oversees fi ve clinical projects that are now running concurrently. Previous to this position, she worked for various divisions of Wyeth Research in their Clinical Department for 32 years. Email:
Jules T Mitchel
Judith M Schloss Markowitz
Print this page
Send to a friend
Privacy statement
News and Press Releases

Exscientia and Sanofi Establish Strategic Research Collaboration to Develop AI-driven Pipeline of Precision-Engineered Medicines

PARIS & OXFORD, England & BOSTON--(BUSINESS WIRE)-- Sanofi and Exscientia announced today a groundbreaking research collaboration and license agreement to develop up to 15 novel small molecule candidates across oncology and immunology, leveraging Exscientia’s end-to-end AI-driven platform utilising actual patient samples. The companies have been working together since 2016 and in 2019, Sanofi in-licensed Exscientia’s novel bispecific small molecule candidate capable of targeting two distinct targets in inflammation and immunology.
More info >>

White Papers

The Key To Tableting Success: How To Choose The Right Tooling

Natoli Engineering Company, Inc.

Choosing the right tooling can increase tablet output, decrease waste and ultimately determine the success of a product launch.
More info >>




©2000-2011 Samedan Ltd.
Add to favourites

Print this page

Send to a friend
Privacy statement