home > ict > spring 2013 > epro update
International Clinical Trials

ePRO Update

Patient-reported outcomes (PROs) and their use as clinical trial endpoints have received increased attention. This was catalysed in 2006 with the release of draft guidance on this subject by the Food and Drug Administration (FDA), and in 2009, when the FDA released the final version of this guidance, Patient-reported outcome measures: Use in medical product development to support labelling claims, which outlines the process for using PRO measures as effectiveness endpoints in clinical trials (1).

PRO Instrument Use Decline?

In the US between 1997 and 2002, PRO instruments were reported in 30 per cent of the product labels of new products approved by the FDA (2). A more recent study reported that 24 per cent of product labels approved between 2006 and 2010 included PRO instruments – a six per cent decline between the two periods (3).

However, there are a few important differences between these two reviews. Firstly, the FDA’s Center for Drug Evaluation and Research approved more than 10 per cent fewer drugs in 2006-2010 (156 approvals) compared to 1997-2002 (172 approvals). Secondly, the more recent study, by Gnanasakthy et al, reviewed 99 fewer new molecular entities (NME) compared to the earlier research by Willke et al, or 116 NMEs versus 215 NMEs respectively.

These differences are primarily due to the inclusion or exclusion criteria that each review team employed. For example, the earlier study included 45 new drugs and vaccines approved by the Centre for Biologic Evaluation and Research (CBER). It does not appear that the later study considered drugs approved by CBER. As a result, the comparison and interpretation of these data may not be as straightforward as it appears. However, the data do provide evidence that the patient’s voice plays an important role in the evaluation of new medical products.

ePRO Platforms

Many PRO instruments were designed to be administered via paper-and-pencil, but electronically adapted PROs (ePROs) are now available. These have the advantages of less administrative burden, avoidance of secondary data entry errors, easier implementation of skip patterns, and more accurate and complete data (4-7).

There are a variety of electronic platforms available for the adaptation of paper-based PRO instruments. These primarily consist of screen-based platforms – for example, tablet computers – and auditory platforms, such as interactive voice response (IVR) systems.

Screen Technology:
Screen-based platforms adapt the original paper-based version of the instrument to a computerised version of the questionnaire’s items and responses. These platforms include desktop, laptop and tablet computers, handheld or palm computers and web-based systems. Touchscreen applications have become increasingly common with the introduction of new cellular phone and tablet technology.

When considering the implementation of screen-based devices in a study, investigators should consider the tradeoff of device mobility or storage relative to the device screen size. For example, handheld computers are lightweight and portable, but the limited screen space can be a drawback. This may require the respondent to scroll to view the entire question and response set. It also limits the use of larger, easier to read fonts. However, the portability of the handheld computer means it is potentially more useful for real-time assessment of the patient experience, such as event-based diaries.

IVR Systems:
Auditory devices provide the respondent with an audio version of the questions and response choices. Specifically, IVR platforms are telephone-based systems that interact with callers using a pre-recorded voice question and response script. No additional hardware is required for the respondent other than a telephone; little if any respondent training is necessary; data are stored directly to the central database; and IVR systems can record voice responses. The use of the recorded voice prompts has been shown to reduce the literacy skill requirements of study participants (8,9).

When choosing among the different ePRO platforms, one should consider the type of response sets contained in the PRO measure being adapted, any skip logic, the recall period, and the target population.

Good Practice Recommendations

There are specific issues to address when implementing a paper-based PRO instrument on an electronic platform. Based on the FDA’s PRO guidance, an instrument that has been adapted to an electronic platform is considered different from the original instrument, thus requiring additional evidence of validity and reliability. However, the guidance does not outline the specific steps that must be taken to validate an ePRO instrument for use as an endpoint in a clinical trial.

To fill this void, the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) ePRO Task Force was commissioned to draft a report addressing the necessary steps to assure the adequacy of a PRO that has been adapted to an electronic system. The resulting good practice report presents recommendations for the evidence necessary to support the equivalence of ePRO instruments relative to the paper-based PRO instruments from which they were adapted (10).

Adaptation Answers

The Task Force recommended that the amount of change that is made to adapt a paper-based instrument to an electronic platform dictates the evidence necessary to have confidence in the equivalence of the scores resulting from the new mode. For minor changes – for instance, simply placing the items from a paper-based instrument into a screen text format without changing font size, item content or response options – small-scale (5-20 subjects) cognitive interviewing studies may be conducted. Moderate levels of change, such as migration to an IVR system (that is, moving from a visual process to an auditory process), may require quantitatively testing the equivalence of scores obtained via the two modalities.

When substantial modifications are made, such as changing the content or meaning of a measure, the content validity and psychometric characteristics of the new measure must be established de novo. Overall, the ISPOR ePRO Task Force’s report fills some of the gaps created when the FDA issued its guidance.

In addition, to facilitate the creation and dissemination of evidence supporting the use of ePRO instruments, the Critical Path Institute established the ePRO Consortium in 2011. This comprises seven firms that provide electronic data collection (EDC) technologies for capturing PRO endpoints in clinical trials. These non-traditional collaborators work together to advance the quality, practicality and acceptability of EDC methods for endpoint assessment. Best practices for adapting paper-based PRO measures, and for the development of new ePRO instruments, have been created by the Consortium in an effort to drive improvements in the consistency and quality of the data collected.

Label Rejections

Despite the environmental externality that may result from collecting data electronically, there are many benefits provided to clinical studies that incorporate ePRO instruments. One study examined the reasons for the rejection of PRO label claims (11). Study design, data quality and interpretation issues accounted for 27 per cent of rejections, where reviewers questioned the clinical meaningfulness of patient responses, noted issues of bias introduced by open-label study designs, and commented on missing data or dropout rates and other indicators of data quality. In 10 per cent of the rejections, administrative considerations impacted the FDA’s review decision, including lack of appropriate documentation describing training procedures, administration of the tool, and inadequate descriptions of measures.

We do not know whether the inclusion of ePRO measures in these trials would have changed the regulatory decision. However, technological advances have clearly increased the control over study variables, enhancing data collection and interpretation in clinical trials. New PRO measures are being developed to utilise the capabilities provided by electronic platforms. Some of these instruments cannot be effectively deployed on paper because they are event-based, and the concatenation of items requires programmatic solutions.

Improving Science

While the advances in data collection have created new challenges, the science is improving rapidly. Efforts aimed at enhancing the methodological underpinnings of ePRO instruments will facilitate the widespread adoption of electronic data capture. By increasing the awareness and adoption of collecting patient data, and collecting it electronically, we open new possibilities for developing effective treatments in challenging disease areas.

Read full article from PDF >>

Rate this article You must be a member of the site to make a vote.  
Average rating:

There are no comments in regards to this article.

J Jason Lundy is Director of the ePRO Consortium and Assistant Director of the PRO Consortium at the Critical Path Institute. He is responsible for advancing and documenting the science of electronic data collection of PRO endpoint measures in clinical trials and overseeing the migration of PRO instruments to multiple ePRO platforms. J Jason holds a PhD in Pharmaceutical Economics, Policy and Outcomes Research from the University of Arizona, College of Pharmacy.
J Jason Lundy
Print this page
Send to a friend
Privacy statement
News and Press Releases

Sphere Fluidics closes a $40 million funding round led by Sofinnova Partners and Redmile Group

Cambridge, UK, 28 October 2021: Sphere Fluidics, a company that has developed and is commercialising single cell analysis systems underpinned by its proprietary picodroplet technology, announced today that it has closed a $40 million (circa £30 million) investment round. The round was led by Sofinnova Partners (Paris, France) and Redmile Group (San Francisco, USA) co-investing on equal terms.
More info >>

White Papers

Systems Engineering for Complex Portable Medical Device Development


As the demand for complex, portable medical devices continues to grow, reducing risk and increasing efficiency during the development of these products should be paramount. Taking a systems-engineering (SE) approach to development provides a holistic, organized, and deliberate method for identifying as well as reducing both patient and business risks early in the process. Furthermore, it facilitates efficient progression throughout the entire product development life cycle.
More info >>




©2000-2011 Samedan Ltd.
Add to favourites

Print this page

Send to a friend
Privacy statement