home > ict > autumn 2011 > a measured decision
International Clinical Trials

A Measured Decision

Keith Meadows of DHP Research & Consultancy Ltd considers the process of selecting appropriate patient reported outcome measures, and discusses some of the conceptual issues involved

In recent years there has been a growing prominence of patients’ involvement in the care that they receive. The result of this is that the assessment of outcomes, based on their perspective from using patient reported outcome (PRO) measures, has been increasingly used alongside the traditional clinical ways of measuring health and the effects of the treatment on the patient.

The therapeutic marketplace is becoming saturated with multiple medications, and where differentiation in clinical outcomes is often unclear or minimal and drugs are starting to lose their patent protection, data derived from PROs can provide added value in supporting key biomedical endpoints, reimbursement, product differentiation and marketing.

It is important to select an appropriate PRO, and so far the focus has largely been on the qualities of the PRO in terms of psychometric properties, ease of administration, scoring and interpretation of scores. However, what is particularly significant is the development of an effective measurement strategy which encompasses a framework for selecting the appropriate PRO for hypothesis generation and testing, analysis and interpretation of the data.

This article briefly discusses some of the key issues to be considered in selecting an appropriate PRO, including the development of an effective PRO measurement strategy, defining the PRO concept, the importance of a conceptual model and conceptual framework and, finally, the essential psychometric requirements of a PRO.


PROs are those outcomes known only to the patient. Subjective in nature, PROs are the patient’s perceptions of the impact of the disease and treatment. They can include the reporting of symptoms, health status, health-related quality of life (HRQoL), well-being and satisfaction with care received. PROs provide a unique insight into the patient’s evaluation of the benefits of specific treatments, their side effects, their biological and physiological dysfunctioning, symptoms, and functional impairment on personal and family life.

Clinical outcomes such as an HbA1c for a patient with diabetes, or the peak flow rate of an asthma sufferer, are useful outcomes in terms of treatment effectiveness.

However, as indicators of the impact the disease might be having on the patient’s quality of life (QoL), clinical outcomes alone do not provide the full picture. For example, they do not take into account the restrictions on the diabetic’s social activities due to a fear of going into a hypoglycaemic coma.

The Food and Drug Administration (FDA) PRO guidance for the industry describes a PRO as “…any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else. The outcome can be measured in absolute terms (for example, severity of a symptom, sign, or state of a disease) or as a change from a previous measure. In clinical trials, a PRO instrument can be used to measure the effect of a medical intervention on one or more concepts…” (1).

The complexity of the PRO concept can range from the uni-dimensional, such as pain, to the multidimensional, such as health status. It focuses on the quality of health and may include biological and physiological dysfunctioning, symptoms and functional impairment, such as the ability to walk around a block or climb a flight of stairs.

PRO concepts can be measured generically, and are therefore of relevance to a wide range of patient groups and the general population. As such they can be used for comparison across different disease groups as well as with healthy populations. However, due to the generic nature of the content, they will most likely include items that are irrelevant to many patient and disease groups, as well as excluding content of particular relevance to a specific disease group.

In contrast, disease-specific or condition-specific PROs have been developed to capture those PRO concepts of importance to a specific patient group. Therefore, they are more likely to be clinically relevant and sensitive to detecting clinically important changes in the measured PRO concept.


Selecting the appropriate PRO for a specific study requires consideration of a number of factors, including: the study population; study objectives and design; and the PRO measurement properties.

The FDA PRO guidance for the industry highlights the key requirements of any PRO instrument “…whether existing, modifi ed, or newly developed, as a measure to support medical product labelling claims depends on whether its characteristics, conceptual framework, content validity, and other measurement properties are satisfactory” (1). With literally hundreds of PROs claiming to measure a range of health concepts – including symptoms and satisfaction, QoL, health status, and well-being – all with varying degrees of reliability and validity – selecting the appropriate PRO as part of an effective measurement strategy can be a major challenge for the outcomes team.

For an effective PRO measurement model, it is essential that the PRO concept is adequately measured to enable the interpretation of scores and understanding of the fi ndings (2). Often, however, the choice is based on whether the PRO has been used in previous studies, or if the name of the instrument appears to be appropriate for its intended use (3).

However, it is not uncommon that in the PRO selection process, the more commonly measured health concepts – for example, QoL, HRQoL, health status and well-being – are used interchangeably. Speight et al, in their review of the 10 most frequently used instruments to measure QoL in diabetes since 1995, found only three that actually could be considered as measuring QoL (4).

While there is no universally accepted defi nition of QoL, there is the general consensus that it is based on the individual’s subjective evaluation of the psychological, physical and social aspects of their life, and is changing over time as a result of different infl uences such as treatment. It is the extent to which the treatment, management or living with a disease or condition is perceived as affecting the individual’s life where issues such as family life, work and school are an important priority in addition to health (5).

The measurement of HRQoL is often referred to as the degree to which the treatment and the disease are perceived by the individual as affecting those areas of their life – in addition to their health – which are considered important (4).

In contrast, measures of health-status such as the SF-36 and EQ-5D, which are in common use, focus on the quality of health such as functional impairment and symptoms rather than QoL (6-7). Though health status can have a significant effect on the individual’s QoL, the concept is not interchangeable with QoL.

Measures of psychological well-being focus on those aspects of psychological illness including anxiety, depression, coping, positive well-being and adjustment, as well as a sense of control and self-esteem. Typical measures of well-being include the Beck Depression Inventory (BDI) and the Hospital Anxiety and Depression Scale (HADS) (8-9). Although related to the patient’s QoL, psychological well-being is not synonymous with it.

It is a central aspect of the study design to ensure both that the appropriate PRO concept(s) is selected and that it is adequately measured to ensure meaningful and insightful interpretation of scores. Central to this decision-making process is the conceptual model and framework.


A conceptual model can be described within the context of FDA regulatory decision-making as a model, which identifi es and describes the PRO concepts and hypotheses underlying a PRObased product labelling claim (2). More generally, and guided by the specific nature of the disease or health problem, a review of the scientific literature, study objectives and input from key opinion leaders and patients, a conceptual model identifies the interrelationship between concepts and outcomes. An example would involve HRQoL, pain and physical functioning that are considered to be related to a specific disease/ condition or health problem.

A conceptual model is valuable in terms of its ability to facilitate the exploration of a disease area, identify potential treatment benefits as well as the selection of endpoints to evaluate them and, more importantly, the selection or development of a specific PRO instrument (2).

Figure 1 shows the conceptual model proposed by Wilson and Cleary, in which they defi ned HRQoL as ‘…a taxonomy of patient outcomes…’, comprising five levels: (i) physiological factors, (ii) symptom status, (iii) functional health, (iv) general health perceptions and (v) overall quality of life (3,10). Figure 2 illustrates a specific model of the relationship between symptoms of hypoglycaemia in Type 1 diabetes and HRQoL. From this model we are able to identify which of its elements are likely to be impacted by a specific treatment, thus enabling operational meaning to be given to a range of hypotheses relating to treatment effects on symptom reduction and subsequent impact on the individual HRQoL domains. Having hypothesised which HRQoL domains would benefit from the treatment effects, the process of selecting an appropriate PRO can commence.

It is important to note that the conceptual model as depicted in Figure 2 is similar to that defined by the FDA in their guidance for industry as an endpoint model. The endpoint model, however, differs in so much that it is “a diagram of the hierarchy of relationships among all endpoints, both PRO and non-PRO, that corresponds to the clinical trial’s objectives, design and data analysis plan” (see Figure 3) (1).


Arguably item content and aggregation into specific domains are the most important qualities of a PRO, as postulated hypotheses or targeted labelling claims should be based on specific PRO concepts. The grouping and scoring of items into specific domains can be represented by a conceptual framework, which is a diagram that presents a description of the relationships between items, domain and the concepts measured by the PRO instrument and the scores produced by the instrument (see Figure 4).

Created during the development of the PRO, a conceptual framework is based on an extensive review of the diseasespecific literature, input from health professionals, clinicians and patients, as well as an iterative process of item reduction and validation using psychometric procedures.

The importance of a well-conceived conceptual framework in the PRO selection process is that it can support the rationale for instrument selection – based on its item content and aggregation – in relation to a specific hypothesis or product claim. Consider, for example, the claim that ‘Treatment A compared to Treatment B reduces the level of diabetes-related activity restricting anxiety in patients with Type 1 diabetes’. A well-conceived conceptual framework may include items such as worrying thoughts about acute and long-term complications, patient perceived behavioural restrictions in social activities, and anxiety reducing behaviour.

In contrast, a conceptual framework that also includes the symptoms of poor diabetes control – which can result in raised levels of anxiety – could affect the quality of the analysis in accounting for the treatment benefits on reducing anxiety. Both symptoms and anxiety restricting behaviour items must be grouped and scored separately in order to determine treatment effects.

The problems created by the absence of a well-conceived conceptual framework in selecting a PRO have been comprehensibly detailed elsewhere (2). This includes: a lack of clarity in the grouping of items into domains and scoring; analysis, such as combining symptoms and psychological distress that can mask the treatment benefits on both; and limiting the interpretation of the PRO scores as it is unclear what these scores represent.


Demonstrable evidence that the selected PRO has been developed with scientific rigour and satisfactory psychometric properties is essential for an effective PRO measurement strategy, in which our understanding of the meaning of the PRO data is maximised.

Evidence of a PRO’s validity and reliability should not be seen in isolation from the preceding discussion, but instead as central to the measurement strategy. For example, it is essential that there is supportive evidence that the grouping of items, within a given PRO domain as depicted in the conceptual framework, is based on appropriate methodological procedures such as confirmatory factor analysis or Rasch analysis.

Not only is it important that the correct methodologies are applied, but empirical evidence that the specific domains are measuring what they are claiming to measure (that is, construct validity) is needed. The extent to which the PRO scores are related to a known measure of the same concept (criterion validity) is also significant.

In terms of reliability, the PRO instrument should demonstrate the ability to provide consistent, reproducible estimates of true treatment effect such as test-retest reliability. Consistency in agreement among responses to different questions within a given domain (internal consistency) also needs to be shown.

A PRO instrument should also demonstrate its ability to detect a change in scores as related to change in other similar measures, which indicate that the patient’s state has changed with respect to the concept of interest.

A more detailed discussion on psychometric theory and the required psychometric properties of instruments is beyond the scope of this article and the interested reader is referred elsewhere (11,12).


In addition to the satisfactory psychometric properties as an essential requirement in the selection process of a PRO for a specific study, there are a number of key conceptual issues that need to be considered to aid the selection of an appropriate PRO in order to enhance our understanding and interpretation of PRO data. The significance of the conceptual model was highlighted in terms of its ability to identify potential treatment benefits, enable operational meaning to be given to a range of hypotheses relating to treatment effects and, importantly, the selection or development of a specific PRO.

The importance of a well-conceived conceptual framework within the context of selecting a PRO selection process in supporting the rationale for PRO instrument selection was also emphasised, as were the problems created by the absence of a framework. These included a lack of clarity in the grouping of items into domains and scoring; analysis; and limiting the interpretation of the PRO scores.

Incorporation of the patient’s own perspective of their illness and the effects of treatment is also imperative in the evaluation of treatment effects, and as more information becomes available, our understanding as to what to measure and how to measure must evolve (2).


  1. Food and Drug Administration (FDA), Guidance for Industry, Patient-reported outcome measures: use in medical product development to support labelling claims, Rockville, MD, 2009
  2. Rothman ML, Beltran P, Capperlleri JC, Lipscomb J and Teschendorf B, The Mayo/FDA patient-reported outcomes consensus meeting group, patient-reported outcomes: conceptual issues, Value in Health 10: Supp 2, S66-S75, 2007
  3. Polonsky WH, Understanding and assessing diabetes-specific quality of life, Diabetes Spectr 13: p36, 2000
  4. Speight J, Reany MD and Barnard KD, Not all roads lead to Rome – a review of quality of life measurement in adults with diabetes, Diabetic Medicine 26: pp315-327, 2009
  5. McGee HM, O’Boyle CA, Hicky A, O’Malley K and Joyce CR, Assessing the quality of life of the individual: the SEIQoL with a healthy and gastroenterology unit population, Psychol Med 21: pp749-759, 1991
  6. Ware JE and Sherbourne CD, The MOS 36-item Short Form Health Survey (SF-36), I Conceptual framework and item selection, Med Care 30: pp473-483, 1992
  7. EuroQol Group, EuroQol – a new facility for the measurement of health-related quality of life, Health Policy 16: pp199-208, 1990
  8. Beck A, Ward C, Mendelson M, Mock J and Erbaugh J, An inventory for measuring depression, Arch Gen Psychiatry 4: pp561-571, 1961
  9. Zigmond AS and Snaith RP, The hospital anxiety and depression scale, Acta Psychiatr Scand 67: pp361-370, 1981
  10. Wilson IB and Cleary PD, Linking clinical variables with health related quality of life, JAMA 273: pp59-65, 1995
  11. Nunnally JC, Psychometric Theory, 3rd edition. New York: McGraw Hill, 1994 12. Loewenthal KM, An Introduction to Psychological Tests and Scales, London: UCL Press Limited, 1996

Read full article from PDF >>

Rate this article You must be a member of the site to make a vote.  
Average rating:

There are no comments in regards to this article.


Keith Meadows is Founder and Director of DHP Research & Consultancy Ltd. He has over 25 years of experience in health-related research with a focus on the development and implementation of patient reported outcome measures. His particular interest is in the measurement of quality of life and health outcomes of patients with diabetes. Keith has published widely and has undertaken research across much of Europe. He is author of the Diabetes Health Profi le (DHP), a diabetes-specific patient reported outcome measure designed to assess the quality of life of patients living with diabetes. Email:

Keith Meadows
Print this page
Send to a friend
Privacy statement
News and Press Releases

Key milestone in molded glass: SGD Pharma elevates Type I offering with three tailored product and service solutions

Puteaux Cedex, FRANCE – 23rd March: SGD Pharma, a leading supplier of glass primary packaging in the pharmaceutical industry and global expert in Type I molded glass, introduces an industry first offering of Type I molded glass vials. These market-driven value propositions represent the latest step in SGD Pharma’s ongoing improvements to its product offers and services. According to their specific business and parenteral drug requirements, Pharma and Biotech players would not only benefit from SGD Pharma’s enhanced high quality and reliable products, but also from an extended offer, ranging from Total Cost of Ownership (TCO) optimization to the highest levels of services.
More info >>

White Papers

Bio/Pharmaceutical Methods: Do your Analytical Methods Hold Up to Regulatory Scrutiny?

Eurofins BioPharma Product Testing

As FDA guidelines evolve and drug products on the market begin to age, bio/pharmaceutical manufacturers face scrutiny of the original methods used to support these products. Therefore, manufacturers must re-evaluate these methods to ensure they comply with current FDA expectations. Some manufacturers are even facing consent decrees imposed by the agency requiring them to bring methods up to current standards within a specified timeframe. This exercise requires significant amounts of time and resources, which many manufacturers do not have since they are focused on getting their next products to market. For that reason, some clients have been turning to Eurofins Lancaster Laboratories for support.
More info >>




©2000-2011 Samedan Ltd.
Add to favourites

Print this page

Send to a friend
Privacy statement