home > > spring 2014 > sign of the times

Sign of the Times

Qualitative and quantitative processes for pre- and post-approval signal detection continue to evolve, with new regulations requiring more systematic and comprehensive review and data screening – both for investigational medicinal products and those which are approved/marketed.

Aside from data cleaning and coding standardisation, traditional medical review of data continues to be the core of qualitative signal detection, with quantitative data analysis feeding into the qualitative process. Qualitative signal detection involves real-time review and screening of individual case safety report (ICSR) data, along with scheduled reviews of aggregate data, in order to identify new and potential signals.

An appropriate visualisation platform can be very useful in augmenting signal detection and evaluation, by bolstering routine and ad hoc review methods through simultaneous signal screening of ICSR and aggregate data. One way the platform accomplishes this is through drill-down capability from aggregate data to ICSRs. With the right visualisation platform, one is able to instantaneously review aggregate data and ICSR data, and can more easily decipher complex patterns within a safety dataset. It is important to note that a visualisation-based application does not replace the critical need for medical and safety review of aggregate data; it merely facilitates the review and signal identification process.

Safety Signals

In the last decade, the term ‘safety signal’ has continued to be clarified in the context of signal detection, as regulators and the pharmacovigilance field make progress in the standardisation of pharmacovigilance terms. In 2002, the World Health Organization (WHO) defined a safety signal as: “Reported information on a possible causal relationship between an adverse event and a drug, the relationship being previously unknown or incompletely documented. Usually more than a single case report is required to generate a signal, depending on the seriousness of the event and quality of the information” (1).

In 2005, the US Food and Drug Administration (FDA) released the guidance document Good Pharmacovigilance Practices and Pharmacoepidemiologic Assessment, which defined a safety signal as: “A concern about an excess of adverse events compared to what would be expected to be associated with a product’s use. Signals can arise from post-marketing data and other sources, such as preclinical data and events associated with other products in the same pharmacologic class. It is possible that even a single well-documented case report can be viewed as a signal, particularly if the report describes a positive rechallenge or if the event is extremely rare in the absence of drug use” (2).

The FDA also noted that: “Signals generally indicate the need for further investigation, which may or may not lead to the conclusion that the product caused the event. After a signal is identified, it should be further assessed to determine whether it represents a potential safety risk and whether other action should be taken.”

Refined Definition

Later in 2009, Hauben and Aronson defined a signal of suspected causality as: “Information that arises from one or multiple sources (including observations and experiments), which suggests a new potentially causal association, or a new aspect of a known association, between an intervention and an event or set of related events, either adverse or beneficial, which would command regulatory, societal or clinical attention, and is judged to be of sufficient likelihood to justify verifiable and, when necessary, remedial actions” (3).

Hauben and Aronson’s definition was adopted with a slight modification in the report by the Council for International Organizations of Medical Sciences (CIOMS) (4). It has since been adopted by the European Medicines Agency (EMA) as published in their Good Pharmacovigilance Practices (GVP) Module IX – Signal Management as: “Information that arises from one or multiple sources (including observations and experiments), which suggests a new potentially causal association, or a new aspect of a known association, between an intervention and an event or set of related events, either adverse or beneficial, that is judged to be of sufficient likelihood to justify verificatory action” (5).

Therefore, the goal of any signal detection process is to support efficient review and screening of data from multiple data sources, in order to identify new safety risks (and potential risks) that meet current regulatory definition, as well as changes in the safety characteristics of a previously identified signal. This includes new risks and risk subpopulations, as well as an increase in incidence or crude reporting rates (for spontaneous events), severity, duration, reversibility, and so on.

Standardised Data

Due to the myriad of visualisation tools available, application of any such tool should be specific to, and relevant for, the dataset and emerging safety patterns being observed. An appropriate visualisation platform can accept standardised data from multiple sources and add significant value to traditional medical review in specific ways, including:

  • Quick and detailed analysis of relatively large datasets:
    Simplifying generation, review and analysis of frequencies (incidence or crude reporting rates), for quick identification of the most common events (expected pharmacology-based events), as well as unexpected idiosyncratic and rare events not previously reported for an index product
    Supporting complex safety data analysis and better accuracy in overall drug safety profile review and evaluation, including time from initial drug exposure to occurrence of an adverse drug reaction (ADR) and causality assessment
    Speeding up and improving the process of detecting potentially medically significant events or designated medical or target event
  • Supporting both proactive and reactive signal-to-risk translation, identification, characterisation, and detailed risk assessment and management

Screening data for a signal can be likened to searching for a ‘needle in a haystack.’ Consequently, identification requires design of an appropriate signal strategy and a plan that delineates a clear process. Reproducible methodology with viable tools to implement index approaches, as well as demonstrable expertise and experience, are also required. Appropriate visualisation techniques can be integrated into the design of a signal strategy to augment expert medical review.

Sources of signals in the pre-market setting include preclinical, toxicology and clinical trial data. Once a product has been approved, data sources expand to include: unsolicited sources (such as spontaneous and literature reports); large observational databases (for example, claims and electronic medical records database); internet sources, print and other media; and solicited sources, including post-authorisation clinical studies, registries, post-authorisation safety studies, risk management programmes, partner exchange of safety information, regulatory authorities, and so on. Manual qualitative review of data parameters is manageable when data volume is low. However, as data volume increases, it becomes more time-consuming and less efficient to rely solely on manual data review. The right visualisation tool supports efficient review of large volumes of data.

Visualisation Analysis

Using an appropriate visualisation tool supports not only more rapid review of data, but also assists expert reviewers in uncovering hidden patterns and trends embedded in a dataset that may represent an emerging safety concern, or valid changes to the characteristics of a known signal. Furthermore, it increases efficiency of data assessment through the support of multi-dimensional parameter review within a dataset, providing the option to present such data using different visualisation charts (treemaps and cross tables, for example), and filtering it by any parameter – such as MedDRA System Organ Class (SOC), preferred term (PT), demographic subgroup, and so on. 

A powerful visualisation tool to support in-depth review of multiple data parameters, presented within the same view, is a cross table. This is a two-way, pivot or multi-dimensional table, consisting of columns and rows which allow one to summarise and display large volumes of data in a simplified manner in one view. It also provides relational information between an index row variable and column variable (6) – for example, observed (reported) adverse events by MedDRA SOC and PT by case identity, relative to reported concomitant medications and/or medical history.

A cross table analysis can help uncover suspect and/or concomitant medication trends that may or may not be contributing to a suspected or potential signal, directly or via drug-drug interaction(s). It can also be used to generate any combination of data parameters, including a simple outcome analysis to isolate ‘death’ cases in comparison to ‘no death’ cases, segmented by MedDRA SOC and PT. A cross table such as this can also handle additional complexity by adding more data parameters and/or filtering the dataset to segregate index data for specific sub-group analysis.

Other Tools

A treemap condenses large volumes of data and splits it into a specific hierarchical structure, presented as rectangles that are sized and ordered by a quantitative variable (6). In signal detection, a treemap supports review of safety trends across an entire dataset and can also be segmented by MedDRA SOC and PT, medical history, demographic group(s), concomitant medications, and so on.

A line graph trellis is another powerful visualisation tool for performing trending examination and can be used as a time series analysis, segmented by a specific period, or cumulatively over the period represented in a dataset. This can be useful when conducting a death analysis (segmented by index MedDRA SOCs), for example.

Case Series Reviews

Visualisation methods can be used to support case series analysis, as warranted, in the course of identification and validation of a signal. Standardised MedDRA Queries (SMQs) can be used to identify the right candidates for an index case series analysis. The MedDRA Maintenance and Support Services Organization defines SMQs as: “Groupings of MedDRA terms, ordinarily at the PT level, that relate to a defined medical condition or area of interest…and are intended to aid in the identification and retrieval of potentially relevant individual case safety report.”

Case series should consist of case reports in which there is suspected exposure to the same drug and the possibility of an observed adverse event (AE)/ADR with the drug. Case series analysis requires careful review of those cases contributing to an index signal, along with a search for additional cases that meet the definition criteria of the medical term or concept. Such cases and follow-up information are to be reviewed and evaluated for clinical content and completeness, ensuring that all duplicate cases are removed (2). Case reports in a series may initially provide clues to a possible safety pattern per demographic, or another risk group for an AE/ADR which is a safety concern.

An appropriate visualisation platform facilitates conduct of a case series analysis by enabling the expert reviewer to quickly identify data flaws and constraints, including incomplete and inconsistent data, duplicate reports and coding errors. Templates can be set up in advance to factor in:

  • Minimum considerations in evaluating causal relationships across the case series
  • Time-to-onset of an event
  • Biological and pharmacological plausibility, based on mechanism of action
  • Treatment-emergent trend of the event
  • Consistency with known class-specific event
  • Supporting legacy data from preclinical and toxicology studies
  • Clinical trials and pharmacoepidemiological studies
  • Event resolution following dechallenge
  • Lack of other possible contributory factors, such as concomitant medications and prior or concurrent medical history (2,4)

Analysis templates can also be set up within a visualisation platform to support summary descriptive analysis, including:

  • Demographic analysis
  • Dosage analysis
  • Different routes of administration and lot/batch analysis relative to the index event, where applicable
  • Duration of patient exposure to the suspect medication
  • Time of initial exposure to onset of adverse event
  • Distribution of reported concomitant medications
  • Frequency (incidence or crude reporting rates)
  • Trending analysis of occurrence of events over the period represented in the dataset (2,4)

An appropriate visualisation platform can be used to augment and facilitate pre- and post-market data exploration, and, therefore, can be a powerful way to support multiple safety-related data review and assessment activities, including the following:

  • Conducting signal detection workshops (preclinical):
    – Data review/data analysis
    – Toxicological data to potential signal translation
  • Signal detection, evaluation and management (preclinical through to post-approval):
    – Development of signal strategy and plan
    – Pre- and post-market qualitative and quantitative data review (traditional medical review) and analysis
    – Generation of signal detection report
    – Generation of signal assessment report
    – Signal tracking and management
  • Post-data mining of spontaneous data from the FDA, Freedom of Information Act, Adverse Event Reporting System or WHO Vigibase (post-approval):
    – Expert review of statistical data mining output from application of statistical data mining algorithms
    – Concurrent qualitative data review of ICSRs, contributing to statistically significant scores (review and assessment in clinical context)
  • Medical error review and analysis
  • Off-label use data review and analysis
  • Electronic health data extract review and analysis

Keys to Success

Innovative visualisation-based platforms can effectively support comprehensive and efficient analysis of data during scheduled and ad hoc safety data review; ICSR and cumulative data review; medication error analysis; overdose; off-label use; misuse, abuse and occupational exposure analysis; and any other form of aggregate data review and quantitative analysis. Furthermore, the visualisation platform can be used to augment signal validation, signal-to-risk translation, risk characterisation benefit-risk analysis, development of aggregate reports, and risk management programmes. Three keys to success when implementing signal detection include starting with:

  • Appropriate data that is relatively clean and de-duplicated (especially when using post-market spontaneous data)
  • High-quality standardised coding
  • Normalisation to meet the standardised requirements of data tables used in the signal detection process
  • A sound strategy, plan and core medical review process

In addition, the following are essential elements that should be integrated into the overall signal detection strategy development:

  • Availability of appropriate legacy data (preclinical and clinical especially), and current reference safety information (RSI) of the index suspect product under evaluation
  • Verified source of current RSI for any reported concomitant medication
  • Well-designed, tested and validated literature search and review process
  • Appropriate threshold algorithms and application schema
  • A reliable, authenticated source for epidemiology and natural history background data

With these critical elements in place, a solid basis is formed for validation of the findings resulting from signal detection analysis and further risk assessment (2,4,5-7).

1. Safety of medicines: a guide to detecting and reporting adverse drug reactions, WHO, 2002. Visit: http://whqlibdoc.who. int/hq/2002/who_edm_qsm_2002.2.pdf
2. Guidance for Industry: Good Pharmacovigilance Practices and Pharmacoepidemiologic Assessment, FDA, March 2005
3. Hauben M and Aronson JK, Defining ‘signal’ and its subtypes in pharmacovigilance based on a systematic review of previous definitions, Drug Safety 32(2): pp99-110, 2009
4. Practical Aspects of Signal Detection in Pharmacovigilance, CIOMS Working Group VIII, 2010
5. GVP Module IX – Signal Management, EMA
6. Visit: spotfire_client_help/intro/ intro_user_interface.htm
7. Guidance for Industry: Premarket risk assessment, FDA, March 2005

Read full article from PDF >>

Rate this article You must be a member of the site to make a vote.  
Average rating:

There are no comments in regards to this article.


Dr Jacinta Aniagolu-Johnson, the Senior Director of Safety and Risk Management (Americas) at PRA, is a US-trained scientist with expertise in molecular immunology. She holds a doctorate degree in Medical Microbiology with specialisation in Molecular Immunology from Howard University College of Medicine, Washington DC. With over 18 years of experience, her professional career has spanned scientific and biomedical research, teaching, technical consulting for the United Nations Development Program on HIV research, and clinical research and drug safety in oncology, CNS and infectious/ respiratory diseases. In academia, Jacinta performed immunology-based cardiovascular research and was part of a team that developed a novel, experimental anti-cholesterol antibody detection assay.

Dr Jacinta Aniagolu-Johnson
Print this page
Send to a friend
Privacy statement
News and Press Releases

Byondis’ Investigational Antibody-Drug Conjugate SYD985 Selected for I-SPY 2 TRIAL™ in Early-Stage Breast Cancer

Byondis B.V. (formerly Synthon Biopharmaceuticals) today announced that Quantum Leap Healthcare Collaborative™ (Quantum Leap) selected the company’s investigational antibody-drug conjugate (ADC) SYD985 ([vic-]trastuzumab duocarmazine) for a new investigational treatment arm in its ongoing I-SPY 2 TRIAL™ for neoadjuvant treatment of locally advanced breast cancer.
More info >>

White Papers

The Emerging Use of Social Media in Orphan Disease Drug Research

Worldwide Clinical Trials Ltd

The use of various social media platforms, such as Twitter, Instagram, Facebook, and LinkedIn, for dissemination of clinical trial information has created a cornucopia of possibilities enabling clinical trial performance. Characterized by a people-to-people immediacy, social media permits extraordinarily adaptive modification of study information in real-time, and it does so in a highly targeted fashion.
More info >>

Industry Events

World Vaccine Congress Washington

27-29 September 2020, Walter E Washington Convention Center, Washington, US

The World Vaccine Congress is an award-winning series of conferences and exhibitions that have grown to become the largest and most established vaccine meeting of its kind across the globe. Our credibility is show through the prestigious scientific advisory board that spend months of hard work creating a new and topical agenda, year on year.
More info >>



©2000-2011 Samedan Ltd.
Add to favourites

Print this page

Send to a friend
Privacy statement