Biomarker discovery is not straightforward. Extra attention at the study design stage can make the development process much more efficient, realising the potential of these valuable tools.
There is currently an unprecedented amount of interest in the use of genomic biomarkers in the field of personalised medicine. These biomarkers, which can range from single genes up to multivariate gene signatures, are now routinely being developed into clinical tests to stratify patient populations. The scope of these tests is wide reaching, including diagnostic, prognostic and predictive assays (see Table 1). Despite their potential for use in clinical tests, the vast majority of biomarkers never make it to the clinic, primarily due to shortfalls in the biomarker discovery process.
Biomarker Discovery and Product Development
The discovery of a biomarker and its subsequent development into a clinically validated test is a complex process requiring input from multiple teams including laboratory staff, bioinformatics, biostatistics, project managers, medical writers, software developers, QA, QC, manufacturing, regulatory personnel and clinical trial experts. Effective communication between these teams is central to the success of a biomarker discovery and development programme.
Biomarker Discovery
This generally consists of three phases: study design, sample processing and data analysis. The study design phase is instrumental to the success of any biomarker development programme, as mistakes made here will have wide ranging implications throughout the entire development process, and may ultimately lead to the failure of the biomarker.
Study design requires input from all members of the project team, especially bioinformatics and data analysis personnel. If the study design is not carried out correctly, the right questions will not be asked, and poor or misleading answers will be obtained. At this stage, several biological and technical factors need to be taken into account.
Definition of the Scope of the Test
Multiple questions need to be answered including: will the test be prognostic or predictive; which disease type will it be used for; what tissue type it will be used on; what technology will it use; will it be a kit or a lab based test; and so on. Defining these factors early in the study allows the discovery process to be tailored to mirror the scope of the proposed product. Failure to address these questions may lead to changes in the product scope midway through the process. At the least, this will add time and additional cost to the process. At worst it will lead to complete failure of the biomarker validation and perhaps even failure of the associated product.
Inclusion and Exclusion Criteria
If these criteria are not correctly defined, biomarker discovery may be carried out using unsuitable samples. Ideally biomarker discovery should use similar samples (matrix and dose levels of drug, for example) to those that will be used in the clinic. If this is not feasible, a migration study may be required prior to validation of the final test (see Product Development, page 96).
Define Clinical Information Requirements
If insufficient clinical data are collected, it will be difficult to carry out the study design or data analysis. For example, if gender information is not collected, it’s possible that all the ‘responding’ patients in a study could be male, whereas all the ‘non-responding’ patients are female. Subsequent data analysis may identify gender-specific genes as those that identify drug response, whereas in fact this may have been caused by poor study design. Even more worrying is that this error may go unnoticed until the clinical validation phase of product development.
Definition of Endpoints
It’s important to define which factor(s) will be used to stratify patients. For example, tumour response data can be used to identify biomarkers that predict response to treatment. However, tumour response can be measured in a variety of ways and the definition of ‘responder’ and ‘non-responder’ patients can vary between different diseases. The endpoint used will also dictate the type of biomarker that can be produced. For example, if you want to develop a biomarker to identify which tumours respond to treatment, using diseasefree survival as an endpoint will not be suitable (this would produce a prognostic biomarker).
Determination of Sample Size
This is probably the most common error made in biomarker discovery studies and is caused by the inevitable trade off between the cost of the study, timelines and the quality of the end product. Generally, too few samples are analysed, leading to an over-estimation of the biological significance of the biomarker. Consequently, the performance figures of the proposed test will be artificially high or statistically unsound, which can lead to a failure of the validation process. .
Randomisation
A vital component of the study design is the randomisation of samples for processing. This ensures that any observed differences between the sample groups are due to differences in treatment alone, not the effects of confounding variables or sample bias. This also allows the minimisation of batch effects caused by different operators or reagent batches.
Balancing
During the study design it’s also important to balance sample groups with respect to certain clinical factors, for example ensuring equivalent numbers of male and female patients are in the ‘responder’ and ‘non-responder’ groups. Failure to correctly balance samples will lead to a bias in the data analysis results.
A successful study design will maximise the chances of finding a robust biomarker. Once biomarker discovery is complete, the marker can transition into the product development phase, during which the marker is developed and validated as a clinical test.
Product Development
This can be considered as a two stage process: product design finalisation (pre-analytical stage), and validation and verification. The likelihood of failure at each of these stages is dramatically increased if a poor biomarker discovery process has been carried out. These problems may be compounded if biomarker discovery and product development are carried out by two separate organisations/companies due to the high risk of communication issues during project transfer. If biomarker discovery and product development are not carried out by a single organisation, the chances of success can be maximised by involving members of the product development team during the discovery process.
The initial product design finalisation stage comprises all the steps necessary to achieve a ‘product lock’, that is to have a finalised product that can enter verification and validation. This can consist of multiple steps depending on the nature of the test being developed, including:
Platform Migration
This is required if, for example, biomarker discovery was carried out using microarrays, but the clinical test is being developed as a qPCR test. Alternatively, the biomarker may have been developed using fresh frozen tissue and then migrated into a test that can be used with formalin fixed, paraffin embedded tissue. Depending on the nature of these studies, they may represent a brand new biomarker discovery study, and should be treated as such.
Design Control
The FDA requires that medical device manufacturers wanting to market certain types of products in the US follow specific design control requirements, which are a formalised set of steps focused on managing the design of a product. As such, design control represents a valuable tool in the product development process.
Instructions for Use
While unlikely to cause the failure of a product, the production of these documents is aided by defining the scope of the product early in the process.
Reference samples
Positive/negative control samples are generally supplied as part of the final product (these are also used for analytical validation studies). Generation of these samples requires test thresholds to be accurately defined during biomarker discovery or product design finalisation.
Software Development
This may also be required, depending on the nature of the test and the technology platform used.
Kit Manufacture
This is generally carried out under ISO13485 depending on the intended regulatory route for the test.
Once the product has been finalised, it can be taken forward into analytical and clinical validation. The type and level of validation required will vary depending on the intended use of the test. If the test is being developed for a CLIA lab, no clinical validation is required, only analytical validation. However, if the test is to be used as an on-label companion diagnostic, much more in-depth analytical and clinical validation studies are needed.
Analytical validation is required to show that the test is robust – that the same result will be produced using different operators, reagents, labs and equipment. These studies generally include precision, specificity, LoD, LoB and LoQ, reportable range, accuracy and standard deviation. Failure during analytical validation can be caused by a multitude of factors, for example lot-to-lot variations of reagents leading to inconsistent results when processing identical samples. Early detection of these issues can be carried out alongside the biomarker discovery process, allowing future migration studies to be carried out using a suitable kit.
Clinical validation is required to show that the correct result is produced by the test – that a ‘positive’ patient produces a ‘positive’ result. These studies may be retrospective or prospective, but will generally require a purposely designed clinical trial. As previously discussed, many factors can be responsible for the failure of clinical validation studies, most of which can be avoided by carrying out a few additional steps during the planning phase of the study.
Conclusion
In recent years there has been a huge amount of interest in using biomarkers for clinical applications. Numerous high profile publications and the withdrawal of several biomarkers from clinical validation have highlighted the importance of the study design process, particularly in biomarker discovery (1,2). This has led to an improved understanding of the need for biomarker studies to be properly designed and implemented, particularly amongst large CROs and diagnostics companies.
This will ultimately lead to an increase in the number of candidate biomarkers that make it into the clinic, allowing personalised medicine to become a reality.
References
- Baggerly KA and Coombes KR, What information should be required to support clinical ‘omics’ publications? Clinical Chemistry 57: pp688-690, 2011
- Baggerly KA and Coombes KR, Deriving chemosensitivity from cell lines: Forensic bioinformatics and reproducible research in high-throughput biology, Ann Appl 3: pp1,309-1,334, 2009
