Proceedings of the Fourth Seattle Symposium in Biostatistics: Clinical Trials

Randomized clinical trials are the standard method for evaluating new interventions or comparing existing ones. Trials which use clinical outcomes as the primary outcome can be large, require lengthy follow-up, and can be expensive. For these reasons, researchers have sought to use intermediate outcomes such as biomarkers as a substitute or surrogate for the clinical outcome. Over a decade ago, this practice had become common. Fleming and DeMets (Ann Intern Med 125:605–613, 1996) reported many cases where the use of a biomarker as a surrogate outcome failed to reliably assess the effect of the intervention, in some cases missing harmful effects including mortality. Recently, the Institute of Medicine (IOM) reviewed the state of the art and came to similar conclusions that biomarkers have often proved to be unreliable as a surrogate [Committee on Qualifications of Biomarkers and Surrogate Endpoints in Chronic Disease, Michael C, Ball J (eds) (2010) Evaluation of biomarkers and surrogate endpoints in chronic disease. National Academies Press, Washington]. They proposed that biomarkers must meet certain criteria including analytic validity, strong correlation with the clinical outcome and the ability to capture the full effects of the intervention. The use of a biomarker as a surrogate must be done so in the context of its intended use, and done so with great caution. While the IOM report further clarifies the necessary requirements of a potential biomarker as a surrogate, the report still recommends caution in using surrogate outcomes in final phases of intervention evaluation as did Fleming and DeMets (Ann Intern Med 125:605–613, 2004).

Biomarkers provide opportunities to maximize the knowledge gained from randomized controlled trials. Applications may include the identification of subpopulations that experience differential treatment effects; the assessment of adherence to treatment or intervention goals; and the elucidation of key biological pathways through which the treatments affect clinical outcomes. This last biomarker role also has implications for the development and initial testing of potential treatments. These types of applications are illustrated using biomarker studies in the Women’s Health Initiative postmenopausal hormone therapy and low-fat dietary pattern trials. Related topics are also described where further methodology developments would be helpful.

Biomarkers are characteristics that are objectively measured and evaluated as indicators of normal or pathogenic biologic processes. There are a multitude of ways that biomarkers are used including diagnosis of disease, prognosis of clinical outcomes, patient staging and treatment selection, indication of response to a prior natural exposure, and characterization of response to a biomedical intervention. In the general setting of vaccine trials for infectious diseases, the identification of biomarkers ultimately depends on the specifics of the vaccine and the pathogen and typically focuses on aspects of the immune response to vaccine and/or detection and typing of the pathogen. A central use of immune response biomarkers in vaccine research is as primary endpoints for early phase vaccine trials. At one end of the spectrum, these biomarkers define the extent to which the vaccine is biologically active while at the other end they characterize the nature of the response with sufficient detail to provide some sense of the plausibility that the responses would be clinically protective.

This general theme of identification and validation of immune response biomarkers as surrogate endpoints is a specific area of interest in vaccine trials. In contrast, pathogen-based biomarkers are often used as components of clinical endpoints in situations where clinical outcomes may be attributed to causes other than the pathogen targeted by vaccination. Such biomarkers can lend specificity to trial endpoints that increases sensitivity to detect meaningful vaccine effects. A final important use of both immune response and pathogen-based biomarkers in vaccine trials is the identification of clues that can help to guide the iterative development of vaccines.

This chapter covers a broad range of issues centered around the topic of optimizing therapies for individuals and the role of the clinical trial in reaching that goal. We describe how the clinical trial has been increasingly relied upon to provide the evidence for the patient level or patient marker level differential benefit or risk of therapies and how the study design choices can change depending upon the various study objectives. We consider the definition and role of prognostic and predictive classifiers or markers in the different clinical trial designs used for selecting and evaluating enriched study populations, and the difference between the retrospective and prospective approaches to evaluating differential treatment effects among marker subgroups. Some examples are given to illustrate the issues. Two of the most challenging aspects of identifying and validating a predictive marker are the simultaneous need to quantify the performance characteristics (sensitivity and specificity) of the classifier and the choice of whether the study design should include all comers or selection of the marker positive only subgroup. The motivation for these clinical trial approaches to individualizing therapy is to maximize the benefits and minimize the safety risks of therapies for patients.

Developments in genomics are providing a biological basis for the heterogeneity of clinical course and response to treatment that have long been apparent to clinicians. The ability to molecularly characterize human diseases presents new opportunities to develop more effective treatments and new challenges for the design and analysis of clinical trials.

In oncology, treatment of broad populations with regimens that benefit a minority of patients is less economically sustainable with expensive molecularly targeted therapeutics. The established molecular heterogeneity of human diseases requires the development of new paradigms for the design and analysis of randomized clinical trials as a reliable basis for predictive medicine.

We review prospective designs for the development of new therapeutics and predictive biomarkers to inform their use. We cover designs for a wide range of settings. At one extreme is the development of a new drug with a single candidate biomarker and strong biological evidence that marker negative patients are unlikely to benefit from the new drug. At the other extreme are phase III clinical trials involving both genome-wide discovery of a predictive classifier and internal validation of that classifier. We have outlined a prediction-based approach to the analysis of randomized clinical trials that both preserve the type I error and provide a reliable internally validated basis for predicting which patients are most likely or unlikely to benefit from a new regimen.

Characterizing the genetic basis of responses in clinical trials has been made substantially easier and more powerful through the use of single nucleotide polymorphism data. A million of more of these markers can now be scored cheaply with commercial SNP-chips and ten million or more additional SNPs can be inferred by imputation. These rich datasets offer a deep look at the human genome and they are likely to tag many of the response causal genes. It has already become common for SNP types to be included in drug box labels.The ease and low cost of obtaining SNP profiles in clinical trials comes with the price of noisy data: it is common to have to reject data at 10% of the assayed SNPs. However, those SNPs that pass rigorous data cleaning protocols not only offer the chance of identifying the genes that affect response variables but also may reveal information about the genetic architectures of the trial participants and the populations to which they belong, as well as the relationships among the participants. Among novel applications of SNP profiles is the ability to determine HLA type without expensive sequencing.

Many Phase 3 randomized clinical trials are currently being conducted multinationally with too few participants from any individual country to allow reliable inference about the beneficial or harmful effects of the tested product using data from that country alone. Instead, the conclusions for a given country will come from the totality of the data. Insofar as “country” is just another subgroup defined by baseline variables, this strategy is defensible. On the other hand, in cases where “country” stands as a surrogate for country-specific variables that importantly influence the benefits and harms of an intervention, inferring from the study population at large to specific countries may be less appropriate. Such variables may include the nature of the disease being studied, the country-specific standard of care, the patterns of safety reporting, and the extent of adherence to study protocol. This paper presents four examples of studies where the observed treatment effect in the USA differed considerably from the effect observed elsewhere. It argues that the problem is in some sense intractable because a study large enough to provide precise estimates of effect sizes within specific countries would likely be infeasible. Instead, although the paper recommends generally applying the overall result to the participating countries, it provides suggestions for strategies in the design and analysis phase to mitigate potential inferential ambiguities.

Pharmaceutical sponsors use a variety of approaches to make important benefit/risk decisions about their products internally. Benefit/risk assessment is equally important when regulators evaluate a product for marketing approval and payers evaluate it for reimbursement decision. Once a product receives marketing authorization, it is critical to communicate pertinent benefit and risk information to patients and health-care providers. All of the above can be made easier by the use of a common framework. In this paper, we review where we are in benefit/risk assessment. This includes endeavors by academic institutions, regulators, and the pharmaceutical industry. Despite concerns about quantitative benefit/risk assessment expressed by some, we argue that without a way to quantitatively incorporate the relative importance of factors impacting benefit/risk assessment, it will be hard to bring transparent decisions to questions such as “does the benefit of this product outweigh the risk.”

Reliable evidence is needed from clinical research about whether the interventions used in clinical practice are safe as well as effective. Regarding risk, safety is not established by failure to establish excess risk, such as obtaining confidence intervals for the relative risk of safety events that include unity. Absence of evidence is not evidence of absence. Rather, safety is established if available data about safety are sufficiently favorable and reliable to rule out the threshold for unacceptable risk, where this threshold should be determined by considering the strength of the evidence for efficacy.

Important insights about safety usually will be provided before marketing through Phase 1, 2, and 3 clinical trials. These insights, especially regarding risks associated with long-term use of the intervention and risks of rare but clinically compelling events, are enhanced by post-marketing active and passive surveillance, and especially by large, long-term randomized trials that provide the most reliable approach for identifying and addressing safety signals. The integrity of these randomized trials is enhanced by preventing irregularities in the quality of trial conduct that would reduce their sensitivity to detecting clinically meaningful safety risks caused by the experimental regimen.

After considering approaches to identifying and addressing safety risks and discussing performance standards to improve the quality of conduct of safety trials, we will consider further the vulnerability to undetected safety risks when evidence for efficacy has been limited to documentation of effects on surrogate endpoints such as biomarkers, and then discuss important considerations regarding cardiovascular safety trials conducted in the setting of type 2 diabetes mellitus.

Adverse effects are an expected consequence of drug use, but how to detect them when they are rare or represent only a small increase over background rates remains a formidable problem. It is useful to review the past history of how important adverse effects were discovered so that we can improve our ability to detect them more rapidly.

We discover serious adverse effects in different ways, depending on how rare they are and whether they occur spontaneously (i.e., in the absence of a drug), and on how great the increase over background rate is. Depending on such factors we rely on spontaneous reports, epidemiologic data (if the risk increase is reasonably large), and on large controlled trials or meta-analyses (when the increase in risk is smaller).

These methods, the history of their use, and their usefulness in the future are considered.

The use of group sequential methodology has become widespread in the conduct of clinic trials. As each clinical trial presents unique scientific, statistical, and logistical constraints, it is important to carefully evaluate candidate group sequential designs to ensure desirable operating characteristics. At the implementation stage of a clinical trial design it is also essential to account for deviations from original design specifications in order to control operating characteristics such as type I and II error rates. These changes might include the number and/or timing of analyses as well as deviations from the originally assumed variability of outcome measures. Due to the computational complexity involved in evaluating, monitoring, and analyzing a group sequential procedure, specialized software is required. In this manuscript we demonstrate how the RCTdesign package (http://​www.​rctdesign.​org) in R can be used to select, implement, and analyze a group sequential stopping rule. Throughout, we illustrate trial design and monitoring in the context of a group sequential survival trial of an experimental monoclonal antibody in patients with relapsed chronic lymphocytic leukemia (CLL).

The current availability of dense sets of marker SNPs for the human genome is having a large impact on genetic studies and offers new possibilities for clinical trials. This chapter offers a unified basis for the analysis of marker and response data, emphasizing the central importance of the correlation, or linkage disequilibrium, between SNP markers and the genes that affect response. It is convenient to phrase the development of association mapping in the language of quantitative genetics, using additive and non-additive components of variance. A novel feature of dense SNP data is that good estimates can be made of actual inbreeding and relatedness. These estimates are more relevant than values predicted from family pedigree, and are all that are available in the absence of family data.The dimensionality of SNP marker datasets has required the development of new methods that are appropriate for a large number of statistical comparisons, and the development of computational methods that allow high-dimensional regression. These methods are reviewed here, as is the use of biological annotation for both viewing the relevance of empirical associations, and to structure analysis in order to focus on those markers with the highest expectation for association with the outcomes under study.