Do we measure the right end points? A systematic review of primary outcomes in recent neonatal randomized clinical trials

doi:10.1067/mpd.2001.110299

The Journal of Pediatrics

Volume 138, Issue 1, January 2001, Pages 76-80

Presented in part at the Annual Meeting of the Pediatric Academic Societies, San Francisco, May 1-5, 1999, and at the 40th Annual Meeting of the European Society for Pediatric Research, Copenhagen, June 24-29, 1999.

https://doi.org/10.1067/mpd.2001.110299 Get rights and content

Abstract

Objectives: We performed a systematic review of primary outcomes in recently published neonatal randomized clinical trials to determine what end points are chosen for the evaluation of therapies in newborn infants. Methods: MEDLINE was searched in May 1998 for neonatal RCTs that had been published since 1993 in the British Medical Journal , The Journal of Pediatrics , Lancet , The New England Journal of Medicine , and Pediatrics. The primary outcome was identified wherever possible. Continuous measures were distinguished from discrete outcome events. The latter were classified as “long-term” if they were ascertained at least 6 months after birth. The hypothesized relative risk reduction for primary outcome event rates was calculated, where possible. Results: Of 119 eligible reports, 91 had an identifiable primary outcome that was a continuous measure in 46 trials and a discrete event in 45 trials. The median relative risk reduction was 50% for 28 of 40 short-term trials and 40% for 3 of 5 long-term trials. Conclusions: Most authors of recently published neonatal RCTs in 5 high-profile journals either failed to specify a primary end point or chose a continuous outcome measure. Trials with a discrete primary outcome were often short-term and designed to detect large risk reductions. Therefore modest but important treatment effects were likely missed by many of these trials. (J Pediatr 2001;138:76-80)

Section snippets

Methods

MEDLINE was searched in May 1998 for neonatal RCTs published since 1993. Journals included were the British Medical Journal , The Journal of Pediatrics , Lancet , The New England Journal of Medicine , and Pediatrics. A neonatal RCT was defined as a study in which newborn infants were assigned prospectively and by random allocation to 1 of 2 or more interventions. Excluded were reports in which mothers were randomized, even if the outcome was ascertained in the newborn. Trials of diagnostic

Results

The MEDLINE search retrieved 320 citations. Of these, 119 reports fulfilled our eligibility criteria and are the subject of this review (Figure); 46 RCTs had a continuous measure as the primary outcome.

Figure. Profile of RCTs in this systematic review.

Examples included various measures of oxygenation and ventilation (n = 12), hematocrit, hemoglobin, or the number of blood transfusions (n = 7), various measures of pain (n = 5), mean arterial blood pressure (n = 1), plasma bilirubin (n = 1),

Discussion

Clinicians are urged to base their assessment of treatment benefits and risks primarily on the results of randomized clinical trials.² Although we restricted our search to a recent 5-year period and 5 selected journals, we found >100 original trial reports. Will the application of those RCT results improve clinically important outcomes of newborn patients? Our systematic review suggests that this is quite uncertain because of flaws in the choice of the primary study end point. Almost one

Acknowledgements

We thank Prof Robin Roberts for his helpful comments.

References (16)

SG Paryani et al.
Treatment of asymptomatic congenital syphilis: benzathine versus procaine penicillin G therapy
J Pediatr
(1994)
NL Sloan et al.
Maternidad Isidro Ayora Study Team. Kangaroo mother method: randomised controlled trial of an alternative method of care for stabilised low-birthweight infants
Lancet
(1994)
JH Humphrey et al.
Impact of neonatal vitamin A supplementation on infant morbidity and mortality
J Pediatr
(1996)
J Doyle et al.
Does vitamin C cause hemolysis in premature newborn infants? Results of a multicenter double-blind, randomized, controlled trial
J Pediatr
(1997)
AB. Hill
The clinical trial
N Engl J Med
(1952)
GH Guyatt et al.
Users’ guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid?
JAMA
(1993)
CL. Meinert
Toward more definitive clinical trials
Control Clin Trials
(1980)
GH Guyatt et al.
Users’ guides to the medical literature. II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients?
JAMA
(1994)

There are more references available in the full text version of this article.

Cited by (46)

Better medicines for neonates: Improving medicine development, testing, and prescribing
2017, Early Human Development
Citation Excerpt :
Using a systematic review on 119 published papers on neonatal RCT, Zhang et al. documented that the postulated median relative risk reduction in these studies was 40–50%. Consequently, modest but still clinical relevant treatment effects can been missed [16]. Poor outcome selection remains an important cause of study failure.
Pharmacotherapy is a powerful tool to improve the outcome of neonates. Unfortunately, the potential health impact of pharmacotherapy in neonates remains underexplored. This necessitates a structured approach to go beyond the current practice of trial and error, reflected in off-label prescription. The existing regulatory framework hereby provides a structure to reflect about aspects like pharmacokinetic models for dose selection and outcome assessment, including long-term safety. Future medicine development should also be driven by neonatal needs, diseases and pathophysiology, since surfactant is the latest product developed for preterm neonates. The potential impact is illustrated by ongoing repurposing (propranolol, allopurinol, erythropoietin, Insulin-like Growth Factor-1) projects.
Clinical researchers will be crucial to close the knowledge gap by developing dose selection tools and outcome assessment tools and by exploring pathophysiological mechanisms. The final step of such a structured approach cycle is the subsequent translation of accumulated knowledge into improved prescribing.
Primary Outcomes Reporting in Trials (PORTal): a systematic review of inadequate reporting in pediatric randomized controlled trials
2017, Journal of Clinical Epidemiology
Citation Excerpt :
RCTs are preferentially included in knowledge synthesis efforts such as systematic reviews and meta-analyses, which inform decision makers at every level. Many RCTs are published annually in high-impact journals; however, there is growing concern with regards to the reporting of outcomes and consequently the reporting of the measurement properties of the outcome measures, namely their validity and reliability [1–4]. As clinical trials are “only as credible as their outcomes” [5], a lack of reporting and validation implies that tremendous expense, effort, and resources may not be used optimally.
Conduct a systematic review of pediatric randomized controlled trials (RCTs) published in high-impact journals to assess the reporting of primary outcomes and the psychometric properties of their measures.
Systematic review with screening and simultaneous data extraction conducted by two independent reviewers. Electronic searches of six general medicine and four pediatric journals were conducted in MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials (CENTRAL) databases. RCTs of a single phase/step in a single publication, published in English between 2000 and 2010 with participants less than 21 years of age were included.
A random sample of 20% (n = 445) of 2,229 initial references was screened and 206 (46%) met inclusion criteria. Half (48.5%) of included studies reported a singular primary outcome, 27% did not identify any primary outcome, and 24% identified multiple primary outcomes (range 2–20). Twenty-one trials used an instrument to measure their primary outcome, but only 7 (33%) reported its psychometric properties.
Pediatric trials published in top medical journals have inadequate reporting of their primary outcomes and the psychometric properties of their outcome measures. Whether the issue is one of poor reporting and/or poor validation will be further investigated.
Stopping the swinging pendulum of postnatal corticosteroid use
2014, Journal of Pediatrics
One-Year Respiratory Outcomes of Preterm Infants Enrolled in the Nitric Oxide (to Prevent) Chronic Lung Disease Trial
2008, Journal of Pediatrics
To identify whether inhaled nitric oxide treatment decreased indicators of long-term pulmonary morbidities after discharge from the neonatal intensive care unit.
The Nitric Oxide (to Prevent) Chronic Lung Disease trial enrolled preterm infants (<1250 g) between 7 to 21 days of age who were ventilated and at high risk for bronchopulmonary dysplasia. Follow-up occurred at 12 ± 3 months of age adjusted for prematurity; long-term pulmonary morbidity and other outcomes were reported by parents during structured blinded interviews.
A total of 456 infants (85%) were seen at 1 year. Compared with control infants, infants randomized to inhaled nitric oxide received significantly less bronchodilators (odds ratio [OR] 0.53 [95% confidence interval 0.36-0.78]), inhaled steroids (OR 0.50 [0.32-0.77]), systemic steroids (OR 0.56 [0.32-0.97]), diuretics (OR 0.54 [0.34-0.85]), and supplemental oxygen (OR 0.65 [0.44-0.95]) after discharge from the neonatal intensive care unit. There were no significant differences between parental report of rehospitalizations (OR 0.83 [0.57-1.21]) or wheezing or whistling in the chest (OR 0.70 [0.48-1.03]).
Infants treated with inhaled nitric oxide received fewer outpatient respiratory medications than the control group. However, any decision to institute routine use of this dosing regimen should also take into account the results of the 24-month neurodevelopmental assessment.
Reply
2007, Journal of Pediatrics
Composite outcomes in randomized clinical trials: arguments for and against
2007, American Journal of Obstetrics and Gynecology
Citation Excerpt :
The use of a binary score overcomes concerns about correlation between individual outcomes. Composite outcomes must be associated with the primary objective of the trial,2 be biologically plausible, and meaningful to clinicians and patients.1,3 For obstetric trials that use composite neonatal outcomes, it is also important that the outcome selected should translate into clinically important long-term outcome.
Composite outcomes that combine a number of individual outcomes (such as types of morbidity) are frequently used as primary outcomes in obstetrical trials. The main argument for their use is to ensure that trials can answer important clinical questions in a timely fashion, without needing huge sample sizes. Arguments against their use are that composite outcomes may be difficult to use and interpret, leading to errors in sample size estimation, possible contradictory trial results, and difficulty in interpreting findings. Such problems may reduce the credibility of the research, and may impact on the implementation of findings. Composite outcomes are an attractive solution to help to overcome the problem of limited available resources for clinical trials. However, future studies should carefully consider both the advantages and disadvantages before using composite outcomes. Rigorous development and reporting of composite outcomes is essential if the research is to be useful.

View all citing articles on Scopus

^☆: Reprint requests: Barbara Schmidt, MD, McMaster University, 1200 Main St West, Room HSC 3N11E, Hamilton, Ontario, Canada, L8N 3Z5.

View full text

Original ArticlesDo we measure the right end points? A systematic review of primary outcomes in recent neonatal randomized clinical trials☆

Abstract

Section snippets

Methods

Results

Discussion

Acknowledgements

J Pediatr

Lancet

J Pediatr

J Pediatr

The clinical trial

N Engl J Med

Users’ guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid?

JAMA

Toward more definitive clinical trials

Control Clin Trials

Users’ guides to the medical literature. II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients?

JAMA

Original Articles
Do we measure the right end points? A systematic review of primary outcomes in recent neonatal randomized clinical trials☆