Accessibility of trial reports for drugs stalling in development: a systematic assessment of registered trials

Non-publication of clinical trial results has been recognized as a serious scientific and ethical problem. Underreporting frustrates evaluation of a drug’s utility and safety, and fails to redeem the sacrifice of trial participants.

Thus far, policy measures to counteract non-publication have focused on trials of interventions used in practice. However, 9/10 interventions entering clinical testing never achieve marketing licensure. What happens to the results of those trials?

Figure depicting the rates of publication of trials of licensed drugs compared with trials of stalled drugs–overall and by major subgroup.

In our most recent publication, my colleagues and I systematically quantified the proportion of trials of unlicensed interventions that are not published.

We used clinicaltrials.gov trial registration records to create a sample of clinical trials of drugs that achieved licensure between 2005 and 2009 (“licensed drugs” or “translated drugs”) and drugs that stalled in clinical development (“stalled drugs”) in the same time period. Our sample included registered phase II, III or IV trials that closed between January 1st, 2006 and December 31st, 2008 and tested a drug in the treatment of cancer, cardiovascular disease or neurological disorders. We felt this sample provided a relevant and contemporary look into a wide swathe of drug development activity. We then searched Google Scholar, PubMed and Embase, and contacted investigators to determine the publication status of each trial in our sample at least 5 years after reported primary endpoint collection.

Whereas 75% (72/96) of registered trials of licensed drugs were published, only 37% (30/81) trials of stalled drugs were published. The adjusted hazard ratio for publication was 2.7 (95% confidence interval 1.7 to 4.3) in favour of licensed drug trials–that is, clinical trials of licensed drugs were almost three times as likely to publish findings as trials of stalled drugs. Higher publication rates for licensed drug trials were observed regardless of disease type, sponsorship (industry involvement versus not), trial phase, and location across the globe.

Figure depicting the proportion of trials of licensed and unlicensed interventions that are published as a function of time from reported primary endpoint collection. The publication of stalled drug trials plateaus over time around 37%, whereas the publication of translated drug trials attains 75% in the same time period.

Moreover, a total of 20,135 patients participated in trials of stalled drugs that were never published. In addition to the alarming implications for these patients, trials in unsuccessful translation trajectories contain a wealth of scientific information for research planning, such as validation of pathophysiological theories driving drug development, as well as data about drug safety and pharmacology. All of this information vanishes when trials of unsuccessful interventions are not published.

Our key finding is that much of the information collected in unsuccessful drug trials is inaccessible to the broader research and practice communities. Our results provide an evidence base and rationale for policy reforms aimed at promoting transparency, ethics, and accountability in clinical research. One such potential reform is the Notice of Proposed Rulemaking entitled “Clinical Trials Registration and Results Submission” issued by the US Department of Health and Human Services in November 2014. The proposal, which moves to implement the FDAAA summary results reporting requirements for trials of licensed drugs and to extend them to trials of unlicensed drugs, was closed to public comments on March 23rd, 2015. The rule is now undergoing revision.

2015 May 26 | Leave a comment | Tags: bioethics clinical trials drug development information loss nonpublication result reporting unlicensed drugs

@Manual{stream2015-781,
    title = {Accessibility of trial reports for drugs stalling in development: a systematic assessment of registered trials},
    journal = {STREAM research},
    author = {Amanda Hakala},
    address = {Montreal, Canada},
    date = 2015,
    month = may,
    day = 26,
    url = {https://www.translationalethics.com/2015/05/26/accessibility-of-trial-reports-for-drugs-stalling-in-development-a-systematic-assessment-of-registered-trials/}
}

The Landscape of Early Phase Research

As Jonathan is fond of saying: Drugs are poisons. It is only through an arduous process of testing and refinement that a drug is eventually transformed into a therapy. Much of this transformative work falls to the early phases of clinical testing. In early phase studies, researchers are looking to identify the optimal values for the various parameters that make up a medical intervention. These parameters are things like dose, schedule, mode of administration, co-interventions, and so on. Once these have been locked down, the “intervention ensemble” (as we call it) is ready for the second phase of testing, where its clinical utility is either confirmed or disconfirmed in randomized controlled trials.

In our piece from this latest issue of the Kennedy Institute of Ethics Journal, Jonathan and I present a novel conceptual tool for thinking about the early phases of drug testing. As suggested in the image above, we represent this process as an exploration of a 3-dimensional “ensemble space.” Each x-y point on the landscape corresponds to some combination of parameters–a particular dose and delivery site, say. The z-axis is then the risk/benefit profile of that combination. This model allows us to re-frame the goal of early phase testing as an exploration of the intervention landscape–a systematic search through the space of possible parameters, looking for peaks that have promise of clinical utility.

We then go on to show how the concept of ensemble space can also be used to analyze the comparative advantages of alternative research strategies. For example, given that the landscape is initially unknown, where should researchers begin their search? Should they jump out into the deep end, to so speak, in the hopes of hitting the peak on the first try? Or should they proceed more cautiously–methodologically working their way out from the least-risky regions, mapping the overall landscape as they go?

I won’t give away the ending here, because you should go read the article! Although readers familiar with Jonathan’s and my work can probably infer which of those options we would support. (Hint: Early phase research must be justified on the basis of knowledge-value, not direct patient-subject benefit.)

UPDATE: I’m very happy to report that this paper has been selected as the editor’s pick for the KIEJ this quarter!

2014 July 4 | Leave a comment | Tags: bioethics cell therapy first in human trials methodology translational research

@Manual{stream2014-567,
    title = {The Landscape of Early Phase Research},
    journal = {STREAM research},
    author = {Spencer Phillips Hey},
    address = {Montreal, Canada},
    date = 2014,
    month = jul,
    day = 4,
    url = {https://www.translationalethics.com/2014/07/04/the-landscape-of-early-phase-research/}
}

The Literature Isn’t Just Biased, It’s Also Late to the Party

Animal studies of drug efficacy are an important resource for designing and performing clinical trials. They provide evidence of a drug’s potential clinical utility, inform the design of trials, and establish the ethical basis for testing drugs in human. Several recent studies suggest that many preclinical investigations are withheld from publication. Such nonreporting likely reflects that private drug developers have little incentive to publish preclinical studies. However, it potentially deprives stakeholders of complete evidence for making risk/benefit judgments and frustrates the search for explanations when drugs fail to recapitulate the promise shown in animals.

In a future issue of The British Journal of Pharmacology, my co-authors and I investigate how much preclinical evidence is actually available in the published literature, and when it makes an appearance, if at all.

Although we identified a large number of preclinical studies, the vast majority was reported only after publication of the first trial. In fact, for 17% of the drugs in our sample, no efficacy studies were published before the first trial report. And when a similar analysis was performed looking at preclinical studies and clinical trials matched by disease area, the numbers were more dismal. For more than a third of indications tested in trials, we were unable to identify any published efficacy studies in models of the same indication.

There are two possible explanations for this observation, both of which have troubling implications. Research teams might not be performing efficacy studies until after trials are initiated and/or published. Though this would seem surprising and inconsistent with ethics policies, FDA regulations do not emphasize the review of animal efficacy data when approving the conduct of phase 1 trials. Another explanation is that drug developers precede trials with animal studies, but withhold them or publish them only after trials are complete. This interpretation also raises concerns, as delay of publication circumvents mechanisms—like peer review and replication—that promote systematic and valid risk/benefit assessment for trials.

The take home message is this: animal efficacy studies supporting specific trials are often published long after the trial itself is published, if at all. This represents a threat to human protections, animal ethics, and scientific integrity. We suggest that animal care committees, ethics review boards, and biomedical journals should take measures to correct these practices, such as requiring the prospective registration of preclinical studies or by creating publication incentives that are meaningful for private drug developers.

2014 June 30 | Leave a comment | Tags: bioethics efficacy preclinical studies publication bias translational research

@Manual{stream2014-542,
    title = {The Literature Isn’t Just Biased, It’s Also Late to the Party},
    journal = {STREAM research},
    author = {Carole Federico},
    address = {Montreal, Canada},
    date = 2014,
    month = jun,
    day = 30,
    url = {https://www.translationalethics.com/2014/06/30/the-literature-isnt-just-biased-its-also-late-to-the-party/}
}

Teaching Kills Blogging: Somewhat Recent Developments…

Dear Faithful Readers: Teaching has cut my blogging to a trickle, though the teaching has now begun to taper off. My silence is not for want of major developments in the last two months. Among a few highlights:

2010 April 16 | Leave a comment | Tags: bioethics blogging health care reform patents recent developments translational research

@Manual{stream2010-69,
    title = {Teaching Kills Blogging: Somewhat Recent Developments…},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2010,
    month = apr,
    day = 16,
    url = {https://www.translationalethics.com/2010/04/16/teaching-kills-blogging-somewhat-recent-developments/}
}