Can Researchers Accurately Predict Trial Outcomes?

Lead Investigator: Jonathan Kimmelman
Co-investigators: Jamie Brehaut, Dean Fergusson, Alex John London, David Mandel, Ian Shrier, Russell Steele

Funding Agency: Canadian Institutes of Health Research
Term: August 2013 – July 2020

Introduction

Clinical trials are saturated with risk and uncertainty. Patients confront exposure to unproven substances. Principal investigators face uncertainty about recruiting subjects. Research ethics boards encounter uncertainty about the trade-off of risk and social utility. And drug developers wager large financial sums on potentially fruitless endeavors. How these different parties manage risk and uncertainty has profound relevance for the ethical review and disclosure during informed consent, as well as, how research systems allocate limited resources for addressing questions pertaining to health-care. By definition, risk and benefit assessment involve estimating probabilities, and these projections should strive to be as accurate as possible. However, the quality of risk, benefit, and feasibility estimation in clinical trials has yet to be rigorously evaluated. This grant proposes to use techniques derived from decision sciences to determine whether investigators and independent experts can effectively forecast the risks, benefits, and feasibility of trials.

Design

We propose a study of investigator and expert reviewer forecasts in a naturalistic setting of active trials. We will use a semi-automated system to query principal investigators for these four predictions shortly after their trials appear on clinicaltrials.gov. We will also use Bayesian elicitation to compare the forecasting performance of principal investigators against a sample of independent experts, and explore how these actors arrive at forecasts. Last, we will use ethical analysis to unravel the implications of our findings for trial design, ethical review, and data monitoring.

Protections for Participants

Our protocol has been reviewed and approved by McGill’s Institutional Review Board (IRB). Questionnaires will be sent initially by email. Positive responses will be interpreted as consent. When responses are received, the predictions will be automatically recorded in our database. Participant names will be retained for follow-up purposes only and will be destroyed at the time of final analysis. No identifying information will be published.

Who We Are

This study is being led by Jonathan Kimmelman, head of the STREAM research group and associate professor in the Biomedical Ethics Unit / Social Studies of Medicine department at McGill University. The project manager is Spencer Phillips Hey, a postdoctoral fellow in the Biomedical Ethics Unit at McGill University.

Questions

Should you have questions or concerns about our study, please contact jonathan.kimmelman@mcgill.ca or spencer.hey@mcgill.ca.

2013 September 17 | No comments please

@Manual{stream2013-327,
    title = {Can Researchers Accurately Predict Trial Outcomes?},
    journal = {STREAM research},
    author = {STREAM admin},
    address = {Montreal, Canada},
    date = 2013,
    month = sep,
    day = 17,
    url = {https://www.translationalethics.com/projects/forecast-study/}
}