Accessibility of trial reports for drugs stalling in development: a systematic assessment of registered trials

by

Non-publication of clinical trial results has been recognized as a serious scientific and ethical problem. Underreporting frustrates evaluation of a drug’s utility and safety, and fails to redeem the sacrifice of trial participants.

Thus far, policy measures to counteract non-publication have focused on trials of interventions used in practice. However, 9/10 interventions entering clinical testing never achieve marketing licensure. What happens to the results of those trials?

Figure depicting the rates of publication of trials of licensed drugs compared with trials of stalled drugs--overall and by major subgroup.

Figure depicting the rates of publication of trials of licensed drugs compared with trials of stalled drugs–overall and by major subgroup.

In our most recent publication, my colleagues and I systematically quantified the proportion of trials of unlicensed interventions that are not published.

We used clinicaltrials.gov trial registration records to create a sample of clinical trials of drugs that achieved licensure between 2005 and 2009 (“licensed drugs” or “translated drugs”) and drugs that stalled in clinical development (“stalled drugs”) in the same time period. Our sample included registered phase II, III or IV trials that closed between January 1st, 2006 and December 31st, 2008 and tested a drug in the treatment of cancer, cardiovascular disease or neurological disorders. We felt this sample provided a relevant and contemporary look into a wide swathe of drug development activity. We then searched Google Scholar, PubMed and Embase, and contacted investigators to determine the publication status of each trial in our sample at least 5 years after reported primary endpoint collection.

Whereas 75% (72/96) of registered trials of licensed drugs were published, only 37% (30/81) trials of stalled drugs were published. The adjusted hazard ratio for publication was 2.7 (95% confidence interval 1.7 to 4.3) in favour of licensed drug trials–that is, clinical trials of licensed drugs were almost three times as likely to publish findings as trials of stalled drugs. Higher publication rates for licensed drug trials were observed regardless of disease type, sponsorship (industry involvement versus not), trial phase, and location across the globe.

Figure depicting the proportion of trials of licensed and unlicensed interventions that are published as a function of time from reported primary endpoint collection. The publication of stalled drug trials plateaus over time around 37%, whereas the publication of translated drug trials attains 75% in the same time period.

Figure depicting the proportion of trials of licensed and unlicensed interventions that are published as a function of time from reported primary endpoint collection. The publication of stalled drug trials plateaus over time around 37%, whereas the publication of translated drug trials attains 75% in the same time period.

Moreover, a total of 20,135 patients participated in trials of stalled drugs that were never published. In addition to the alarming implications for these patients, trials in unsuccessful translation trajectories contain a wealth of scientific information for research planning, such as validation of pathophysiological theories driving drug development, as well as data about drug safety and pharmacology. All of this information vanishes when trials of unsuccessful interventions are not published.

Our key finding is that much of the information collected in unsuccessful drug trials is inaccessible to the broader research and practice communities. Our results provide an evidence base and rationale for policy reforms aimed at promoting transparency, ethics, and accountability in clinical research. One such potential reform is the Notice of Proposed Rulemaking entitled “Clinical Trials Registration and Results Submission” issued by the US Department of Health and Human Services in November 2014. The proposal, which moves to implement the FDAAA summary results reporting requirements for trials of licensed drugs and to extend them to trials of unlicensed drugs, was closed to public comments on March 23rd, 2015. The rule is now undergoing revision.

2015 May 26 | Leave a comment | Tags:

BibTeX

@Manual{stream2015-781,
    title = {Accessibility of trial reports for drugs stalling in development: a systematic assessment of registered trials},
    journal = {STREAM research},
    author = {Amanda Hakala},
    address = {Montreal, Canada},
    date = 2015,
    month = may,
    day = 26,
    url = {http://www.translationalethics.com/2015/05/26/accessibility-of-trial-reports-for-drugs-stalling-in-development-a-systematic-assessment-of-registered-trials/}
}

MLA

Amanda Hakala. "Accessibility of trial reports for drugs stalling in development: a systematic assessment of registered trials" Web blog post. STREAM research. 26 May 2015. Web. 22 Oct 2024. <http://www.translationalethics.com/2015/05/26/accessibility-of-trial-reports-for-drugs-stalling-in-development-a-systematic-assessment-of-registered-trials/>

APA

Amanda Hakala. (2015, May 26). Accessibility of trial reports for drugs stalling in development: a systematic assessment of registered trials [Web log post]. Retrieved from http://www.translationalethics.com/2015/05/26/accessibility-of-trial-reports-for-drugs-stalling-in-development-a-systematic-assessment-of-registered-trials/

Tweet this post

Are Trials Necessary?

by

Today’s New York Times ran a heartbreaking story by Amy Harmon about two cousins who developed melanoma. One was entered into a cancer clinical trial and received the investigational drug PLX4032. The other was ineligible for the trial, and therefore unable to access the experimental drug. Guess which cousin died?


The article is one in a series of Harmon articles that seems to raise questions about whether rules governing drug testing and research are depriving desperately ill patients timely access to curative therapy. In this article, the narrative takes aim at two practices: 1- the practice of including control groups within trials, and randomly determining that some patients will receive standard care that is widely regarded as inadequate; 2- excluding patient access to drugs that have not yet demonstrated unequivocal therapeutic advantage.

As with many of my blog entries, I preface this one by saying that I am not a cancer doc, and therefore not in a position to evaluate whether PLX4032 is the wonder drug this story makes it out to be. I also preface my comment on this article by acknowledging the incredible pain and anxiety that patients suffer when denied access to a trial, or when denied access to a preferred drug within a trial. These disclaimers aside, I found the tenor of this article very problematic.

First, the reason investigators randomly determine treatment choice in trials is because, at the outside of a well designed trial, there is genuine uncertainty about whether the new drug is better, the same, or worse than the (inadequate) standard treatment. Many doctors participate in trials because they fervently believe the new regimen is better than the standard one. But the evidence shows, again and again, that on average, new drugs outperform old ones in a small portion of instances (maybe around 15-20%). It is just as likely that new drugs will underperform standard treatments- making patients sicker perhaps, or failing to deliver as much punch. So one concern about the article is the premise that doctor’s personal beliefs about which cancer drug will perform better in a randomized controlled trial carries some moral weight. The evidence shows doctors in the aggregate haven’t a clue- which is why functional healthcare systems run trials.

A second troubling premise here is that there is no harm to allowing public consumption of drugs that are not yet validated in rigorous clinical trials. CEOs of many pharmaceutical companies perhaps may share this view. But the historical record shows otherwise: in fact, many patients are severely harmed when drugs are introduced into clinical use before they have been established as safe and effective. Perhaps a few readers out there may be familiar with thalidomide? Or autologous bone marrow transplantation for breast cancer? Ever considered the price tag on these new cancer drugs, and do you want your government or insurance company purchasing a potentially useless drug?

Still, article zeros in on an ethical tension that is very difficult to eradicate from clinical research. Patients want- and are entitled- to be treated as individuals. Physicians also prefer to treat patients as individuals. Clinical trials, however require that patients be treated as tokens of larger populations- that they be treated, in a sense, as “stand ins” for future patients. Randomization has not been shown to deprive patients of access to life preserving drugs. However, it does rob patients of fulfilling their desire to be treated as individuals and to exercise personal choice. And this is one of the reasons why the field of research ethics is endlessly fascinating, important, and nettlesome. (photo credit: travelingMango 2008).

2010 September 20 | 5 comments | Tags:

BibTeX

@Manual{stream2010-59,
    title = {Are Trials Necessary?},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2010,
    month = sep,
    day = 20,
    url = {http://www.translationalethics.com/2010/09/20/are-trials-necessary/}
}

MLA

Jonathan Kimmelman. "Are Trials Necessary?" Web blog post. STREAM research. 20 Sep 2010. Web. 22 Oct 2024. <http://www.translationalethics.com/2010/09/20/are-trials-necessary/>

APA

Jonathan Kimmelman. (2010, Sep 20). Are Trials Necessary? [Web log post]. Retrieved from http://www.translationalethics.com/2010/09/20/are-trials-necessary/

Tweet this post

Everything You Always Wanted to Know About Clinical Research in China, But Were Afraid to Ask

by

After scandals involving tainted toothpaste, poisonous pet food, adulterated milk, contaminated heparin, and counterfeit medicines, and a thriving trade in organs, one shudders to imagine how well human subjects are protected in drug studies performed in China. Apart from an occasional report in the medical literature, there is little easily accessible information about Chinese human protections: the regulations and laws, compliance and enforcement, and professional standards. This information would be interesting in its own right; however, it is all the more essential given trends towards trans-national trials.


A recent report issued the Medical Research Council (UK) provides some indication of China’s system of human protections, and how researchers in countries like UK might proceed when locating trials in China. The executive summary finds that Chinese regulations substantially parallel those of the International Committee on Harmonization (ICH). Informed consent and independent ethics review is required for any study. However, the UK and China “differ greatly in their approaches to enforcing guidelines for the conduct of research at the national level. In China, although there is some scrutiny of clinical trials, there is comparatively little inspection or review of compliance.” Other intriguing mentions are concerns about undue inducement in China: “the high costs of healthcare and medicines, and the dependence on local providers means that particular attention [for UK researchers pursuing studies in China] must be given to potential inducements to participate in research. Collaboration with China may offer attractive opportunities for large-scale recruitment, but potential UK collaborators must be alert to the risk that unethical inducements may be offered to potential participants. … given the high cost of accessing health care in China, a ‘free health check’ may be a relatively greater inducement than it would be deemed to be in the UK.”


A perusal of the Chinese regulations- at least the ones provided in this report- indicate the following:


• China places heavy emphasis on procedure (e.g. IRB review) and informed consent, rather than substance (e.g. prohibitions on certain practices; definitions of minimal risk; categories of patients)


• China seems to take a very permissive stand (like ICH) on the use of placebos. Indeed, there is no mention of studies involving placebo.


• There is no mention of justice considerations- for example, post-trial access or responsiveness.


(photo credit: 2 dogs, 07/03/25 12:32:09 Shanghai, 2007)

2009 August 20 | Leave a comment | Tags:

BibTeX

@Manual{stream2009-87,
    title = {Everything You Always Wanted to Know About Clinical Research in China, But Were Afraid to Ask},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2009,
    month = aug,
    day = 20,
    url = {http://www.translationalethics.com/2009/08/20/everything-you-always-wanted-to-know-about-clinical-research-in-china-but-were-afraid-to-ask/}
}

MLA

Jonathan Kimmelman. "Everything You Always Wanted to Know About Clinical Research in China, But Were Afraid to Ask" Web blog post. STREAM research. 20 Aug 2009. Web. 22 Oct 2024. <http://www.translationalethics.com/2009/08/20/everything-you-always-wanted-to-know-about-clinical-research-in-china-but-were-afraid-to-ask/>

APA

Jonathan Kimmelman. (2009, Aug 20). Everything You Always Wanted to Know About Clinical Research in China, But Were Afraid to Ask [Web log post]. Retrieved from http://www.translationalethics.com/2009/08/20/everything-you-always-wanted-to-know-about-clinical-research-in-china-but-were-afraid-to-ask/

Tweet this post

Help Wanted, Part 2

by

So, what are some of the intriguing ethical questions of Kolata’s August 2d article? Here is one: when researchers conduct studies and ethics committees review protocols, resource allocation is an important consideration. If, as Kolata alleges, mediocre trials siphon eligible patients away from good trials, then there is a case to be made that IRBs and investigators need to ponder carefully the effects proposed trials will have on other studies- even when proposed trials have a favorable direct benefit-risk balance for volunteers who enter them.


Second, if resource allocation is a key consideration in realms where patients are scarce, investigators (and IRBs) need reliable criteria for assessing the broader social value of study protocols. They further need some way of being able to compare one protocol against a body of others that are either underway or in the pipeline. The current system provides no straightforward way of doing this.

Third, if 50% of trials fail to recruit sufficient numbers to produce meaningful results, investigators, IRBs, DSMBs, and granting agencies are doing a lousy job ensuring high ethical standards in human research. It is well established that, for any study to redeem the burdens that volunteers endure on enrollment, it must produce valuable findings. It is disturbing, to say the least, that many volunteers enter studies that go nowhere, and that investigators, IRBs, and funding agencies are not realistically projecting recruitment.

Last, Kolata suggests that many cancer trials are merely aimed at “polishing a doctor’s résumé.” It would make a useful contribution to the field of cancer research- and bioethics- to measure the frequency of this practice. Meantime, this inability of IRBs to detect this kind of conduct, and stop it in its tracks, signals an important deficiency in human protections. Which leads me to my next post… (photo credit: ziggy fresh 2006)

2009 August 9 | Leave a comment | Tags:

BibTeX

@Manual{stream2009-88,
    title = {Help Wanted, Part 2},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2009,
    month = aug,
    day = 9,
    url = {http://www.translationalethics.com/2009/08/09/help-wanted-part-2/}
}

MLA

Jonathan Kimmelman. "Help Wanted, Part 2" Web blog post. STREAM research. 09 Aug 2009. Web. 22 Oct 2024. <http://www.translationalethics.com/2009/08/09/help-wanted-part-2/>

APA

Jonathan Kimmelman. (2009, Aug 09). Help Wanted, Part 2 [Web log post]. Retrieved from http://www.translationalethics.com/2009/08/09/help-wanted-part-2/

Tweet this post

Red Tape: More IRB-bashing

by

Slow news day, I guess, at the New York Times. In today’s paper, American Enterprise Institute scholar Sally Satel laments that “federal ethics regulations” have become “so stringent and unwieldy that the ethics oversight system often impedes the kind of careful research we should be promoting.” And the paperwork, according to Satel, is driving up costs and “‘…pricing large clinical trials out of reach.'”

I enjoy a good IRB bashing now and then. But notice how, in Satel’s analysis, dysfunctionality = driving up costs and “cutting into productivity.” Well yes- that’s what regulations do. Perhaps the IRB system is indeed dysfunctional. On the other hand, the dearth of major scandals in human protection over the last decade, and the trust that many volunteers seem to place in the system of clinical research, provide grounds for at least a little skepticism about the “dysfunctionality” claim.
As for the nature of that dysfunctionality: In my previous posting, I described an article by Gina Kolata that seems to offer an opposing analysis: that by being too permissive, IRBs are approving mediocre trials that siphon patients away from the good ones. (photo credit: McBeth 2005)

| Leave a comment | Tags:

BibTeX

@Manual{stream2009-89,
    title = {Red Tape: More IRB-bashing},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2009,
    month = aug,
    day = 9,
    url = {http://www.translationalethics.com/2009/08/09/red-tape-more-irb-bashing/}
}

MLA

Jonathan Kimmelman. "Red Tape: More IRB-bashing" Web blog post. STREAM research. 09 Aug 2009. Web. 22 Oct 2024. <http://www.translationalethics.com/2009/08/09/red-tape-more-irb-bashing/>

APA

Jonathan Kimmelman. (2009, Aug 09). Red Tape: More IRB-bashing [Web log post]. Retrieved from http://www.translationalethics.com/2009/08/09/red-tape-more-irb-bashing/

Tweet this post

Help Wanted- For the War on Cancer

by

Earlier this week (Aug 2), Gina Kolata of the NYTimes ran a fascinating story about challenges recruiting patients to cancer clinical trials. The story contains interesting facts, credible claims, analysis, and unfortunately, some misleading conjectures. The problem of patient recruitment also invites some hard headed ethical analysis.


First the facts. According to the article, one in five National Cancer Institute-funded trials fails to enroll a single subject; half fail to recruit enough to produce meaningful results. Now some credible claims: many trials are “aimed at polishing a doctor’s résumé, and produce meaningless results; many oncologists avoid cancer studies because they can be a money loser, and many patients shy away from trial participation- particularly when their cancer is less advanced and they can obtain treatment outside of trials.


The article, however, is swathed in some misleading conjectures. The article makes the suggestion that problems with recruitment are “one reason” and “the biggest barrier” to major strides in the “war on cancer” (hence the recruitment poster in the graphic above). Hard to reconcile this with Kolata’s contention elsewhere that many trials are useless. It’s also hard to square the claim with Kolata’s point, earlier in the article, that trials involving really promising drugs usually have no problems with recruitment. In one famous case, a Phase 1 trial testing endostatin at Harvard received 1000 inquires from patients for 3 slots in the trial (Pop quiz: see if you can guess which New York Times reporter wrote an article on endostatin that many commentators criticized for sensationalizing the drug’s promise?). Third, with only about 1 in 20 cancer drug candidates making it from phase 1 tests to FDA approval, a reasonable question to ask is whether preclinical researchers are validating their drug candidates properly. And finally, the article makes no mention of the fact that many studies have exceedingly narrow eligibility criteria. Many patients may be solicited for trial participation- but only a fraction meet eligibility criteria.

Still, Kolata’s article is enlightening and raises a number of intriguing questions that demand ethical analysis. I’ll discuss some of these in my next posting (photo credits: Joan Thewlis, 1918 Recruitment Poster, 2009).

2009 August 6 | Leave a comment | Tags:

BibTeX

@Manual{stream2009-90,
    title = {Help Wanted- For the War on Cancer},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2009,
    month = aug,
    day = 6,
    url = {http://www.translationalethics.com/2009/08/06/help-wanted-for-the-war-on-cancer/}
}

MLA

Jonathan Kimmelman. "Help Wanted- For the War on Cancer" Web blog post. STREAM research. 06 Aug 2009. Web. 22 Oct 2024. <http://www.translationalethics.com/2009/08/06/help-wanted-for-the-war-on-cancer/>

APA

Jonathan Kimmelman. (2009, Aug 06). Help Wanted- For the War on Cancer [Web log post]. Retrieved from http://www.translationalethics.com/2009/08/06/help-wanted-for-the-war-on-cancer/

Tweet this post

Search STREAM

All content © STREAM research

admin@translationalethics.com
Twitter: @stream_research
3647 rue Peel
Montreal QC H3A 1X1

Site admin