Reliability of accelerated confirmatory trials during the COVID-19 pandemic: the consequences of low pre-trial plausibility

Résumé fourni par la source

OBJECTIVE: The reliability of confirmatory clinical trials (i.e. the post-trial probability) depends on the pre-trial probability. During the COVID-19 pandemic, many early COVID-19 confirmatory trials were launched despite limited phase II evidence and weak mechanistic or preclinical support. Whether this accelerated development strategy reduced the reliability of therapeutic evidence remains uncertain. STUDY DESIGN AND SETTING: Overall empirical pre-trial plausibility was estimated at the trial level as the proportion of early randomized trials evaluating interventions that were ultimately considered effective in the reference network meta-analysis. For each trial, counterfactual statistical power was estimated using the reported sample size and the number of events. Bayes' theorem was used to calculate the post-trial probability of efficacy, based on sensitivity (i.e. trial power), specificity (i.e. 1 minus 0.025), and empirical pre-trial plausibility. We then compared the positive predictive values of COVID-19 and non-COVID-19 development pipelines. RESULTS: Of the 463 early trials evaluating 150 candidate molecules, 51 assessed interventions ultimately supported by the reference network meta-analysis, corresponding to an empirical trial-level pre-trial plausibility of 11%. The median power of COVID-19 trials was 9% (min-max: 6-100%). Accordingly, the probability that a statistically significant result reflected true efficacy was 31% (20%-83%), compared to 94% for non-COVID19 trials characterized by the conventional phase II-to-phase III success benchmark of 50%. With an empirical pre-trial plausibility of 11%, a replication framework consisting of two independent, 80%-powered trials yielded a positive predictive value of 99.2%. This contrasts with a single 90%-95%-powered trial, which yielded a positive predictive value of 81%-82%. CONCLUSION: The early COVID-19 trial ecosystem was frequently underpowered and conducted under conditions of low pre-trial plausibility, resulting in lower post-trial probabilities of true efficacy than in conventional development pathways. These findings highlight the need for stronger pre-trial evidence, coordinated trial prioritization, and replication-oriented strategies in future pandemic responses.