4329Mega-studies in heart failure, effect dilution in examination of new therapies

Abstract Aims All phase 3 studies in patients with acute heart failure (AHF) and HF with preserved ejection fraction (HFpEF) have failed in the last decades. We explore the likelihood that the negative results are due to chance and/or to study size and dilution of statistical power. Methods and results First, using simulations, we examined the probability that a positive finding in phase 2 would result in studying truly effective drugs in phase 3. We simulated phase 2 studies under six scenarios where the range of true relative risk (RR) for an outcome of interest varied from 0.5 (major benefit) to 1.15 (some harm). The proportion of simulated studies where the RR <0.8 (we assumed that a 20% or greater risk reduction reflects an effective drug) ranged from 6% to 42% across the six scenarios studied. To further simulate “real life” clinical research, we simulated a continuous surrogate outcome that was linearly related to the true RR in each simulation of each scenario. Regardless of criteria considered for a positive phase 2 trial, results suggest that even in our worst-case scenario, where overall only 6% of drugs taken into phase 2 are effective, roughly 20% of phase 3 studies, if appropriately powered, should have yielded positive results. Given this, we then explored study size in AHF research, as a potential explanation for the high failure rate in these studies. Comparison of published phase 2 and 3 clinical trials with registries in AHF suggest that populations in both large and small trials differ from “real life”. Meta-regression models suggest that both control event rates, and in the serelaxin program as an example, treatment effects, decline with increasing study size greatly reducing power (figure). This effect dilution might be explained by an increasing proportion of patients enrolled in studies who cannot benefit from the study drug. Figure 1. Power at two-sided 0.05 significance level to detect an effect size of hazard ratio of 0.65 (left) or 0.8 (right) with a placebo event rate of 10% (top) and 20% (bottom) at N=100 at various treatment effect dilutions with increasing sample size. Conclusion These data suggest that it is unlikely that the very high rate of negative AHF phase III trials can be explained by chance alone. Potentially, our tendency to increase sample size does not necessarily increase statistical power, due to more heterogenous populations leading to reduced event rates and treatment effects.