Reproducibility of in-Hospital Worsening Heart Failure Event Adjudication in the RELAX-AHF-EU Trial

In-hospital worsening heart failure (WHF) is a commonly employed endpoint in acute heart failure (AHF) trials that implies deterioration of the signs and/or symptoms of AHF that require treatment adaptation. Some recent AHF trials have established a Clinical Event Committee (CEC) to adjudicate in-hospital WHF events to ensure consistency in the assessment of reported events.1, 2 However, it remains largely unexplored whether the CEC adjudication of this endpoint, which reflects a bedside assessment by the treating physician, is reproducible. We here report data on the reproducibility of the CEC adjudication of 20 in-hospital WHF events from the RELAX-AHF-EU trial. RELAX-AHF-EU was a multinational, prospective, randomized, open-label, blinded-endpoint evaluation trial designed to understand the effect of serelaxin when added to standard of care vs. standard of care alone in patients admitted with AHF. The primary endpoint was in-hospital WHF or all-cause death out to 5 days from randomization. All primary endpoint events were ascertained by a CEC blinded to the assigned treatment. The design and results of the RELAX-AHF-EU trial have been reported previously.3 In-hospital WHF was defined as worsening of signs and/or symptoms of heart failure through day 5 that required intensification of intravenous heart failure therapy or mechanical ventilation, or renal or circulatory support. The operation and organization of the CEC were specified in a CEC Charter. All members of the CEC were physicians experienced in the management of patients with AHF. An external service provider (Icon, Dublin, Ireland) collected, reviewed and blinded the supporting documentation for protocol-defined events reported by investigators (electronic case report forms and medical records such as admission summaries, diagnostic test reports, discharge summaries). Completed endpoint packages were subsequently submitted electronically to an interactive workflow system for CEC review. Investigator-reported events were initially adjudicated by two members of the CEC. In case of disagreement, a third committee member reviewed the case. If the third reviewer was not able to make a decision, the final decision was made by the CEC Chair. The first 10 reported events in the trial were adjudicated collectively by all CEC members in a face-to-face meeting, as specified in the CEC Charter. A blinded re-adjudication of 20 in-hospital WHF cases in two batches of 10 cases each was performed. The first 10 cases submitted for re-adjudication were identified by the trial's Sponsor clinical team in collaboration with the CEC Chair, because they were considered particularly challenging. The next 10 cases were selected randomly from those events where there had been agreement between the first level reviewers. Of the 20 in-hospital WHF events submitted for blinded re-adjudication, 11 changed status compared to the original adjudication, corresponding to a discordance rate of 55% (kappa value of 0.08, 95% confidence interval −0.19 to 0.36). A major obstacle for drug development in AHF is uncertainty about the appropriate clinical endpoints to measure, which in turn relates to the limited understanding of the pathophysiology of AHF, the heterogeneity of the patient phenotypes, and the multiple underlying causes of acute decompensation. In-hospital WHF is an important clinical event in AHF and has been associated with longer length of stay and increased risk of death and rehospitalisation.4, 5 Based on the assumption that prevention of in-hospital WHF would translate into improved long-term outcomes, in-hospital WHF has become a key endpoint in many recent AHF trials.6 However, the assessment of an in-hospital WHF episode ultimately rests on clinical judgement; which may vary from investigator to investigator and also depend on local or regional practice. The subjective components integral to the in-hospital WHF event assessment may introduce bias and increased noise in reporting of the endpoint. Efforts to mitigate bias and improve the signal-to-noise ratio has centred on employment of a stringent in-hospital WHF event definition, expanded and standardized data collection through questionnaires and detailed patient narratives, and in some trials the use of a CEC to adjudicate reported events. Expectedly, these measures have resulted in lower event rates.3, 6, 7 The data presented here highlight that classification of in-hospital WHF remains challenging despite the use of a CEC to adjudicate events. These results may relate to several issues. Adjudicators do not assess the patient at the bedside, but rather interpret the reported observations of multiple clinical parameters interpreted and collected by other physicians over time. Many patients have co-morbidities, including serious respiratory disease, which complicate the assessment of changes in clinical status. Finally, deterioration in symptoms may be difficult to attribute to worsening of symptoms rather than initial undertreatment in the initial time period following admission. It is important to recognize the limitations associated with this analysis, including the low number of events, the non-random selection of the first 10 cases submitted for re-adjudication, and the lack of formal incorporation of the process into the trial protocol. Despite these shortcomings, the results strongly suggest that future AHF trials that employ in-hospital WHF adjudicated by a CEC as an endpoint, a thorough quality control process should be pre-specified to understand and document the reproducibility of the adjudications. The value of in-hospital WHF as a measure of clinical response in future AHF trials depends on an in-hospital WHF definition that ensures that the assessment is reproducible and preserves a reasonable event rate. A different question is whether short-term intervention strategies to improve long-term outcomes in AHF remain a viable concept following the many recent failures of AHF trials1 (and RELAX-AHF-2, Teerlink J. and Metra M., unpublished data) or whether short-term symptom relief can be considered a relevant treatment objective in its own right. In conclusion, based on a sample of 20 blinded, but not completely randomly selected re-adjudications, we found that CEC adjudication of in-hospital WHF events in the RELAX-AHF-EU trial may be discordant at an unacceptable level between reviewers. These data should be confirmed or refuted in a well-powered larger study of unselected cases. The trial was funded by Novartis Pharma AG. Conflict of interest: A.P.M., E.L., F.M.T., K.W., E.E., U.D.,G.E., O.W.N., J.L.S. and M.R.C. have consulted for or received research support from Novartis, sponsor of the RELAX-AHF-EU trial. A.P.M. has consulted for Bayer and Fresenius. E.L. has consulted for or received research grants from ZOLL Medical Corporation, Boehringer Ingelheim and Servier. K.W. has consulted for or received research grants from Medtronic, Cardiac Dimension, AstraZeneca, Bayer and Napp. E.E. has consulted for Bayer, Merck Darmstadt and Vifor Pharma. U.D. has consulted for and received research grants from AstraZeneca. G.E. has consulted for or received research grants from Bayer, Vifor Pharma and Boehringer Ingelheim. J.L.S. has consulted for or received research grants from Bayer, Boehringer Ingelheim, Sanofi, Manarini, MERK, Pfizer and ROVI. M.R.C. has consulted for or received grants from Bayer, Servier, Boston Scientific, Medtronic, Abbott, ResMed, Boehringer Ingelheim, Neutronik and Fire1Foundry. J.H., C.C., T.H. and C.G. are all employees of Novartis.