Trials and tribulations of coronavirus disease-2019 research: with a few bright lights in the fog

The scientific community is responding to coronavirus disease-2019 (COVID-19) at an unprecedented speed, openly sharing biomedical knowledge, educational initiatives, and clinical studies.1 Over 35 000 COVID-19 articles have been referenced in PubMed since the first New England Journal of Medicine COVID-19 report on 24 January 2020.2 Moreover, approximately 7000 preprints have appeared on the medRxiv and bioRxiv platforms. Cardiological journals have published many important articles on COVID-19 cardiovascular implications.3 The urgency to fight the disease has triggered a compelling rush to undertake research. An extraordinary number of studies have been rapidly designed; 2749 COVID-19-related investigations are now registered in ClinicalTrials.gov, including 1538 interventional or randomized, and 1187 observational.4,5 Many of these studies, however, are too small and have major flaws. For example, of the 145 registered trials dealing with hydroxychloroquine, 32 have sample sizes of 100 or less, 10 have no control group, 12 are nonrandomized, and only 50 are multicentre.6 An appraisal of 35 studies approved by the Italian Drug Agency (AIFA) following positive evaluation by a single national Ethics Committee (the National Institute for Infectious Disease Lazzaro Spallanzani IRCCS, Rome) found methodological flaws and small sample size in most. Although 29 of 35 studies (83%) were randomized, 22 were open-label, and more than half of these were susceptible to biased endpoint evaluation. Many were based on over-optimistic assumption of benefit, resulting in a high risk of inconclusive results.7 An analysis of over 1200 COVID-19 studies by the electronic journal Web Stat, in cooperation with Applied XLab, also reported frequent under-power (39% recruiting fewer than 100 patients), inertia or lack of patients (38% not starting enrollment) and over-emphasis on potential treatments, such as hydroxychloroquine. For interventional studies, the failure to analyse a properly randomized prospective control arm has repeatedly led to false conclusions in the past. This was the case, for example, for hormone-replacement therapy; early retrospective analyses suggested that therapy cuts heart disease occurrence in half but subsequent randomized placebo-controlled trials (RCTs) showed no impact on heart disease, with increased risk of breast cancer, stroke and venous thromboembolism.8 In addition to interventional studies, a deluge of observational studies has been planned worldwide. For instance, the Ethics Committee of Emilia Centro in Bologna alone approved over 190 observational studies by a fast track procedure in February to July 2020. Observational studies can provide useful complementary information to RCTs but they also require rigorous methodology and adequate sample size. Protocol duplications and lack of coordination are causing massive waste.9 An emerging problem in the myriad of ongoing COVID-19 investigations is the difficulty to find patients for enrollment, at least in parts of Europe with current outbreak mitigation. About one-third of the studies (819 out of 2749 studies) posted on ClinicalTrials.gov are not yet recruiting patients. Good news for the public, but a further drawback for clinical research, given the risk of premature halting of studies. In this scenario, duplication and lack of aggregation are particularly relevant major flaws of many studies. Sharing individual patient data from similar investigations would be desirable.7,10 Collaborative pooling of resources to achieve sufficient power of scale has been encouraged at national and international levels by authoritative scientists11 and institutional organisations, such as WHO12 and the European Medicines Agency (EMA).13 This effort would reduce the waste of resources and avoid duplicates of underpowered studies. The race to publish original findings on COVID-19 and the absence of careful data review have generated scepticism and in some cases diffuse alarm, with potential prejudice extended to non-COVID-19 studies as well. Two major journals have expressed concern about the databases underlying two recent COVID-19 studies. On 3 June 2020, the Lancet announced retraction of a highly cited registry study suggesting that hydroxychloroquine might cause more harm than benefit.14,15 Hours later, the New England Journal of Medicine announced retraction of a cohort study by some of the same authors of the retracted Lancet article, reporting on cardiovascular comorbidities in COVID-19.16,17 The two articles had originally been published online on 22 May and 1 May 2020. Three authors of the Lancet article based their retraction on the inability to access the databases and verify the integrity of the analyses conducted by Chicago-based Surgisphere Corp and study coauthor Sapan Desai, Surgisphere's founder and Chief Executive Officer. The expression of concern published by both journals followed an open letter, endorsed by more than 200 scientists, ethicists, and clinicians, posted on 28 May 2020, questioning the data and ethics of the Lancet study. Retraction of two studies in leading medical journals not only is a vulnus for the whole scientific community and an attempt to the reputation of medical research but also has very relevant clinical implications, considering that, on the basis of the not-yet-retracted Lancet data, the WHO had temporarily halted enrollment in the hydroxychloroquine component of the SOLIDARITY trial. Another criticism of COVID-19 studies is how reporting occurs. The postings in medRxiv, a free online archive and distribution server for unpublished manuscripts, have increased tremendously in the first semester of 2020. Preprints are preliminary reports of works that have not been certified by peer review. They should not be relied upon to guide clinical practice or health-related behaviour and should not be reported in news-media as established information. Preprints have the merit of providing valuable early access to study results. As a counterpart, an early irresponsible spread of not-yet-validated results may induce false expectations and illusions. This was the case of the first small study of hydroxychloroquine reported on 20 March 202018 that prompted the president of the United States to enthusiastically tweet 'Hydroxycloroquine & azithromycin, taken together, have a real chance to be one of the biggest game changers in the history of medicine'. A further criticism is the not-so-rare reporting of preliminary results in social media and in the popular press before scientific presentation and publication occur. This was the case of the dexamethasone study announced on 16 June 2020 not in a research article but as a media announcement. These ways of communicating may generate confusion and conflicting interpretations of the results among clinicians, public, and politicians.19 Cochrane COVID Rapid Reviews is seeking to provide rapid evidence-based information to clinicians, policy makers, and the public to inform decision-making during the coronavirus pandemic. Cochrane has engaged with WHO and other international stakeholders to build a bank of questions, which is constantly being prioritized and refined. Screening searches keep up to date with non-Cochrane rapid reviews to help reduce duplication of effort. Similarly, to evaluate the quality of the huge number of COVID-19 articles, the Massachusetts Institute of Technology in Boston and the University of California in Berkeley are launching together an open access, rapid-review overlay journal called 'Rapid Reviews: COVID-19 (RR:C19)'; the aim is to accelerate peer review and deliver real-time, verified scientific information that policy makers and health leaders can use. Assisted by artificial intelligence/machine learning tools, a global team will validate, or debunk, the rapidly growing volume of COVID-19-related manuscripts on preprint servers. Two recent examples demonstrate that even in the context of a pandemic, it is possible to perform scientifically robust and ethically sound clinical research. The examples are represented by the Adaptive COVID-19 Treatment Trial (ACTT-1) with remdesivir20 and, even more so, by the RECOVERY trial arm with dexamethasone,21 providing some clear guidance on therapeutic strategies for COVID-19. The study with remdesivir is a randomized, double-blind, placebo-controlled trial of intravenous remdesevir in hospitalized COVID-19 patients with evidence of lower respiratory tract involvement. The trial was halted prematurely after randomization of 1059 patients because of remdesivir superiority over placebo in shortening median recovery time (11 vs. 15 days) with a nonsignificant reduction of 14-day mortality (7.1% with remdesivir vs. 11.9% with placebo). Unfortunately, the premature interruption of ACTT-1 prevented a definite evaluation of the effects of remdesivir on total mortality. The dexamethasone trial arm is part of RECOVERY, a randomized, controlled, open-label trial comparing a range of possible treatments in hospitalized COVID-19 patients. A total of 2024 patients were randomized to oral or intravenous dexamethasone and 4231 to receive usual care. The primary outcome was 28-day mortality. The survival benefit of the glucocorticoid dexamethasone for COVID-19 patients who were receiving mechanical ventilation at the time of randomization was clearly shown (29.3% mortality with dexamethasone vs. 41.4% with usual care). RECOVERY is an adaptive trial, that is, a large, simple trial using a platform or master approach in which agents can be added or subtracted from the randomization as trial data emerge or new agents become available. RECOVERY investigators have already reported a lack of efficacy for hydroxychloroquine, ending that trial arm in June 2020,22 and 20 days later announced in a press release also a lack of efficacy of lopinavir-ritonavir, an antiviral treatment commonly used to treat HIV.23 The trial continues to investigate the roles of azithromicyn, tocilizumab, and convalescent plasma (collected from donors who recovered from COVID-19). The keys for the success of these trials are their simplicity and institutional endorsement (e.g. by the National Health Service of the UK). Noticeably, 15% of all COVID-19 patients who were hospitalized in the UK were enrolled in RECOVERY. The two trials with remdesivir and dexamethasone clearly demonstrate that even in the setting of an outbreak, it is possible to do rigorous, high-quality clinical research.24 The rewarding cascade of this approach is the identification of two effective therapies in the nightmare of the many proposed treatments. The same rigour of approach as in the RECOVERY trial is being pursued by other trials with national and global coordination and public--private partnership. They include the WHO-endorsed SOLIDARITY trial, testing four therapeutic strategies, the Accelerating Covid-19 Research and Development (ACCORD) platform in the UK, and the Accelerating Covid-19 Therapeutics Interventions and Vaccines (ACTIVE) in the United States. Finally, patient participation in clinical trials represents an ethical value and a generosity towards the community. By participating in RCTs, patients directly contribute to the discovery of new therapies. The COVID-19 pandemic has raised the further issue of fair allocation of scarce medical resources and, in the opinion of some ethicists, people who participate in research to prove effectiveness and safety of therapeutics should be rewarded for their contribution by receiving priority interventions.25 Similarly, it would be desirable to overcome any individualism that may characterize researchers and to consider the ethical implications of enrolling a patient into valuable clinical research with a true potential favourable impact on patient outcomes. Conducting high-quality rigorous studies in the midst of a dramatic health emergency is not easy. However, ongoing RCTs like SOLIDARITY and RECOVERY demonstrate that it is possible and is probably the right way. In the absence of effective treatments, it is ethical to randomize patients to different promising strategies and collaborative investigations. Simple, pragmatic, open-label designs, with hard endpoints not susceptible to biased evaluation should be pursued. A pandemic emergency is a reason to work faster but the potential risks for patients caused by experimental interventions should not be forgotten. Another no less important risk is the generation of data with small chances of showing what really works. Rigorous research -- even and especially -- within a pandemic is an ethical imperative for both science and patient safety. Acknowledgements All authors equally contributed to the critical evaluation and to the whole content of the manuscript. Conflicts of interest G.D.P. has received personal fees for lectures from Boheringer Ingelheim, Bayer, BMS-Pfizer, and Daiichi Sankyo outside the present work. A.P.M. received personal fees from Bayer, Fresenius, Novartis for the participation in study committees outside the present work. C.G. has received personal fees from MSD, ViiV, Gilead, and Janseen Cilag for services outside the present work. F.A. has received personal fees from Amgen, Bayer, BMS-Pfizer, and Daiichi Sankyo outside the present work.