Publishing for science or science for publications? The role of open science to reduce research waste

In an earlier issue, Chapelle et al report on "an epidemic of redundant meta-analyses."1 They show all the evidence to answer a single clinical question: Should cancer patients with venous thrombosis receive either a direct oral anticoagulant or low molecular weight heparin? Indeed, the combined data from four trials suggest a clinical benefit for direct oral anticoagulants, but this is not probably not the main reason why this letter was published in the Journal. The letter draws our attention to something different, and in the greater scheme of things, arguably more important. When the authors searched the literature to help draft a French clinical guideline on the topic, they noticed that even though there are five trials published on the topic, there are currently no less than 20 meta-analyses published. They also show that the published meta-analyses are not only numerous, but also ubiquitous and published nearly simultaneously: these 20 meta-analyses were published in 17 different journals in the span of 3 years. What can explain this? Let us first look at some reasons that are in line with a solid scientific workflow. Reproducibility and replication of research is a cornerstone of good scientific practice. The ability to repeat the activity and independently come to the same conclusions as others – or not! – is what sets science apart from mere experience-based reasoning. Another more practical reason for more than one publication on a similar topic can be that this topic lies in multiple clinical domains, so publishing in different journals each catering those fields with their own focus might be prudent, perhaps even desirable. Also, with the arrival of additional data meta-analyses can or perhaps even should be updated, a practice reminiscent of the Cochrane approach (www.cochrane.org) or living systematic reviews.2 But do these reasons render the conclusion of “an epidemic of redundant meta-analyses” void? No. First of all, even though science should in principle be reproducible to show the diligence of our work, it does not mean that we have to replicate everything ad nauseam. Sure, the topic in case does cover two fields (thrombosis and cancer) and not all trials were not published all simultaneously, but with 11 of the 20 reviews published within 6 months after the publication of the most recent trial it is clear that something else is going on. Are there reasons that we can think of that are less in line with a solid scientific workflow? Let's start with the situation in which multiple people came up with the same idea at roughly the same time. There are two versions of this: benign in the form of chance or malignant in the form of scooping. Sometimes the time is just right to answer a certain question and multiple people start new projects. In those cases, preregistration of research ideas is often mentioned as a method of prevention of duplicate work, but more on that later. Other reasons might be that meta-analyses are also often seen as good tool for starting researchers to get a lay of the land, ignoring that a solid meta-analysis requires highly skilled and experienced researchers in their field. Perhaps what we see here is the result of a push of the authors for a certain message. This too comes in two forms: benign when the arguments are made on behalf of patients, but less so when the argument is made for other interests (e.g., commercial ones). Either way, this is not proper science. Finally, and perhaps the most worryingly, is that what we see is an example that science projects are executed to acquire more publications for the publication’s sake, and not, like it should be, publications that further science and medicine. This practice, which we will dub “science for publications” is one of the root causes of what has become known as “research waste,” research efforts that have little or no added scientific value. Given the constraints in time and resources in academic medicine, writing, publishing, and reading research waste has serious consequences.2 So what can we do to prevent research waste, especially originating from double work? The identification of and research into “research waste” has given impetus to numerous activities that can be categorized under the broad umbrella of Open Science. Would Open Science practices have prevented the “epidemic of redundant meta-analyses”? One of the underlying ideas of Open Science is that when scientists are open about what they are doing, and what they have been up to, double work can be prevented. For example, Prospero (https://www.crd.york.ac.uk/prospero/), a registry in which authors can file their initiative to execute a systematic review and meta-analysis can indeed fulfill that role. However, Chapelle et al show that only 10 of the 20 meta-analyses were indeed preregistered. To make things worse, Prospero still shows three entries of ongoing systematic reviews focused on the same clinical question (CRD42020185420, CRD42020170082, and CRD42019112344 identified on 13 April 2021). Prospero works are far from optimal: there is a backlog in approval of new registrations and only 35 or fewer of all registrations result in an actual publication, depending on the field. But even if that were not the case, the data from Chapelle et al3-5 show that of the 10 preregistered meta-analyses, no fewer than eight proposals were submitted to Prospero, whereas at least one, and often multiple entries, for the same meta-analysis were already registered. This suggests that it is not the lack of knowledge on the efforts of others to answer the same research question that led to all these meta-analyses. Another way to reduce redundant publications is to share research before it is peer-reviewed by publishing it on a preprint server such as medRxiv.org. Although the coronavirus disease 2019 pandemic has popularized this practice, it is only used for a small fraction of all research output. Would further adoption of this practice be a way to further reduce research waste? The data collected by Chapelle et al suggest that the time window between “received” and “published online” was short. Preprints will only prevent double work when there is a sufficiently large window between these two timepoints during which other researchers have to decide whether or not to start a new project. In the case of Chapelle et al, this is clearly not the case, and one could also wonder whether preprints would deter authors with ulterior motives. But what if the authors of the last trial just added an update to the most recent existing forest plot in the discussion section of their trial publication? This would not only bring the trial results in a quantified context, but in this case might prevent 11 subsequent useless publications. Sure, this approach requires some additional coordination, and disregards the idea that meta-analyses require a completely different set of skills, but if the authors of the earlier meta-analysis shared the analytical code and a small dataset on platforms like an Open Science Framework or GitHub, the analysis would be straightforward and the effort in terms of time and resources minimal. Again, this assumes sincere scientific motives of authors. This brings us full circle. The phenomenon described by Chapelle et al is an extreme version of research waste. As long as the scientific enterprise incentivizes research waste and science for publications, time and resources are wasted. Open science practices cannot counteract this because they do not address the root cause. Of note, compared with a more classic approach, an open science workflow has several additional steps that costs both time and resources that is clearly at odds of the incentive structure in science. Authors have to adopt to a new way of working that does not only involve counting notches on belts. Drivers of change in science – editors, administrators, and funders – have to realize that demanding adherence to selected open science practices whilst not addressing the elephant in the room will only be counterproductive on the long term. The reason to publish needs to be put back in line with the overall goal of the medical sciences (i.e., promote health of all and healing of the sick through a deep understanding of the human body). Only if we do this can research waste truly be reduced. The authors report no conflict of interest. Bob Siegerink: conceptualization, writing of original draft. Frits R. Rosendaal: conceptualization, writing, review, and editing.