Nature has an article making the rounds this week, decrying the dangers of preprints.
On the surface, this is a bit like an article by foxes decrying the dangers of henhouses. There’s a pretty big conflict of interest when a journal like Nature, that makes huge amounts of money out of research scientists would be happy to publish for free, gets snippy about scientists sharing their work elsewhere. I was expecting an article about how “important” the peer review process is, how we can’t just “let anyone” publish, and the like.
Instead, I was pleasantly surprised. The article is about a real challenge, the weakening of journalistic embargoes. While this is still a problem I think journalists can think their way around, it’s a bit subtler than the usual argument.
For the record, peer review is usually presented as much more important than it actually is. When a scientific article gets submitted to a journal, it gets sent to two or three experts in the field for comment. In the best cases, these experts read the paper carefully and send criticism back. They don’t replicate the experiments, they don’t even (except for a few heroic souls) reproduce the calculations. That kind of careful reading is important, but it’s hardly unique: it’s something scientists do on their own when they want to build off of someone else’s paper, and it’s what good journalists get when they send a paper to experts for comments before writing an article. If peer review in a journal is important, it’s to ensure that this careful reading happens at least once, a sort of minimal evidence that the paper is good enough to appear on a scientist’s CV.
The Nature article points out that peer review serves another purpose, specifically one of delay. While a journal is preparing to publish an article they can send it out to journalists, after making them sign an agreement (an embargo) that they won’t tell the public until the journal publishes. This gives the journalists a bit of lead time, so the more responsible ones can research and fact-check before publishing.
Open-access preprints cut out the lead time. If the paper just appears online with no warning and no embargoes, journalists can write about it immediately. The unethical journalists can skip fact-checking and publish first, and the ethical ones have to follow soon after, or risk publishing “old news”. Nobody gets the time to properly vet, or understand, a new paper.
There’s a simple solution I’ve seen from a few folks on Twitter: “Don’t be an unethical journalist!” That doesn’t actually solve the problem though. The question is, if you’re an ethical journalist, but other people are unethical journalists, what do you do?
Apparently, what some ethical journalists do is to carry on as if preprints didn’t exist. The Nature article describes journalists who, after a preprint has been covered extensively by others, wait until a journal publishes it and then cover it as if nothing had happened. The article frames this as virtuous, but doomed: journalists sticking to their ethics even if it means publishing “old news”.
To be 100% clear here, this is not virtuous. If you present a paper’s publication in a journal as news, when it was already released as a preprint, you are actively misleading the public. I can’t count the number of times I’ve gotten messages from readers, confused because they saw a scientific result covered again months later and thought it was new. It leads to a sort of mental “double-counting”, where the public assumes that the scientific result was found twice, and therefore that it’s more solid. Unless the publication itself is unexpected (something that wasn’t expected to pass peer review, or something controversial like Mochizuki’s proof of the ABC conjecture) mere publication in a journal of an already-public result is not news.
What science journalists need to do here is to step back, and think about how their colleagues cover stories. Current events these days don’t have embargoes, they aren’t fed through carefully managed press releases. There’s a flurry of initial coverage, and it gets things wrong and misses details and misleads people, because science isn’t the only field that’s complicated, real life is complicated. Journalists have adapted to this schedule, mostly, by specializing. Some journalists and news outlets cover breaking news as it happens, others cover it later with more in-depth analysis. Crucially, the latter journalists don’t present the topic as new. They write explicitly in the light of previous news, as a response to existing discussion. That way, the public isn’t misled, and their existing misunderstandings can be corrected.
The Nature article brings up public health, and other topics where misunderstandings can do lasting damage, as areas where embargoes are useful. While I agree, I would hope many of these areas would figure out embargoes on their own. My field certainly does: the big results of scientific collaborations aren’t just put online as preprints, they’re released only after the collaboration sets up its own journalistic embargoes, and prepares its own press releases. In a world of preprints, this sort of practice needs to happen for important controversial public health and environmental results as well. Unethical scientists might still release too fast, to keep journalists from fact-checking, but they could do that anyway, without preprints. You don’t need a preprint to call a journalist on the phone and claim you cured cancer.
As open-access preprints become the norm, journalists will have to adapt. I’m confident they will be able to, but only if they stop treating science journalism as unique, and start treating it as news. Science journalism isn’t teaching, you’re not just passing down facts someone else has vetted. You’re asking the same questions as any other journalist: who did what? And what really happened? If you can do that, preprints shouldn’t be scary.
An insightful view.
In practice, lots of the would-be serious science journalists really discuss the paper just published in a paper journal as news, ignoring the pre-existence. The publication in a paper journal is their justification for discussing it at all, but the true motivation why they do it is usually that the paper says something sexy, far-reaching, crazy, or there are some political and societal implications that they like.
Given the true motivation, I find the very act of publication in a paper journal to be a deeply insufficient reason to hype something. When the waiting was done at all, the journalists should wait at least for some 5-10 followups (citations) of that paper, too. A paper chosen by a combination of “paper publication” and “cool ambitions” is still a pretty lousy one in average, not much different from a preprint chosen by the “cool” factor. If one wants to eliminate some pure nonsense that is hyped by the authors, the paper publication just doesn’t seem enough because paper can withstand any treatment.
I had seen that Nature reference somewhere else this week. I thought it was a 20-year-old article, sort of a ‘flashback’, ignored even clicking on it. Stunned to see it written this week. The ‘author’ of the article states that preprints might be wrong, so we should not let them out in the wild. Hilarious.
Real, published papers in top journals have a less than 50% chance of being repeatable (if they are experimental results) and theory is, well, just that. https://www.nature.com/collections/prbfkwmwvz/ , https://www.bbc.com/news/science-environment-39054778, etc. There are zero problems with preprints – the whole idea is that they are not even reviewed – and many many problems with peer review.
As far as I could see the author was less concerned that preprints might be wrong, as that journalists won’t have enough time to understand them without a journalistic embargo. I can understand that on some level, scientific articles take time to understand… but so do current events! If most journalists can get by without an embargo, I suspect science journalists can as well.
Tetragraviton, I think that you are vastly overstating the usefulness of the delay for the journalists. Journalists aren’t thinking about a scientific topic for a week or month. Most of such articles may be written down immediately – in an hour – and journalists don’t think much, don’t investigate, don’t ask questions about their own sentences – most of them are not even able to do such things. So the delays and embargos don’t really help the quality much and in effect, their only role is to make some journalists more important than others if they can just release something immediately at some moment, while others learn it from them.
Pingback: Preprints and science news | Trading Knowledge