This week, we had a colloquium talk by the managing editor of the Open Journal of Astrophysics.

The Open Journal of Astrophysics is an example of an arXiv overlay journal. In the old days, journals shouldered the difficult task of compiling scientists’ work into a readable format and sending them to university libraries all over the world so people could stay up to date with the work of distant colleagues. They used to charge libraries for the journals, now some instead charge authors per paper they want to publish.

Now, most of that is unnecessary due to online resources, in my field the arXiv. We prepare our papers using free tools like LaTeX, then upload them to arXiv.org, a website that makes the papers freely accessible for everybody. I don’t think I’ve ever read a paper in a physical journal in my field, and I only check journal websites if I think there’s a mistake in the arXiv version. The rest of the time, I just use the arXiv.

Still, journals do one thing the arXiv doesn’t do, and that’s refereeing. Each paper a journal receives is sent out to a few expert referees. The referees read the paper, and either reject it, accept it as-is, or demand changes before they can accept it. The journal then publishes accepted papers only.

The goal of arXiv overlay journals is to make this feature of journals also unnecessary. To do this, they notice that if every paper is already on arXiv, they don’t need to host papers or print them or typeset them. They just need to find suitable referees, and announce which papers passed.

The Open Journal of Astrophysics is a relatively small arXiv overlay journal. They operate quite cheaply, in part because the people running it can handle most of it as a minor distraction from their day job. SciPost is much bigger, and has to spend more per paper to operate. Still, it spends a lot less than journals charge authors.

We had a spirited discussion after the talk, and someone brought up an interesting point: why do we need to announce which papers passed? Can’t we just publish everything?

What, in the end, are the referees actually for? Why do we need them?

One function of referees is to check for mistakes. This is most important in mathematics, where referees might spend years making sure every step in a proof works as intended. Other fields vary, from theoretical physics (where we can check some things sometimes, but often have to make do with spotting poorly explained parts of a calculation), to fields that do experiments in the real world (where referees can look for warning signs and shady statistics, but won’t actually reproduce the experiment). A mistake found by a referee can be a boon to not just the wider scientific community, but to the author as well. Most scientists would prefer their papers to be correct, so we’re often happy to hear about a genuine mistake.

If this was all referees were for, though, then you don’t actually need to reject any papers. As a colleague of mine suggested, you just need the referees to publish their reports. Then the papers could be published along with comments from the referees, and possibly also responses from the author. Readers could see any mistakes the referees found, and judge for themselves what they show about the result.

Referees already publish their reports in SciPost much of the time, though not currently in the Open Journal of Astrophysics. Both journals still reject some papers, though. In part, that’s because they serve another function: referees are supposed to tell us which papers are “good”.

Some journals are more prestigious and fancy than others. Nature and Science are the most famous, though people in my field almost never bother to publish in either. Still, we have a hierarchy in mind, with Physical Review Letters on the high end and JHEP on the lower one. Publishing in a fancier and more prestigious journal is supposed to say something about you as a scientist, to say that your work is fancier and more prestigious. If you can’t publish in any journal at all, then your work wasn’t interesting enough to merit getting credit for it, and maybe you should have worked harder.

What does that credit buy you? Ostensibly, everything. Jobs are more likely to hire you if you’ve published in more prestigious places, and grant agencies will be more likely to give you money.

In practice, though, this depends a lot on who’s making the decisions. Some people will weigh these kinds of things highly, especially if they aren’t familiar with a candidate’s work. Others will be able to rely on other things, from numbers of papers and citations to informal assessments of a scientist’s impact. I genuinely don’t know whether the journals I published in made any impact at all when I was hired, and I’m a bit afraid to ask. I haven’t yet sat on the kind of committee that makes these decisions, so I don’t know what things look like from the other side either.

But I do know that, on a certain level, journals and publications can’t matter quite as much as we think. As I mentioned, my field doesn’t use Nature or Science, while others do. A grant agency or hiring committee comparing two scientists would have to take that into account, just as they have to take into account the thousands of authors on every single paper by the ATLAS and CMS experiments. If a field started publishing every paper regardless of quality, they’d have to adapt there too, and find a new way to judge people compatible with that.

Can we just publish everything, papers and referee letters and responses and letters and reviews? Maybe. I think there are fields where this could really work well, and fields where it would collapse into the invective of a YouTube comments section. I’m not sure where my own field sits. Theoretical particle physics is relatively small and close-knit, but it’s also cool and popular, with many strong and dumb opinions floating around. I’d like to believe we could handle it, that we could prune back the professional cruft and turn our field into a real conversation between scholars. But I don’t know.