An older professor in my field has a quirk: every time he organizes a conference, he publishes all the talks in a conference proceeding.
In some fields, this would be quite normal. In computer science, where progress flows like a torrent, new developments are announced at conferences long before they have the time to be written up carefully as a published paper. Conference proceedings are summaries of what was presented at the conference, published so that anyone can catch up on the new developments.
In my field, this is rarer. A few results at each conference will be genuinely new, never-before-published discoveries. Most, though, are talks on older results, results already available online. Writing them up again in summarized form as a conference proceeding seems like a massive waste of time.
The cynical explanation is that this professor is doing this for the citations. Each conference proceeding one of his students publishes is another publication on their CV, another work that they can demand people cite whenever someone uses their ideas or software, something that puts them above others’ students without actually doing any extra scientific work.
I don’t think that’s how this professor thinks about it, though. He certainly cares about his students’ careers, and will fight for them to get cited as much as possible. But he asks everyone at the conference to publish a proceeding, not just his students. I think he’d argue that proceedings are helpful, that they can summarize papers in new ways and make them more accessible. And if they give everyone involved a bit more glory, if they let them add new entries to their CV and get fancy books on their shelves, so much the better for everyone.
My guess is, he really believes something like that. And I’m fairly sure he’s wrong.
The occasional conference proceeding helps, but only because it makes us more flexible. Sometimes, it’s important to let others know about a new result that hasn’t been published yet, and we let conference proceedings go into less detail than a full published paper, so this can speed things up. Sometimes, an old result can benefit from a new, clearer explanation, which normally couldn’t be published without it being a new result (or lecture notes). It’s good to have the option of a conference proceeding.
But there is absolutely no reason to have one for every single talk at a conference.
Between the cynical reason and the explicit reason, there’s the banal one. This guy insists on conference proceedings because they were more useful in the past, because they’re useful in other fields, and because he’s been doing them himself for years. He insists on them because to him, they’re a part of what it means to be a responsible scientist.
And people go along with it. Because they don’t want to get into a fight with this guy, certainly. But also because it’s a bit of extra work that could give a bit of a career boost, so what’s the harm?
I think something similar to this is why academic journals still work the way they do.
In the past, journals were the way physicists heard about new discoveries. They would get each edition in the mail, and read up on new developments. The journal needed to pay professional copyeditors and printers, so they needed money, and they got that money from investors by being part of for-profit companies that sold shares.
Now, though, physicists in my field don’t read journals. We publish our new discoveries online on a non-profit website, formatting them ourselves with software that uses the same programming skills we use in the rest of our professional lives. We then discuss the papers in email threads and journal club meetings. When a paper is wrong, or missing something important, we tell the author, and they fix it.
Oh, and then after that we submit the papers to the same for-profit journals and the same review process that we used to use before we did all this, listing the journals that finally accept the papers on our CVs.
Why do we still do that?
Again, you can be cynical. You can accuse the journals of mafia-ish behavior, you can tie things back to the desperate need to publish in high-ranked journals to get hired. But I think the real answer is a bit more innocent, and human, than that.
Imagine that you’re a senior person in the field. You may remember the time before we had all of these nice web-based publishing options, when journals were the best way to hear about new developments. More importantly than that, though, you’ve worked with these journals. You’ve certainly reviewed papers for them, everyone in the field does that, but you may have also served as an editor, tracking down reviewers and handling communication between the authors and the journal. You’ve seen plenty of cases where the journal mattered, where tracking down the right reviewers caught a mistake or shot down a crackpot’s ambitions, where the editing cleaned something up or made a work more appear more professional. You think of the journals as having high standards, standards you have helped to uphold: when choosing between candidates for a job, you notice that one has several papers in Physical Review Letters, and remember papers you’ve rejected for not meeting what you intuited were that journal’s standards. To you, journals are a key part of being a responsible scientist.
Does any of that make journals worth it, though?
Well, that depends on costs. It depends on alternatives. It depends not merely on what the journals catch, but on how often they do it, and how much would have been caught on its own. It depends on whether the high standards you want to apply to job applicants are already being applied by the people who write their recommendation letters and establish their reputations.
And you’re not in a position to evaluate any of that, of course. Few people are, who don’t spend a ton of time thinking about scientific publishing.
And thus, for the non-senior people, there’s not much reason to push back. One hears a few lofty speeches about Elsevier’s profits, and dreams about the end of the big for-profit journals. But most people aren’t cut out to be crusaders or reformers, especially when they signed up to be scientists. Most people are content not to annoy the most respected people in their field by telling them that something they’ve spent an enormous amount of time on is now pointless. Most people want to be seen as helpful by these people, to not slack off on work like reviewing that they argue needs doing.
And most of us have no reason to think we know that much better, anyway. Again, we’re scientists, not scientific publishing experts.
I don’t think it’s good practice to accuse people of cognitive biases. Everyone thinks they have good reasons to believe what they believe, and the only way to convince them is to address those reasons.
But the way we use journals in physics these days is genuinely baffling. It’s hard to explain, it’s the kind of thing people have been looking quizzically at for years. And this kind of explanation is the only one I’ve found that matches what I’ve seen. Between the cynical explanation and the literal arguments, there’s the basic human desire to do what seems like the responsible thing. That tends to explain a lot.
4 gravitons 
In the past, pre arXiv or the email list-servs of the late 1980s, there were still preprints. They were just physically mailed between the famous institutions of the world. Indeed, this left many at smaller places to wait until something was published in a journal, but there were still a number of papers from this time that were never published in a journal, and yet have come to be very influential (Ken Wilson’s model of particle production in hadron collisions, the so-called “Feynman-Wilson gas”, is an example of this). Amusingly, half of the references of Gross and Wilczek’s paper on the beta-function of QCD are listed as “To be published” and this was in 1973.
For-profit journals are definitely problematic in general, but in particle physics there are a number of non-profit journals (Physical Review and JHEP, at least). JHEP even pays referees a nominal stipend per paper; of course, not a lot, but definitely a symbolic amount at least representative of the work required of a referee. For the past 6+ years, nearly all journals that primarily publish particle physics papers have been members of SCOAP3 that agree to publish all papers free and open access to all. This of course was a consequence of arXiv which is already open source.
I disagree that journals serve no purpose and I do not agree with the simple statement that talking to an author about an error means they will fix it. Some will update their arXiv submission based on response from a talk or private communication, some may even retract a submission, but many many will simply leave it on arXiv because they can and there is no penalty to posting incorrect, erroneous, or misleading work there. Simply requiring one expert read your paper in detail, to check for logical consistency, that claims in the abstract are backed up in the body, that the body of literature is represented appropriately, is a significant test and validation for a paper. Peer review is definitely not perfect, but it is something, and absolutely no barrier to publishing would result in requiring wading through vast swaths of uselessness (which is apparently an issue especially in ML/AI these days). Sure, I might only read papers from arXiv, but having it subsequently published in a journal (and that the author updated the arXiv submission) is still a useful quality check. Actually, one could argue that today, with improvements in AI, that requiring a human to actually read through a paper is extremely vital for finding “fake” papers.
LikeLike
The APS journals are nonprofit, but isn’t JHEP published by Springer? It’s on behalf of SISSA, which I’m assuming doesn’t make a profit, but are you sure that Springer isn’t making a profit off of its services there?
(Those two are essentially all the traditional journals people in high-energy use these days in my experience, so if both are non-profit then I should probably put a correction in this post. I do think there’s a good case that moving to a cheaper more SciPost-like model, or even a more explicit arXiv overlay, would be an improvement since it would get essentially all of the benefits while costing the organizations that pay into SCOAP3 a lot less.)
I do think you’re overestimating how much those quality checks matter. Don’t forget, people can neglect to update arXiv even if their paper is changed by a journal. I’ve done that before (with superficial stuff, I updated the references the referees brought up but didn’t change the terms I made up the way the editor wanted. The terms in the arXiv version caught on in the community, not the ones in the journal version).
Realistically, I only check the journal version if I run into something that I think is actually a mistake, usually because I’m trying to code up a formula and it doesn’t match, and only because that’s usually faster than asking the author. A decent chunk of the time the journal version has the same error, because referees usually can’t catch that kind of thing.
Under what circumstances do you typically check the journal version, or check whether something has been published? Concretely, when does it come up? I feel like when I see a suspicious paper it comes up in an arXiv meeting before it’s had time to get into a journal, and when I see a low-quality paper it’s pretty clear from the abstract.
Will this change with AI? Maybe. But papers only have an effect when people build off of them, and anyone building off of a paper by necessity is reproducing some of it, and thus doing a more in-depth check than a reviewer can.
LikeLike
JHEP is published by Springer (since 2010; before that it was published by IOP), but is owned by SISSA. The SCOAP3 agreement states that, regarding open-access: “In contrast to other Open Access models where libraries, authors or agencies pay a fee for each article, in SCOAP3 participating publishers have a single contract with CERN that centrally pays Article Processing Charges (APCs) for the publication services provided. The publishers in turn eliminate or reduce subscription costs to all customers. The SCOAP3partners redirect the funds previously used to pay the subscriptions into a common pool which is then used for the central APC payments. This ensures efficient operations and allows to competitively set an APC level while primarily using funds already available in the publication system.”
Indeed, I rarely check a journal version over the arXiv, and some papers are easy to see are not worth one’s time simply from a scan on arXiv. And I agree that some changes imposed by a journal are not relevant for arXiv replacement (lots of formatting details at the proof stage, for example). For some subfields of particle physics where there is a detailed calculation or technique that is built upon, then so many eyes and brains will check it that peer-review is of limited use.
However, more and more, machine learning is dominating particle physics and typically private code is used on some simulation. Because machine learning is a black box, and there’s no logical way to check that the authors are reporting their results accurately, this becomes much more muddy. There’s no way to reproduce what someone did in a dominantly ML study (if no code is made public), so any required in-depth reading of the paper is valuable, that at the very least something non-trivial is contained in the paper.
LikeLike
Yeah, SCOAP3 is communal, and that is an improvement over how journals work in most fields, due to collective negotiation. (And open access, but that part is getting more common, it’s just in many other fields authors pay APCs.) Still, paying Springer whatever SISSA pays is probably not giving the SCOAP3 partners as much bang for their buck as they could get.
I don’t know how much reviewers can do to check things when the code is private and the methods are compute-intensive. Have you run into cases in jet physics where some paper would have gotten used (and screwed up some other physics goal) were it not for a reviewer digging up an issue with it?
LikeLike
Well, not that directly because if the code is private, there’s not much you can do as a referee. However, I have had many instances when a public code was used and the authors misinterpreted input or output, or didn’t represent their use well in the paper and that needed clarification at least, if not completely new simulation. Jet physics is a bit special as it straddles theory and experiment, so there are also many papers I have reviewed in which a theoretical result was misunderstood that needed to be addressed.
LikeLike
Some of us aren’t so obsessed with new results. I’m anthropologist. A new project is to assess the human impact of the spread of certain ideas via the web. For this project, it is important to know when those ideas appeared and became more popular. Further, since not everybody defines the key ideas in the same way, it is important to check out seemingly related ideas. I’m less than one full day into the literature review, and of about 20 “keeper” cites, 3 are to conference proceedings. I also found reference to another 5 conference papers that appeared relevant but for which no proceedings were available; have to send off for them.
LikeLike
To clarify, are you studying the spread of ideas among academics at conferences, and that’s why you’re looking at conference proceedings?
If so, I agree that proceedings create a useful record of what was discussed when for older work. For more modern conferences, at least in my field, slides and recordings of talks are posted online, so you’d have more direct access (albeit less searchable) through that.
LikeLike
No, I am studying the spread of ideas through the web and seeking to describe the impact of those ideas of web consumers. The computer scientists have the skills and resources to study huge swaths of the web, and their results are in the conference proceedings. It’s the human subjects research where we add value.
LikeLike
Yeah, as I mention in the post computer science is a very different field. They publish things first in conference proceedings before they publish on arXiv, whereas in my field that basically never happens these days. (The one case I can think of is someone who is planning an update to some software they’ve released…so a CS topic, in the end!)
LikeLike