Tag Archives: DoingScience

Toy Models

In academia, scientists don’t always work with what they actually care about. A lot of the time, they use what academics call toy models. A toy model can be a theory with simpler mathematics than the theories that describe the real world, but it can also be something that is itself real, just simpler or easier to work with, like nematodes, fruit flies, or college students.

Some people in industry seem to think this is all academics do. I’ve seen a few job ads that emphasize experience dealing with “real-world data”, and a few people skeptical that someone used to academia would be able to deal with the messy challenges of the business world.

There’s a grain of truth to this, but I don’t think industry has a monopoly on mess. To see why, let’s think about how academics write computer code.

There are a lot of things that one is in-principle supposed to do to code well, and most academics do none of them. Good code has test suites, so that if you change something you can check whether it still works by testing it on all the things that could go wrong. Good code is modular, with functions doing specific things and re-used whenever appropriate. Good code follows shared conventions, so that others can pick up your code and understand how you did it.

Some academics do these things, for example those who build numerical simulations on supercomputers. But for most academics, coding best-practices range from impractical to outright counterproductive. Testing is perhaps the clearest example. To design a test suite, you have to have some idea what kinds of things your code will run into, what kind of input you expect what the output is supposed to be. Many academic projects, though, are the first of their kind. Academics code up something to do a calculation nobody has done before, not knowing the result, or they make code to analyze a dataset nobody has worked with before. By the time they understand the problem well enough to write a test suite, they’ve already solved the problem, and they’re on to the next project, which may need something totally different.

From the perspective of these academics, if you have a problem well-defined enough that you can build a test suite, well enough that you can have stable conventions and reusable functions…then you have a toy model, not a real problem from the real world.

…and of course, that’s not quite fair either, right?

The truth is, academics and businesspeople want to work with toy models. Toy models are well-behaved, and easy, and you can do a lot with them. The real world isn’t a toy model…but it can be, if you make it one.

This means planning your experiments, whether in business or in science. It means making sure the data you gather is labeled and organized before you begin. It means coming up with processes, and procedures, and making as much of the work as possible a standardized, replicable thing. That’s desirable regardless, whether you’re making a consistent product instead of artisanal one-offs or a well-documented scientific study that another team can replicate.

Academia and industry both must handle mess. They handle different kinds of mess in different circumstances, and manage it in different ways, and this can be a real challenge for someone trying to go from one world to another. But neither world is intrinsically messier or cleaner. Nobody has a monopoly on toy models.

Does Science Require Publication?

Seen on Twitter:

As is traditional, twitter erupted into dumb arguments over this. Some made fun of Yann LeCun for implying that Elon Musk will be forgotten, which despite any other faults of his seems unlikely. Science popularizer Sabine Hossenfelder pointed out that there are two senses of “publish” getting confused here: publish as in “make public” and publish as in “put in a scientific journal”. The latter tends to be necessary for scientists in practice, but is not required in principle. (The way journals work has changed a lot over just the last century!) The former, Sabine argued, is still 100% necessary.

Plenty of people on twitter still disagreed (this always happens). It got me thinking a bit about the role of publication in science.

When we talk about what science requires or doesn’t require, what are we actually talking about?

“Science” is a word, and like any word its meaning is determined by how it is used. Scientists use the word “science” of course, as do schools and governments and journalists. But if we’re getting into arguments about what does or does not count as science, then we’re asking about a philosophical problem, one in which philosophers of science try to understand what counts as science and what doesn’t.

What do philosophers of science want? Many things, but a big one is to explain why science works so well. Over a few centuries, humanity went from understanding the world in terms of familiar materials and living creatures to decomposing them in terms of molecules and atoms and cells and proteins. In doing this, we radically changed what we were capable of, computers out of the reach of blacksmiths and cures for diseases that weren’t even distinguishable. And while other human endeavors have seen some progress over this time (democracy, human rights…), science’s accomplishment demands an explanation.

Part of that explanation, I think, has to include making results public. Alchemists were interested in many of the things later chemists were, and had started to get some valuable insights. But alchemists were fearful of what their knowledge would bring (especially the ones who actually thought they could turn lead into gold). They published almost only in code. As such, the pieces of progress they made didn’t build up, didn’t aggregate, didn’t become overall progress. It was only when a new scientific culture emerged, when natural philosophers and physicists and chemists started writing to each other as clearly as they could, that knowledge began to build on itself.

Some on twitter pointed out the example of the Manhattan project during World War II. A group of scientists got together and made progress on something almost entirely in secret. Does that not count as science?

I’m willing to bite this bullet: I don’t think it does! When the Soviets tried to replicate the bomb, they mostly had to start from scratch, aside from some smuggled atomic secrets. Today, nations trying to build their own bombs know more, but they still must reinvent most of it. We may think this is a good thing, we may not want more countries to make progress in this way. But I don’t think we can deny that it genuinely does slow progress!

At the same time, to contradict myself a bit: I think you can think of science that happens within a particular community. The scientists of the Manhattan project didn’t publish in journals the Soviets could read. But they did write internal reports, they did publish to each other. I don’t think science by its nature has to include the whole of humanity (if it does, then perhaps studying the inside of black holes really is unscientific). You probably can do science sticking to just your own little world. But it will be slower. Better, for progress’s sake, if you can include people from across the world.

No Unmoved Movers

Economists must find academics confusing.

When investors put money in a company, they have some control over what that company does. They vote to decide a board, and the board votes to hire a CEO. If the company isn’t doing what the investors want, the board can fire the CEO, or the investors can vote in a new board. Everybody is incentivized to do what the people who gave the money want to happen. And usually, those people want the company to increase its profits, since most of them people are companies with their own investors).

Academics are paid by universities and research centers, funded in the aggregate by governments and student tuition and endowments from donors. But individually, they’re also often funded by grants.

What grant-givers want is more ambiguous. The money comes in big lumps from governments and private foundations, which generally want something vague like “scientific progress”. The actual decision of who gets the money are made by committees made up of senior scientists. These people aren’t experts in every topic, so they have to extrapolate, much as investors have to guess whether a new company will be profitable based on past experience. At their best, they use their deep familiarity with scientific research to judge which projects are most likely to work, and which have the most interesting payoffs. At their weakest, though, they stick with ideas they’ve heard of, things they know work because they’ve seen them work before. That, in a nutshell, is why mainstream research prevails: not because the mainstream wants to suppress alternatives, but because sometimes the only way to guess if something will work is raw familiarity.

(What “works” means is another question. The cynical answers are “publishes papers” or “gets citations”, but that’s a bit unfair: in Europe and the US, most funders know that these numbers don’t tell the whole story. The trivial answer is “achieves what you said it would”, but that can’t be the whole story, because some goals are more pointless than others. You might want the answer to be “benefits humanity”, but that’s almost impossible to judge. So in the end the answer is “sounds like good science”, which is vulnerable to all the fads you can imagine…but is pretty much our only option, regardless.)

So are academics incentivized to do what the grant committees want? Sort of.

Science never goes according to plan. Grant committees are made up of scientists, so they know that. So while many grants have a review process afterwards to see whether you achieved what you planned, they aren’t all that picky about it. If you can tell a good story, you can explain why you moved away from your original proposal. You can say the original idea inspired a new direction, or that it became clear that a new approach was necessary. I’ve done this with an EU grant, and they were fine with it.

Looking at this, you might imagine that an academic who’s a half-capable storyteller could get away with anything they wanted. Propose a fashionable project, work on what you actually care about, and tell a good story afterwards to avoid getting in trouble. As long as you’re not literally embezzling the money (the guy who was paying himself rent out of his visitor funding, for instance), what could go wrong? You get the money without the incentives, you move the scientific world and nobody gets to move you.

It’s not quite that easy, though.

Sabine Hossenfelder told herself she could do something like this. She got grants for fashionable topics she thought were pointless, and told herself she’d spend time on the side on the things she felt were actually important. Eventually, she realized she wasn’t actually doing the important things: the faddish research ended up taking all her time. Not able to get grants doing what she actually cared about (and, in one of those weird temporary European positions that only lasts until you run out of grants), she now has to make a living from her science popularization work.

I can’t speak for Hossenfelder, but I’ve also put some thought into how to choose what to research, about whether I could actually be an unmoved mover. A few things get in the way:

First, applying for grants doesn’t just take storytelling skills, it takes scientific knowledge. Grant committees aren’t experts in everything, but they usually send grants to be reviewed by much more appropriate experts. These experts will check if your grant makes sense. In order to make the grant make sense, you have to know enough about the faddish topic to propose something reasonable. You have to keep up with the fad. You have to spend time reading papers, and talking to people in the faddish subfield. This takes work, but also changes your motivation. If you spend time around people excited by an idea, you’ll either get excited too, or be too drained by the dissonance to get any work done.

Second, you can’t change things that much. You still need a plausible story as to how you got from where you are to where you are going.

Third, you need to be a plausible person to do the work. If the committee looks at your CV and sees that you’ve never actually worked on the faddish topic, they’re more likely to give a grant to someone who’s actually worked on it.

Fourth, you have to choose what to do when you hire people. If you never hire any postdocs or students working on the faddish topic, then it will be very obvious that you aren’t trying to research it. If you do hire them, then you’ll be surrounded by people who actually care about the fad, and want your help to understand how to work with it.

Ultimately, to avoid the grant committee’s incentives, you need a golden tongue and a heart of stone, and even then you’ll need to spend some time working on something you think is pointless.

Even if you don’t apply for grants, even if you have a real permanent position or even tenure, you still feel some of these pressures. You’re still surrounded by people who care about particular things, by students and postdocs who need grants and jobs and fellow professors who are confident the mainstream is the right path forward. It takes a lot of strength, and sometimes cruelty, to avoid bowing to that.

So despite the ambiguous rules and lack of oversight, academics still respond to incentives: they can’t just do whatever they feel like. They aren’t bound by shareholders, they aren’t expected to make a profit. But ultimately, the things that do constrain them, expertise and cognitive load, social pressure and compassion for those they mentor, those can be even stronger.

I suspect that those pressures dominate the private sector as well. My guess is that for all that companies think of themselves as trying to maximize profits, the all-too-human motivations we share are more powerful than any corporate governance structure or org chart. But I don’t know yet. Likely, I’ll find out soon.

Making More Nails

They say when all you have is a hammer, everything looks like a nail.

Academics are a bit smarter than that. Confidently predict a world of nails, and you fall to the first paper that shows evidence of a screw. There are limits to how long you can delude yourself when your job is supposed to be all about finding the truth.

You can make your own nails, though.

Suppose there’s something you’re really good at. Maybe, like many of my past colleagues, you can do particle physics calculations faster than anyone else, even when the particles are super-complicated hypothetical gravitons. Maybe you know more than anyone else about how to make a quantum computer, or maybe you just know how to build a “quantum computer“. Maybe you’re an expert in esoteric mathematics, who can re-phrase anything in terms of the arcane language of category theory.

That’s your hammer. Get good enough with it, and anyone with a nail-based problem will come to you to solve it. If nails are trendy, then you’ll impress grant committees and hiring committees, and your students will too.

When nails aren’t trendy, though, you need to try something else. If your job is secure, and you don’t have students with their own insecure jobs banging down your door, then you could spend a while retraining. You could form a reading group, pick up a textbook or two about screwdrivers and wrenches, and learn how to use different tools. Eventually, you might find a screwdriving task you have an advantage with, something you can once again do better than everyone else, and you’ll start getting all those rewards again.

Or, maybe you won’t. You’ll get less funding to hire people, so you’ll do less research, so your work will get less impressive and you’ll get less funding, and so on and so forth.

Instead of risking that, most academics take another path. They take what they’re good at, and invent new problems in the new trendy area to use that expertise.

If everyone is excited about gravitational waves, you turn a black hole calculation into a graviton calculation. If companies are investing in computation in the here-and-now, then you find ways those companies can use insights from your quantum research. If everyone wants to know how AI works, you build a mathematical picture that sort of looks like one part of how AI works, and do category theory to it.

At first, you won’t be competitive. Your hammer isn’t going to work nearly as well as the screwdrivers people have been using forever for these problems, and there will be all sorts of new issues you have to solve just to get your hammer in position in the first place. But that doesn’t matter so much, as long as you’re honest. Academic research is expected to take time, applications aren’t supposed to be obvious. Grant committees care about what you’re trying to do, as long as you have a reasonably plausible story about how you’ll get there.

(Investors are also not immune to a nice story. Customers are also not immune to a nice story. You can take this farther than you might think.)

So, unlike the re-trainers, you survive. And some of the time, you make it work. Your hammer-based screwdriving ends up morphing into something that, some of the time, actually does something the screwdrivers can’t. Instead of delusionally imagining nails, you’ve added a real ersatz nail to the world, where previously there was just a screw.

Making nails is a better path for you. Is it a better path for the world? I’m not sure.

If all those grants you won, all those jobs you and your students got, all that money from investors or customers drawn in by a good story, if that all went to the people who had the screwdrivers in the first place, could they have done a better job?

Sometimes, no. Sometimes you happen upon some real irreproducible magic. Your hammer is Thor’s hammer, and when hefted by the worthy it can do great things.

Sometimes, though, your hammer was just the hammer that got the funding. Now every screwdriver kit has to have a space for a little hammer, when it could have had another specialized screwdriver that fit better in the box.

In the end, the world is build out of these kinds of ill-fitting toolkits. We all try to survive, both as human beings and by our sub-culture’s concept of the good life. We each have our hammers, and regardless of whether the world is full of screws, we have to convince people they want a hammer anyway. Everything we do is built on a vast rickety pile of consequences, the end-results of billions of people desperate to be wanted. For those of us who love clean solutions and ideal paths, this is maddening and frustrating and terrifying. But it’s life, and in a world where we never know the ideal path, screw-nails and nail-screws are the best way we’ve found to get things done.

An “Open-Source” Grant Proposal

Back in the Fall, I spent most of my time writing a grant proposal.

In Europe, getting a European Research Council (ERC) grant is how you know you’ve made it as a researcher. Covering both science and the humanities, ERC grants give a lump of funding big enough to hire a research group, turning you from a lone expert into a local big-shot. The grants last five years, and are organized by “academic age”, the number of years since your PhD. ERC Starting Grants give 1.5 million euros for those with academic age 2-7. At academic age 7-12, you need to apply for the Consolidator Grant. The competition is fiercer, but if you make it through you get 2 million euros. Finally, Advanced Grants give 2.5 million to more advanced researchers.

I’m old, at least in terms of academic age. I applied to the ERC Starting Grant in 2021, but this last year I was too academically old to qualify, so I applied to the Consolidator Grant instead.

I won’t know if they invite me for an interview until June…but since I’m leaving the field, there wouldn’t be much point in going anyway. So I figured, why not share the grant application with you guys?

That’s what I’m doing in this post. I think there are good ideas in here, a few research directions that fellow amplitudeologists might want to consider. (I’ve removed details on one of them, the second work package, because some friends of mine are already working on it.)

The format could also be helpful. My wife is more than a bit of a LaTeX wiz, she coded up Gantt charts and helped with the format of the headers and the color scheme. If you want an ERC proposal that doesn’t look like the default thing you could do with LaTeX or Word, then take a look.

Finally, I suspect some laymen in the audience are just curious what a scientific grant proposal looks like. While I’ve cut a few things (and a few of these were shorter than they ought to have been to begin with), this might satisfy your curiosity.

You can find the proposal in a zip file here: https://drive.proton.me/urls/WTVN0F16HG#mYaz0edaOGha . I’ve included pdfs of the two required parts, B1 and B2, as well as the LaTeX files used to generate them.

For those of you still in the game, good luck with your ERCs!


Update from November 2024:

I wanted to include a bit more information for those who want to build off some of the ideas in the proposal.

I did end up getting offered an interview for this grant, and since the ERC doesn’t give any way to withdraw in their system I ended up going through with the interview. I didn’t get the grant, but I think it would have had a solid chance if I had had the time to focus and prepare for it (rather than mostly being busy applying for industry jobs). If anyone wants to write their own proposal building on some of the research directions I’m proposing here, I’m happy to chat and give you advice. In particular, a few things to keep in mind:

  • You need a good list of pheno applications. In particular, unless you focus your proposal heavily on the gravitational wave side, you need a good list of particle physics applications, because the particle physicists generally won’t think that the gravitational wave side “counts”. I was asked in the interview to name three particle physics measurements this would help with, I had mentioned two in the proposal and could only come up with one off the top of my head. You can do a lot better with preparation.
  • Relatedly, you need some idea of what the pipeline looks like, what these calculations eventually get used for, including the looming question of “why do this analytically rather than numerically?”
  • If you’re including the N=4 super Yang-Mills side of the story, you’ll have to overcome some skepticism. Some of that skepticism can be brushed aside by emphasizing the theory’s track record (canonical differential equations probably wouldn’t exist without research in N=4 symbols), but a meaningful source of skepticism is just whether you can work with dim reg. This is an issue currently facing a few other approaches, so it’s good to have a good answer for it!
  • If you’re relying a lot on the expertise of the people you plan on hiring (I definitely was, especially in planning to hire a mathematician) then ideally you should have some idea of who you could hire. I wasn’t in a position to do this for obvious reasons, but anyone that has a stable position should consider talking to potential hires in advance so you have a list of names.
  • Have justifications in mind for your budget. Yes, you’ll be encouraged by your home institution to just increase every budget line as far as you can get. But you will be asked about anything unusually high, so you really need a picture for what you will spend it on. Along these lines, if your institution imposes any unusual expenses (since my budget was written for the CEA, it had to pay for Mathematica and Maple licenses since the CEA is technically a private business and doesn’t have access to site licenses at academic rates) then you need to be able to justify why it’s still a good host despite that.

What’s in a Subfield?

A while back, someone asked me what my subfield, amplitudeology, is really about. I wrote an answer to that here, a short-term and long-term perspective that line up with the stories we often tell about the field. I talked about how we try to figure out ways to calculate probabilities faster, first for understanding the output of particle colliders like the LHC, then more recently for gravitational wave telescopes. I talked about how the philosophy we use for that carries us farther, how focusing on the minimal information we need to make a prediction gives us hope that we can generalize and even propose totally new theories.

The world doesn’t follow stories, though, not quite so neatly. Try to define something as simple as the word “game” and you run into trouble. Some games have a winner and a loser, some games everyone is on one team, and some games don’t have winners or losers at all. Games can involve physical exercise, computers, boards and dice, or just people telling stories. They can be played for fun, or for money, silly or deadly serious. Most have rules, but some don’t even have that. Instead, games are linked by history: a series of resemblances, people saying that “this” is a game because it’s kind of like “that”.

A subfield isn’t just a word, it’s a group of people. So subfields aren’t defined just by resemblance. Instead, they’re defined by practicality.

To ask what amplitudeology is really about, think about why you might want to call yourself an amplitudeologist. It could be a question of goals, certainly: you might care a lot about making better predictions for the LHC, or you could have some other grand story in mind about how amplitudes will save the world. Instead, though, it could be a matter of training: you learned certain methods, certain mathematics, a certain perspective, and now you apply it to your research, even if it goes further afield from what was considered “amplitudeology” before. It could even be a matter of community, joining with others who you think do cool stuff, even if you don’t share exactly the same goals or the same methods.

Calling yourself an amplitudeologist means you go to their conferences and listen to their talks, means you look to them to collaborate and pay attention to their papers. Those kinds of things define a subfield: not some grand mission statement, but practical questions of interest, what people work on and know and where they’re going with that. Instead of one story, like every other word, amplitudeology has a practical meaning that shifts and changes with time. That’s the way subfields should be: useful to the people who practice them.

What Referees Are For

This week, we had a colloquium talk by the managing editor of the Open Journal of Astrophysics.

The Open Journal of Astrophysics is an example of an arXiv overlay journal. In the old days, journals shouldered the difficult task of compiling scientists’ work into a readable format and sending them to university libraries all over the world so people could stay up to date with the work of distant colleagues. They used to charge libraries for the journals, now some instead charge authors per paper they want to publish.

Now, most of that is unnecessary due to online resources, in my field the arXiv. We prepare our papers using free tools like LaTeX, then upload them to arXiv.org, a website that makes the papers freely accessible for everybody. I don’t think I’ve ever read a paper in a physical journal in my field, and I only check journal websites if I think there’s a mistake in the arXiv version. The rest of the time, I just use the arXiv.

Still, journals do one thing the arXiv doesn’t do, and that’s refereeing. Each paper a journal receives is sent out to a few expert referees. The referees read the paper, and either reject it, accept it as-is, or demand changes before they can accept it. The journal then publishes accepted papers only.

The goal of arXiv overlay journals is to make this feature of journals also unnecessary. To do this, they notice that if every paper is already on arXiv, they don’t need to host papers or print them or typeset them. They just need to find suitable referees, and announce which papers passed.

The Open Journal of Astrophysics is a relatively small arXiv overlay journal. They operate quite cheaply, in part because the people running it can handle most of it as a minor distraction from their day job. SciPost is much bigger, and has to spend more per paper to operate. Still, it spends a lot less than journals charge authors.

We had a spirited discussion after the talk, and someone brought up an interesting point: why do we need to announce which papers passed? Can’t we just publish everything?

What, in the end, are the referees actually for? Why do we need them?

One function of referees is to check for mistakes. This is most important in mathematics, where referees might spend years making sure every step in a proof works as intended. Other fields vary, from theoretical physics (where we can check some things sometimes, but often have to make do with spotting poorly explained parts of a calculation), to fields that do experiments in the real world (where referees can look for warning signs and shady statistics, but won’t actually reproduce the experiment). A mistake found by a referee can be a boon to not just the wider scientific community, but to the author as well. Most scientists would prefer their papers to be correct, so we’re often happy to hear about a genuine mistake.

If this was all referees were for, though, then you don’t actually need to reject any papers. As a colleague of mine suggested, you just need the referees to publish their reports. Then the papers could be published along with comments from the referees, and possibly also responses from the author. Readers could see any mistakes the referees found, and judge for themselves what they show about the result.

Referees already publish their reports in SciPost much of the time, though not currently in the Open Journal of Astrophysics. Both journals still reject some papers, though. In part, that’s because they serve another function: referees are supposed to tell us which papers are “good”.

Some journals are more prestigious and fancy than others. Nature and Science are the most famous, though people in my field almost never bother to publish in either. Still, we have a hierarchy in mind, with Physical Review Letters on the high end and JHEP on the lower one. Publishing in a fancier and more prestigious journal is supposed to say something about you as a scientist, to say that your work is fancier and more prestigious. If you can’t publish in any journal at all, then your work wasn’t interesting enough to merit getting credit for it, and maybe you should have worked harder.

What does that credit buy you? Ostensibly, everything. Jobs are more likely to hire you if you’ve published in more prestigious places, and grant agencies will be more likely to give you money.

In practice, though, this depends a lot on who’s making the decisions. Some people will weigh these kinds of things highly, especially if they aren’t familiar with a candidate’s work. Others will be able to rely on other things, from numbers of papers and citations to informal assessments of a scientist’s impact. I genuinely don’t know whether the journals I published in made any impact at all when I was hired, and I’m a bit afraid to ask. I haven’t yet sat on the kind of committee that makes these decisions, so I don’t know what things look like from the other side either.

But I do know that, on a certain level, journals and publications can’t matter quite as much as we think. As I mentioned, my field doesn’t use Nature or Science, while others do. A grant agency or hiring committee comparing two scientists would have to take that into account, just as they have to take into account the thousands of authors on every single paper by the ATLAS and CMS experiments. If a field started publishing every paper regardless of quality, they’d have to adapt there too, and find a new way to judge people compatible with that.

Can we just publish everything, papers and referee letters and responses and letters and reviews? Maybe. I think there are fields where this could really work well, and fields where it would collapse into the invective of a YouTube comments section. I’m not sure where my own field sits. Theoretical particle physics is relatively small and close-knit, but it’s also cool and popular, with many strong and dumb opinions floating around. I’d like to believe we could handle it, that we could prune back the professional cruft and turn our field into a real conversation between scholars. But I don’t know.

A Significant Calculation

Particle physicists have a weird relationship to journals. We publish all our results for free on a website called the arXiv, and when we need to read a paper that’s the first place we look. But we still submit our work to journals, because we need some way to vouch that we’re doing good work. Explicit numbers (h-index, impact factor) are falling out of favor, but we still need to demonstrate that we get published in good journals, that we do enough work, and that work has an impact on others. We need it to get jobs, to get grants to fund research at those jobs, and to get future jobs for the students and postdocs we hire with those grants. Our employers need it to justify their own funding, to summarize their progress so governments and administrators can decide who gets what.

This can create weird tensions. When people love a topic, they want to talk about it with each other. They want to say all sorts of things, big and small, to contribute new ideas and correct others and move things forward. But as professional physicists, we also have to publish papers. We can publish some “notes”, little statements on the arXiv that we don’t plan to make into a paper, but we don’t really get “credit” for those. So in practice, we try to force anything we want to say into a paper-sized chunk.

That wouldn’t be a problem if paper-sized chunks were flexible, and you can see when journals historically tried to make them that way. Some journals publish “letters”, short pieces a few pages long, to contrast with their usual papers that can run from twenty to a few hundred pages. These “letters” tend to be viewed as prestigious, though, so they end up being judged on roughly the same standards as the normal papers, if not more strictly.

What standards are those? For each journal, you can find an official list. The Journal of High-Energy Physics, for example, instructs reviewers to look for “high scientific
quality, originality and relevance”. That rules out papers that just reproduce old results, but otherwise is frustratingly vague. What constitutes high scientific quality? Relevant to whom?

In practice, reviewers use a much fuzzier criterion: is this “paper-like”? Does this look like other things that get published, or not?

Each field will assess that differently. It’s a criterion of familiarity, of whether a paper looks like what people in the field generally publish. In my field, one rule of thumb is that a paper must contain a significant calculation.

A “significant calculation” is still quite fuzzy, but the idea is to make sure that a paper requires some amount of actual work. Someone has to do something challenging, and the work shouldn’t be half-done: as much as feasible, they should finish, and calculate something new. Ideally, this should be something that nobody had calculated before, but if the perspective is new enough it can be something old. It should “look hard”, though.

That’s a fine way to judge whether someone is working hard, which is something we sometimes want to judge. But since we’re incentivized to make everything into a paper, this means that every time we want to say something, we want to accompany it with some “significant calculation”, some concrete time-consuming work. This can happen even if we want to say something that’s quite direct and simple, a fact that can be quickly justified but nonetheless has been ignored by the field. If we don’t want it to be “just” an un-credited note, we have to find some way to turn it into a “significant calculation”. We do extra work, sometimes pointless work, in order to make something “paper-sized”.

I like to think about what academia would be like without the need to fill out a career. The model I keep imagining is that of a web forum or a blogging platform. There would be the big projects, the in-depth guides and effortposts. But there would also be shorter contributions, people building off each other, comments on longer pieces and quick alerts pinned to the top of the page. We’d have a shared record of knowledge, where everyone contributes what they want to whatever level of detail they want.

I think math is a bit closer to this ideal. Despite their higher standards for review, checking the logic of every paper to make sure it makes sense to publish, math papers can sometimes be very short, or on apparently trivial things. Physics doesn’t quite work this way, and I suspect part of it is our funding sources. If you’re mostly paid to teach, like many mathematicians, your research is more flexible. If you’re paid to research, like many physicists, then people want to make sure your research is productive, and that tends to cram it into measurable boxes.

In today’s world, I don’t think physics can shift cultures that drastically. Even as we build new structures to rival the journals, the career incentives remain. Physics couldn’t become math unless it shed most of the world’s physicists.

In the long run, though…well, we may one day find ourselves in a world where we don’t have to work all our days to keep each other alive. And if we do, hopefully we’ll change how scientists publish.

IPhT’s 60-Year Anniversary

This year is the 60th anniversary of my new employer, the Institut de Physique Théorique of CEA Paris-Saclay (or IPhT for short). In celebration, they’re holding a short conference, with a variety of festivities. They’ve been rushing to complete a film about the institute, and I hear there’s even a vintage arcade game decorated with Feynman diagrams. For me, it will be a chance to learn a bit more about the history of this place, which I currently know shamefully little about.

(For example, despite having his textbook on my shelf, I don’t know much about what our Auditorium’s namesake Claude Itzykson was known for.)

Since I’m busy with the conference this week, I won’t have time for a long blog post. Next week I’ll be able to say more, and tell you what I learned!

Physics’ Unique Nightmare

Halloween is coming up, so let’s talk about the most prominent monster of the physics canon, the nightmare scenario.

Not to be confused with the D&D Nightmare, which once was a convenient source of infinite consumable items for mid-level characters.

Right now, thousands of physicists search for more information about particle physics beyond our current Standard Model. They look at data from the Large Hadron Collider to look for signs of new particles and unexpected behavior, they try to detect a wide range of possible dark matter particles, and they make very precise measurements to try to detect subtle deviations. And in the back of their minds, almost all of those physicists wonder if they’ll find anything at all.

It’s not that we think the Standard Model is right. We know it has problems, deep mathematical issues that make it give nonsense answers and an apparent big mismatch with what we observe about the motion of matter and light in the universe. (You’ve probably heard this mismatch called dark matter and dark energy.)

But none of those problems guarantee an answer soon. The Standard Model will eventually fail, but it may fail only for very difficult and expensive experiments, not a Large Hadron Collider but some sort of galactic-scale Large Earth Collider. It might be that none of the experiments or searches or theories those thousands of physicists are working on will tell them anything they didn’t already know. That’s the nightmare scenario.

I don’t know another field that has a nightmare scenario quite like this. In most fields, one experiment or another might fail, not just not giving the expected evidence but not teaching anything new. But most experiments teach us something new. We don’t have a theory, in almost any field, that has the potential to explain every observation up to the limits of our experiments, but which we still hope to disprove. Only the Standard Model is like that.

And while thousands of physicists are exposed to this nightmare scenario, the majority of physicists aren’t. Physics isn’t just the science of the reductionistic laws of the smallest constituents of matter. It’s also the study of physical systems, from the bubbling chaos of nuclear physics to the formation of planets and galaxies and black holes, to the properties of materials to the movement of bacteria on a petri dish and bees in a hive. It’s also the development of new methods, from better control of individual atoms and quantum states to powerful new tricks for calculation. For some, it can be the discovery, not of reductionistic laws of the smallest scales, but of general laws of the largest scales, of how systems with many different origins can show echoes of the same behavior.

Over time, more and more of those thousands of physicists break away from the nightmare scenario, “waking up” to new questions of these kinds. For some, motivated by puzzles and skill and the beauty of physics, the change is satisfying, a chance to work on ideas that are moving forward, connected with experiment or grounded in evolving mathematics. But if your motivation is really tied to those smallest scales, to that final reductionistic “why”, then such a shift won’t be satisfying, and this is a nightmare you won’t wake up from.

Me, I’m not sure. I’m a tool-builder, and I used to tell myself that tool-builders are always needed. But I find I do care, in the end, what my tools are used for. And as we approach the nightmare scenario, I’m not at all sure I know how to wake up.