Sabine Hossenfelder has a blog post this week chastising particle physicists and cosmologists for following “upside-down Popper”, or assuming a theory is worth working on merely because it’s falsifiable. She describes her colleagues churning out one hypothesis after another, each tweaking an old idea just enough to make it falsifiable in the next experiment, without caring whether the hypothesis is actually likely to be true.
Sabine is much more of an expert in this area of physics (phenomenology) than I am, and I don’t presume to tell her she’s wrong about that community. But the problem she’s describing is part of something bigger, something that affects my part of physics as well.
There’s a core question we’d all like to answer: what should physicists work on? What criteria should guide us?
Falsifiability isn’t the whole story. The next obvious criterion is a sense of simplicity, of Occam’s Razor or mathematical elegance. Sabine has argued against the latter, which prompted a friend of mine to comment that between rejecting falsifiability and elegance, Sabine must want us to stop doing high-energy physics at all!
That’s more than a little unfair, though. I think Sabine has a reasonably clear criterion in mind. It’s the same criterion that most critics of the physics mainstream care about. It’s even the same criterion being used by the “other side”, the sort of people who criticize anything that’s not string/SUSY/inflation.
The criterion is quite a simple one: physics research should be productive. Anything we publish, anything we work on, should bring us closer to understanding the real world.
And before you object that this criterion is obvious, that it’s subjective, that it ignores the very real disagreements between the Sabines and the Luboses of the world…before any of that, please let me finish.
We can’t achieve this criterion. And we shouldn’t.
We can’t demand that all physics be productive without breaking a fundamental bargain, one we made when we accepted that science could be a career.
The Hunchback of Notre Science
It wasn’t always this way. Up until the nineteenth century, “scientist” was a hobby, not a job.
After Newton published his theory of gravity, he was famously accused by Robert Hooke of stealing the idea. There’s some controversy about this, but historians agree on a few points: that Hooke did write a letter to Newton suggesting a 
Why not? In part, because Hooke, unlike Newton, had a job.
Hooke was arguably the first person for whom science was a full-time source of income. As curator of experiments for the Royal Society, it was his responsibility to set up demonstrations for each Royal Society meeting. Later, he also handled correspondence for the Royal Society Journal. These responsibilities took up much of his time, and as a result, even if he was capable of following up on the consequences of
We’re better off than Hooke today. We still have our responsibilities, to journals and teaching and the like, at various stages of our careers. But in the centuries since Hooke expectations have changed, and real original research is no longer something we have to fit in our spare time. It’s now a central expectation of the job.
When scientific research became a career, we accepted a kind of bargain. On the positive side, you no longer have to be independently wealthy to contribute to science. More than that, the existence of professional scientists is the bedrock of technological civilization. With enough scientists around, we get modern medicine and the internet and space programs and the LHC, things that wouldn’t be possible in a world of rare wealthy geniuses.
We pay a price for that bargain, though. If science is a steady job, then it has to provide steady work. A scientist has to be able to go in, every day, and do science.
And the problem is, science doesn’t always work like that. There isn’t always something productive to work on. Even when there is, there isn’t always something productive for you to work on.
Sabine blames “upside-down Popper” on the current publish-or-perish environment in physics. If physics careers weren’t so cut-throat and the metrics they are judged by weren’t so flawed, then maybe people would have time to do slow, careful work on deeper topics rather than pumping out minimally falsifiable papers as fast as possible.
There’s a lot of truth to this, but I think at its core it’s a bit too optimistic. Each of us only has a certain amount of expertise, and sometimes that expertise just isn’t likely to be productive at the moment. Because science is a job, a person in that position can’t just go work at the Royal Mint like Newton did. (The modern-day equivalent would be working for Wall Street, but physicists rarely come back from that.) Instead, they keep doing what they know how to do, slowly branching out, until they’ve either learned something productive or their old topic becomes useful once more. You can think of it as a form of practice, where scientists keep their skills honed until they’re needed.
So if we slow down the rate of publication, if we create metrics for universities that let them hire based on the depth and importance of work and not just number of papers and citations, if we manage all of that then yes we will improve science a great deal. But Lisa Randall still won’t work on Haag’s theorem.
In the end, we’ll still have physicists working on topics that aren’t actually productive.
A physicist lazing about unproductively under an apple tree
So do we have to pay physicists to work on whatever they want, no matter how ridiculous?
No, I’m not saying that. We can’t expect everyone to do productive work all the time, but we can absolutely establish standards to make the work more likely to be productive.
Strange as it may sound, I think our standards for this are already quite good, or at least better than many other fields.
First, there’s falsifiability itself, or specifically our attitude towards it.
Physics’s obsession with falsifiability has one important benefit: it means that when someone proposes a new model of dark matter or inflation that they tweaked to be just beyond the current experiments, they don’t claim to know it’s true. They just claim it hasn’t been falsified yet.
This is quite different from what happens in biology and the social sciences. There, if someone tweaks their study to be just within statistical significance, people typically assume the study demonstrated something real. Doctors base treatments on it, and politicians base policy on it. Upside-down Popper has its flaws, but at least it’s never going to kill anybody, or put anyone in prison.
Admittedly, that’s a pretty low bar. Let’s try to set a higher one.
Moving past falsifiability, what about originality? We have very strong norms against publishing work that someone else has already done.
Ok, you (and probably Sabine) would object, isn’t that easy to get around? Aren’t all these Popper-flippers pretending to be original but really just following the same recipe each time, modifying their theory just enough to stay falsifiable?
To some extent. But if they were really following a recipe, you could beat them easily: just write the recipe down.
Physics progresses best when we can generalize, when we skip from case-by-case to understanding whole swaths of cases at once. Over time, there have been plenty of cases in which people have done that, where a number of fiddly hand-made models have been summarized in one parameter space. Once that happens, the rule of originality kicks in: now, no-one can propose another fiddly model like that again. It’s already covered.
As long as the recipe really is just a recipe, you can do this. You can write up what these people are doing in computer code, release the code, and then that’s that, they have to do something else. The problem is, most of the time it’s not really a recipe. It’s close enough to one that they can rely on it, close enough to one that they can get paper after paper when they need to…but it still requires just enough human involvement, just enough genuine originality, to be worth a paper.
The good news is that the range of “recipes” we can code up increases with time. Some spaces of theories we might never be able to describe in full generality (I’m glad there are people trying to do statistics on the string landscape, but good grief it looks quixotic). Some of the time though, we have a real chance of putting a neat little bow on a subject, labeled “no need to talk about this again”.
This emphasis on originality keeps the field moving. It means that despite our bargain, despite having to tolerate “practice” work as part of full-time physics jobs, we can still nudge people back towards productivity.
One final point: it’s possible you’re completely ok with the idea of physicists spending most of their time “practicing”, but just wish they wouldn’t make such a big deal about it. Maybe you can appreciate that “can I cook up a model where dark matter kills the dinosaurs” is an interesting intellectual exercise, but you don’t think it should be paraded in front of journalists as if it were actually solving a real problem.
In that case, I agree with you, at least up to a point. It is absolutely true that physics has a dysfunctional relationship with the media. We’re too used to describing whatever we’re working on as the most important thing in the universe, and journalists are convinced that’s the only way to get the public to pay attention. This is something we can and should make progress on. An increasing number of journalists are breaking from the trend and focusing not on covering the “next big thing”, but in telling stories about people. We should do all we can to promote those journalists, to spread their work over the hype, to encourage the kind of stories that treat “practice” as interesting puzzles pursued by interesting people, not the solution to the great mysteries of physics. I know that if I ever do anything newsworthy, there are some journalists I’d give the story to before any others.
At the same time, it’s important to understand that some of the dysfunction here isn’t unique to physics, or even to science. Deep down the reason nobody can admit that their physics is “practice” work is the same reason people at job interviews claim to love the company, the same reason college applicants have to tell stirring stories of hardship and couples spend tens of thousands on weddings. We live in a culture in which nothing can ever just be “ok”, in which admitting things are anything other than exceptional is akin to calling them worthless. It’s an arms-race of exaggeration, and it goes far beyond physics.
(I should note that this “culture” may not be as universal as I think it is. If so, it’s possible its presence in physics is due to you guys letting too many of us Americans into the field.)
We made a bargain when we turned science into a career. We bought modernity, but the price we pay is subsidizing some amount of unproductive “practice” work. We can negotiate the terms of our bargain, and we should, tilting the field with incentives to get it closer to the truth. But we’ll never get rid of it entirely, because science is still done by people. And sometimes, despite what we’re willing to admit, people are just “ok”.
4 gravitons 
The system is made for the constant influx of new experimental data for which there is no good current theory. In the particle physics this sitution persisted until the 70s. The actual problem is that since then there is very little new physics discovered. That’s why all this phenomenological stuff remains to be useless. Not because this activity by itself is ridiculous but because last generations of physicists are that unlucky.
But for the “Sabines” that doesn’t matter, it’s just another trigger to complain that some popular theoretical ideas (which they practically always know mostly on the popular level and don’t really understand why someone considers them good) are leading the science in the wrong way, as if even good new theoretical ideas would move us anywhere from the current spot.
At least Sabine herself participated in the so-called “quantum gravity phenomenology” activity that could be seen as an attempt to resolve this conundrum with some novel experimental techniques… though it was based on even wilder speculations than she loves to criticise. But frankly I consider her too not a physicist at all but a professional complainer. There are many reasons for that but the final straw was her article in Forbes where she said that it is good thing if the hint of the new physics on LHC will go away… because it will be bad for naturalness idea.
LikeLiked by 1 person
State of the art high energy theoretical Physics a.k.a String theory is guided by three things:
1) Theoretical consistency and backward low energy compatibility
2) Mathematical consistency
3) high explanatory power
In lack of new experimental data, due to the high energy realm in which the theory is manifesting itself, these should be the guiding principles of every theory; this is science at its best.
LikeLike
Great political speech, Giotis. The only problem with string theory is that 3 is not yet achieved as we are not able to pinpoint sufficiently narrow compactification class. Until then we have only a framework for model-building with good UV completion. Don’t get me wrong, that’s huge but still at this specific point you can’t turn stringy phenomenology into something really useful.
The true success of the string theory is in the failure of alternatives. A lot of people don’t really understand that if the UV-complete quantum gravity was easy to construct and there were lots of different models… the quantum gravity research would be a complete waste of time at this point! “The only game in town” justifies all the fun. That’s why it’s particularly important to learn whether this is indeed the only possibility under sensible assumptions… and whether those assumptions can be deformed in a sensible way.
LikeLike
It is certainly not the only possibility. To make it the only possibility, you have to introduce a lot of “sensible assumptions”, another word for private metaphysical preferences. Without such metaphysical choices, one can use almost every regularization as a theory. (Ok, one would have to care about fermion doubling, big deal.)
LikeLike
Well yes I’m talking about guiding principles when there is a lack of experimental data to guide you.
Which reminds me a fourth one that I missed:
4) Theoretical and Mathematical coherence and cohesion ( some people call this beauty 🙂 )
LikeLike
Two quick gut reactions.
The folk theorem that seems to have some merit to it is that physicists (and mathematicians to a lesser extent) make their biggest break throughs rather early in their careers. The conventional explanation for this has to do with when one peaks in physicist specific aptitudes. But, your account suggests that differing institutional incentives re risk v. predictable output at different stages of a career play a bigger factor. Hence, for example, a post-doc is equally likely based upon his stage of career to make his or her big break at age 27 or 45 because it is the stage of career that influences what hypotheses he or she takes risks on pursuing. The implication, in turn, is that if we could rejigger institutional incentives to be more like those facing grad students, post-docs and assistant professors, and less like those facing associate and full professors, that we’d get more bang for our buck out of the theoretical physics enterprise.
In experimental physics, especially in HEP but in a wide variety of other sub-fields as well (e.g. neutrino physics and mixed media astronomy observations including gravitation and neutrino observatories and space telescopes and many different wavelengths of EM plus cosmic ray observations) the functional working group size have gotten very large, which is one reason why group think can (and must) prevail in HEP for it to work in that domain.
In theoretical physics, the normative working group is a sole individual with a black board and a computer in a room with a couple of journal subscriptions, occasionally expanded to two to half a dozen investigators.
If we genuinely have a shortage of worthwhile projects for particular theoretical physicists to pursue, one naive response would be that the size of the ordinary working group in theoretical physics ought to be larger (maybe a dozen or two investigators in any one unit in the same sub-field) so that a group leader can effectively funnel efforts on the most promising lines of work to multiple investigators in the sub-field in a coordinated fashion (a la the Manhattan project).
Obviously, there are institutional problems with this so long as most scientists are attached to colleges and universities that have a co-equal need to teach students as well as research and don’t have a student demand for, for example, a dozen amplitude specialists at one institution. But, in the era of virtual collaboration and relatively cheap long distance travel, this can be overcome and indeed the volume of global collaborations in physics (outside big physics experimental collaborations) is surging. Perhaps if we gave these collaborations more formal recognition and institutional clout (i.e. money and promotion influence), this more fruitful model could be developed better.
Kindred to point 2, at some point, do we need to make access to subfields “selective” since while we need string theorists and SUSY theorists and inflation theorists, for example, the diminishing returns set in rather severely at some point when we have too many. In other words, should we more strongly incentivize C-list investigators in popular fields prone to the kind of problems discussed to seek out another specialty. Once upon a time, a limited supply of credible academic journals to publish in seemed to accomplish this end, but that tool seems to have ceased to function as a barrier to entry.
Again, if the problem is that people are writing marginal papers because there are a limited number of good angles to pursue at any given time, we should limit the number of paper slots available.
This may be a bit unfair to the C-list investigator who gets a good idea now and then, but winner take all/rent seeking kind of behavior is produced by the current system. Witness, for example, the plethora of Higgs mass predictions at every mass before it was measured, or the multitude of inflation proposals, or the swarm around the 750 GeV anomaly. In a winner take all roulette, there is an incentive to have someone with a bet on every possibility rather than focusing on the most fruitful potential approaches.
LikeLike
I suspect institutional incentives on risk go the other direction: postdocs and grad students work on less risky topics because they have their career to worry about. I think the extent to which tenured professors avoid risks due to incentives is largely because they’re incentivized to get good positions for their postdocs and grad students, which requires giving them less risky projects.
Theoretical physics tends to have large-ish collaborations whenever it can. The collaboration I’m in is fairly large by theoretical physics standards, but part of that’s because there’s enough coding involved that multiple people speed things up, and can do so even at a distance. There is a lot of theoretical physics that is by its nature much more limited to smaller groups or individuals, where collaboration requires being able to walk over to the other person’s office and write on their blackboard whenever the mood arises, and goes much more slowly when that isn’t possible.
It’s also important to remember that most of the time we don’t get projects handed to us, we have to come up with the ideas ourselves. It might be easy to judge someone from the outside as being mired in unproductive work, but that doesn’t mean they themself have any constructive ideas to get out of it.
Access to subfields is already somewhat selective in that different subfields get different access to grant money, but that isn’t much of a constraint in practice. It’s more selective in terms of people. The same sort of gentle pressure that drives people away from unproductive research by slowly moving from one type of work to a closely related one has a much greater impact on grad students, who tend to mostly pick up what their advisor is working on then without the baggage of history. (Example: I do amplitudes because my advisor happened to be working on an amplitudes project at the time, even though he wasn’t an amplitudes researcher in general.)
I don’t really mind the occasional flood of papers on a topic that’s unlikely to pan out precisely because they’re usually low effort. These aren’t people devoting huge amounts of time to the 750 GeV anomaly, they’re putting in enough to get it through and then doing something else. If there’s a problem there it’s one of where the media pays attention, or where the field assigns accolades. (Also the fact that the “let’s all give our opinions about the new anomaly by fitting existing models” thing hasn’t been automated yet.)
LikeLike
“I suspect institutional incentives on risk go the other direction: postdocs and grad students work on less risky topics because they have their career to worry about. I think the extent to which tenured professors avoid risks due to incentives is largely because they’re incentivized to get good positions for their postdocs and grad students, which requires giving them less risky projects.”
Risk may not be the idea word to use because it is too ambiguous and vague in this context.
What grad students, postdocs and assistant professors must do is to survey all of the viable research prospects in a much wider swath of physics than any one person will every end up doing research in (after having just recently received state of the art information about what is going on in a broad swath of subfields through their studies), and make a high stakes bet on which one to pursue knowing that they need to deliver results in just a few years or they won’t find another research position and will get kicked off the physics research island.
This is an irreducible, unavoidable high stakes bet that they have to make whether they want to or not, each and every one of them. And, they make it with far less sunk costs than tenured professors or endowed research fellows, causing them to be comparatively unbiased in their evaluation of different research prospects relative to someone who has already made one of these big bets and is now living with their choice (which was a choice that paid off well enough for them to find a permanent place on the physics research island). New professional physicists (I started off saying young, but it is really a matter of being new), thus have extraordinarily strong incentives to do their absolute level best to identify a research program that will be fruitful and will produce results on a time horizon of a few years. They have the particularly intense incentive to produce that a dog gives to a rabbit.
Tenured professors, in contrast, have longer time horizons, more sunk costs in a particular research specialization, far lower consequences for failure, and also often find that they have more “life” to deal with than they did when they were graduate students or postdocs or assistance professors dealing with family, bureaucratic responsibilities at an institution and in an associated community, and so on.
LikeLike
Ah ok, I see what you were going for there.
Here’s the thing, though: in practice, very few people get to consciously make that bet.
Undergrads choosing grad schools don’t have enough information to pick the most productive field, for the most part. Once they’re in grad school, they have limited choices, mostly whatever the professors there are working on at the time. In practice, it’s even more limited: theorists usually can’t productively supervise more than a few students each, and often don’t have the funding for more than that, so you have to pick among those who are actually accepting students. Rule out a few due to incompatible personalities and the average grad student doesn’t actually get much of a choice.
After that? The sunk costs, while perhaps lower than a tenured professor, are still quite high. Grad school specializes you hard. In order to go from almost complete ignorance of the field to productive research work in four or five years you have to build up a fairly specific set of skills. You can try to maintain broadness, and some people succeed, but it’s not easy.
Of course, people do branch out more as postdocs, and as assistant professors. When they do, though, there’s usually still an adjacency principle at work. You have to be confident not just that a topic is one that you want to work on but that you, specifically, have something to contribute to it. Often that means finding transferable skills. This means you never really jump “too far”.
If anything, tenured professors are able to make those “jumps” much more easily. They can take some time, publish less, and read up on a new area. They’ve seen more of others’ research, had more conversations with their colleagues, which gives them more understanding of what other fields need.
And yeah, the lack of consequences for failure means they won’t always make these choices well. But you do need room to make the choice in the first place, and I don’t think the incentives placed on early-career researchers give them the chance to make that choice at all.
LikeLike
Tangentially related http://phdcomics.com/comics/archive.php?comicid=1990
LikeLike
Dear Tetragraviton, it’s a fine essay of yours. First, I surely agree science should better be productive and move closer towards the understanding of Nature. I am less sure that these two descriptions are as equivalent as you suggest. But they’re OK. On the other hand, they sound so obvious and nearly empty that they could be clichés.
Now, falsifiability has to exist in principle, and it typically does in good physics. I certainly do want to believe that the two of us are on a similar frequency here. On top of that, falsifiability is surely not sufficient for great science and not even for some tolerably good science.
While “dark matter that kills dinosaurs” wasn’t quite my cup of tea – I was reading this Lisa’s book before it was published really intensely, like other books by her, providing her with lots of feedback – I do think it’s completely interesting at some disciplinary level and it’s also a plausible hypothesis that simply has to be paid attention to.
There are some cycles and events in the “deeply historical sciences which include cosmology, geology, astrophysics, paleoclimate science, evolution, perhaps anthropology and others, and lots of the phenomena in them may be less independent than naively believe. So if there are relationships like that, they simply have to be understood. The dark-matter-extinction link (like the plausible link between some very slow cycles of the Earth’s climate and the spiral arms of the Milky Way, like Nir Shaviv discusses) seems like a rather natural one and it needs to be tested – it’s not quite a random combination of concepts. It’s one of the at most “dozens” of remotely plausible, broad hypotheses linking historical events in the interdisciplinary way. So I think that you, like others, have been unfair to Lisa here, and I think so despite the fact that I always bet – and especially bet after the negative article at the Quanta Magazine – that the probability that this link is accurate enough is much less than 50%.
Your comments about the career are interesting. First, I had the tendency to say “the criterion of good physics should have nothing to do with careers”. Newton and Hooke ultimately worked analogously. It doesn’t matter how they got the funding to survive, and an honest scientist shouldn’t really be affected by the money, otherwise he’s corrupt. On the other hand, the sponsors may pay whomever they want and they influence what’s going on in “science in the whole society” by these pressures.
There are some things that shouldn’t be affected by the sponsors. Sponsors shouldn’t tell you “you get X thousand dollars if you write papers arguing for the cancellation of some amplitudes up to X loops” LOL. Or if they tell you, you should still determine the conclusions about the X-loop amplitudes independently of the money, right? This influence would be a clear corrupting influence on science which is wrong. Do you agree? The sponsors could – and do – encourage some kind of productivity. But I think it’s mostly happening.
Concerning the mundane and unproductive training work: In the career research, which I left a decade ago, there exist lots of personalities and approaches to work. Some people are more obsessed with truly game-changing ideas, and I have always silently counted myself to these seers ;-), but that obviously faces the risk that one finds nothing for a year or years. Tom Banks would always be telling me that Nati Seiberg has a standard athletic prescription to write at least 4 papers with deep technical content a year. That keeps him in some athletic form. I think his papers are always pretty impressive so I would never call it unproductive. But yes, when someone less prestigious does things like Seiberg, it may end up with less impressive results. Is it OK to write 4 or perhaps 10 derivative papers a year? I think it’s perfectly fine.
At some moment, the sponsors and hiring committees must make decisions according to some signs that have already emerged. So they could have promised in 10 ten-citation largely derivative papers or 1-2 stellar paper(s) with may have just 20 but carries some suggestion that the author could make some real revolution at a non-negligible probability. There are really different styles to do physics, even with a highly different number of published papers per year, even the number of hours spent in the office per year ;-), and so on. Many such approaches may lead to great results or poor results and one shouldn’t be dogmatic and prescribe something as in a factory simply because the intellectual work requires much more freedom and flexibility and the more game-changing it is, the less predictable the circumstances are. And again, I think that the good places are doing a rather good job in this respect. I can guarantee that people with lots of these strategies and types and degree of “regularity” of work etc. have been hired if they were good enough. It’s simply not true that only people who do research like mundane workers in the factory get hired. Hiring committees ultimately contain some very good folks and those would love to hire a genius for their place. But one can’t hired a guy without a visible publication record just because he screams that he’s a new better Einstein, do you agree? 😉
Good people are doing physics in many different ways, with different focal points, different strategies, different rate of writing on the blackboard and papers, different amount and type of communication, and the hiring committees are also picking people according to slightly different criteria from each other (and the individual members differ from each other as well), and so on. I think that all this diversity is just right and desirable. There are those critics and one could ask: What do they actually want to change about the status quo and why? Would the changes make things better? Are they impartial and motivated by some sensible effort to improve things?
My feeling is that all those who want to become “overlords” – the chronic critic who are often close to P.R. and the media – either totally misunderstand some totally fundamental things about how good science emerges and what’s needed for that – they typically want to throw away almost all the babies out with the bath water; or they understand that their constraints are too tight and not allowing important work, but they want to cripple physics, anyway; or they are proposing things just to elevate themselves. They are sometimes proposing to remove all meritocratic criteria whatsoever (because all of them make life harder for themselves relatively to others!) and hire people and give them grants regardless of anything they have shown, just by their big mouth and arrogance. I am sorry but this is crazy and extremely harmful.
LikeLike
I deleted your follow-up comment because it went into explicit political statements, which I generally moderate out. If you can find a way to rephrase without the political content then you are welcome to re-post.
LikeLike
No problem, I have lived in a totalitarian regime for 16 years so I am used to a similar behavior by jerks like you. Feel free to erase this one as well.
LikeLike