I had two pieces up this month, one in New Scientist and one in Quanta. I figured I’d give a bit of “bonus info” for both here.

The New Scientist piece covered a paper arguing that, according to the supernova evidence, there is no need for dark energy, because the universe isn’t speeding up after all. That’s a pretty dramatic claim, and it’s one the majority of cosmologists don’t agree with. But a small group has been hammering at the consensus.

Long-time followers of this blog have already heard of these folks. I criticized news coverage around a paper with one of the authors, Subir Sarkar, back in 2016, when he was only arguing that the supernova evidence was insufficient to establish dark energy, not that it was literally the wrong way around. He and his co-authors wanted an opportunity to state their case, so I posted their response a few weeks later. Later, I got to know Rameez, another author on the new paper. He ended up doing a guest post in 2019, on the next step in the story.

That story went from a statistical critique to an active proposal for something outside the consensus, in the form of inhomogeneous cosmology, the idea that the universe may be lumpier than typically assumed, with flowing currents that explain things otherwise attributed to exotic physics. That’s another topic I’ve covered before, but in this article I didn’t have the space to go into it.

Sarkar and Rameez are claiming something much more radical than most proponents of inhomogeneous cosmology, arguing against dark energy as a whole, not just more subtle effects. They told me a lot about their reasoning, most of which also didn’t make it into the piece.

It’s largely not going to make it here, either. Not due to space limitations: I’ve been pushing the blog post lengths you folks tolerate for a long time, no reason to stop now. Rather, it’s because I genuinely don’t feel qualified to judge this.

The issue is, cosmology is messy.

How do you figure out how the universe is expanding? You can look at supernovae, and use how dim they are to estimate how far away they are. The issue is, supernovae vary, even famous “standard candles”: they move in different ways, they shine differently depending on different things. Simulations aren’t nearly advanced enough to tell you all this from first principles, and so there are a wash of different corrections based largely on empirical observations, taking this or that correlation seriously and factoring them out of the measurement, while disregarding other correlations as spurious. Which of these combinations of corrections you trust determines whether the universe seems to be accelerating or decelerating.

It makes me glad I went into particle physics instead of cosmology, where you can mostly model everything from first-principles, and compare to your models.

Of course, even that can get you into trouble.

Take that Quanta piece. I wrote an update of the muon g-2 measurement. In some ways, this is an issue that many think of as already solved, by precisely that kind of first-principles modeling. Muons seemed to be violating the Standard Model, according to a calculation based in part on empirical formulas. Lattice QCD researchers figured out how to do that part of the prediction via a first-principles simulation instead, with enough accuracy that it could substitute for the empirical formulas. And lo and behold, the new prediction agreed with the experiment. Eventually, the experts accepted it, and all was well.

Except, as I commented at the time, not really. Because the empirical formulas were based on other experiments, colliding electrons and positrons. And as I now understand in much more detail, the disagreement between those electron-positron experiments is worryingly large.

So nobody is resting on their laurels. The lattice folks improved their calculation, with a result published in Nature this year that prompted Quanta to ask me to report on it. The people using the empirical method are still trying to sort out what happened. And the experimentalists are busy scrutinizing how they analyze their data, and collecting and analyzing more.

A few details that didn’t make it into the piece:

First: it goes beyond the level I was aiming for in the piece, but I really do want to emphasize the role of error estimates. Ultimately, every step in the story was not just one where experiments and predictions disagree, but one where they disagree past their reported error. The problem with disagreement between the new experiments isn’t that they disagree, it’s that they disagree so badly that they’re straining the statistical methods the people using the empirical method use to combine results together, so badly that if taken seriously, those methods would have to throw away ten years of progress of increasing precision. I don’t expect it, but I really hope for a postmortem in which we learn how to better estimate the kinds of experimental errors that can cause this. I haven’t seen that kind of postmortem for other results, so I don’t expect one here. But I have to believe that in the background someone is learning something, and getting better at this.

Second: one interesting question my editors raised was whether it was possible to do a first-principles calculation to compare with the electron-positron experiments. The answer is yes, but with some caveats. One will be recognizable to physicists: lattice QCD can only compute energy-integrated cross-sections, not measurements at particular energies like experiments find. But they can compute integrals with a modulating function…such as one strongly peaked at a particular energy. It’s a familiar type of trick for a quantum field theorist, though in this case it’s trickier than it sounds. I actually misunderstood, and thought that the same people I was talking to about the muon g-2 calculation had done that, and found it agreed with one of the electron-positron experiments over another. That’s not true: their comparisons were more indirect, while other groups attempting more direct comparisons haven’t quite gotten something good enough to do more than gesture at an agreement. So while the direction was correct, there are indeed lattice-based suggestions that favor one experiment over the others, the piece phrased things much too definitely. There’ll be a correction fixing this.