Happy Halloween! I’ve got a yearly tradition on this blog of talking about the spookyside of physics. This year, we’ll think about what happens…when you turn off the lights.
Over history, astronomy has given us larger and larger views of the universe. We started out thinking the planets, Sun, and Moon were human-like, just a short distance away. Measuring distances, we started to understand the size of the Earth, then the Sun, then realized how much farther still the stars were from us. Gradually, we came to understand that some of the stars were much farther away than others. Thinkers like Immanuel Kant speculated that “nebulae” were clouds of stars like our own Milky Way, and in the early 20th century better distance measurements confirmed it, showing that Andromeda was not a nearby cloud, but an entirely different galaxy. By the 1960’s, scientists had observed the universe’s cosmic microwave background, seeing as far out as it was possible to see.
But what if we stopped halfway?
Since the 1920’s, we’ve known the universe is expanding. Since the 1990’s, we’ve thought that that expansion is speeding up: faraway galaxies are getting farther and farther away from us. Space itself is expanding, carrying the galaxies apart…faster than light.
That ever-increasing speed has a consequence. It means that, eventually, each galaxy will fly beyond our view. One by one, the other galaxies will disappear, so far away that light will not have had enough time to reach us.
From our perspective, it will be as if the lights, one by one, started to turn out. Each faraway light, each cloudy blur that hides a whirl of worlds, will wink out. The sky will get darker and darker, until to an astronomer from a distant future, the universe will appear a strangely limited place:
At FirstPrinciples.org, I had a piece covering work by engineering professor Colin McInnes on stability of Dyson spheres and ringworlds. This was a fun one to cover, mostly because of how it straddles the borderline between science fiction and practical physics and engineering. McInnes’s claim to fame is work on solar sails, which seem like a paradigmatic example of that kind of thing: a common sci-fi theme that’s surprisingly viable. His work on stability was interesting to me because it’s the kind of work that a century and a half ago would have been paradigmatic physics. Now, though, very few physicists work on orbital mechanics, and a lot of the core questions have passed on to engineering. It’s fascinating to see how these classic old problems can still have undiscovered solutions, and how the people best equipped to find them now are tinkerers practicing their tools instead of cutting-edge mathematicians.
At Quanta Magazine, I had a piece about reversible computing. Readers may remember I had another piece on that topic at the end of March, a profile on the startup Vaire Computing at FirstPrinciples.org. That piece talked about FirstPrinciples, but didn’t say much about reversible computing. I figured I’d combine the “bonus info” for both posts here.
Neither piece went into much detail about the engineering involved, as it didn’t really make sense in either venue. One thing that amused me a bit is that the core technology that drove Vaire into action is something that actually should be very familiar to a physics or engineering student: a resonator. Theirs is obviously quite a bit more sophisticated than the base model, but at its heart it’s doing the same thing: storing charge and controlling frequency. It turns out that those are both essential to making reversible computers work: you need to store charge so it isn’t lost to ground when you empty a transistor, and you need to control the frequency so you can have waves with gentle transitions instead of the more sharp corners of the waves used in normal computers, thus wasting less heat in rapid changes of voltage. Vaire recently announced they’re getting 50% charge recovery from their test chips, and they’re working on raising that number.
Originally, the Quanta piece was focused more on reversible programming than energy use, as the energy angle seemed a bit more physics-focused than their computer science desk usually goes. The emphasis ended up changing as I worked on the draft, but it meant that an interesting parallel story got lost on the cutting-room floor. There’s a community of people who study reversible computing not from the engineering side, but from the computer science side, studying reversible logic and reversible programming languages. It’s a pursuit that goes back to the 1980’s, where at Caltech around when Feynman was teaching his course on the physics of computing a group of students were figuring out how to set up a reversible programming language. Called Janus, they sent their creation to Landauer, and the letter ended up with Michael Frank after Landauer died. There’s a lovely quote from it regarding their motivation: “We did it out of curiosity over whether such an odd animal as this was possible, and because we were interested in knowing where we put information when we programmed. Janus forced us to pay attention to where our bits went since none could be thrown away.”
Being forced to pay attention to information, in turn, is what has animated the computer science side of the reversible computing community. There are applications to debugging, where you can run code backwards when it gets stuck, to encryption and compression, where you want to be able to recover the information you hid away, and to security, where you want to keep track of information to make sure a hacker can’t figure out things they shouldn’t. Also, for a lot of these people, it’s just a fun puzzle. Early on my attention was caught by a paper by Hannah Earley describing a programming language called Alethe, a word you might recognize from the Greek word for truth, which literally means something like “not-forgetting”.
(Compression is particularly relevant for the “garbage data” you need to output in a reversible computation. If you want to add two numbers reversibly, naively you need to keep both input numbers and their output, but you can be more clever than that and just keep one of the inputs since you can subtract to find the other. There are a lot of substantially more clever tricks in this vein people have figured out over the years.)
I didn’t say anything about the other engineering approaches to reversible computing, that try to do something outside of traditional computer chips. There’s DNA computing, which tries to compute with a bunch of DNA in solution. There’s the old concept of ballistic reversible computing, where you imagine a computer that runs like a bunch of colliding billiard balls, conserving energy. Coordinating such a computer can be a nightmare, and early theoretical ideas were shown to be disrupted by something as tiny as a few stray photons from a distant star. But people like Frank figured out ways around the coordination problem, and groups have experimented with superconductors as places to toss those billiard balls around. The early billiard-inspired designs also had a big impact on quantum computing, where you need reversible gates and the only irreversible operation is the measurement. The name “Toffoli” comes up a lot in quantum computing discussions, I hadn’t known before this that Toffoli gates were originally for reversible computing in general, not specifically quantum computing.
Finally, I only gestured at the sci-fi angle. For reversible computing’s die-hards, it isn’t just a way to make efficient computers now. It’s the ultimate future of the technology, the kind of energy-efficiency civilization will need when we’re covering stars with shells of “computronium” full of busy joyous artificial minds.
And now that I think about it, they should chat with McInnes. He can tell them the kinds of stars they should build around.
First: I am now on bluesky! Instead of having a separate link in the top menu for each social media account, I’ve changed the format so now there are social media buttons in the right-hand sidebar, right under the “Follow” button. Currently, they cover tumblr, twitter, and bluesky, but there may be more in future.
Second, I’ve put a bit more technical advice on my “Open Source Grant Proposal” post, so people interested in proposing similar research can have some ideas about how best to pitch it.
Now, on to the post:
Gravitational wave telescopes are possibly the most exciting research program in physics right now. Big, expensive machines with more on the way in the coming decades, gravitational wave telescopes need both precise theoretical predictions and high-quality data analysis. For some, gravitational wave telescopes have the potential to reveal genuinely new physics, to probe deviations from general relativity that might be related to phenomena like dark matter, though so far no such deviations have been conclusively observed. In the meantime, they’re teaching us new consequences of known physics. For example, the unusual population of black holes observed by LIGO has motivated those who model star clusters to consider processes in which the motion of three stars or black holes is related to each other, discovering that these processes are more important than expected.
Particle colliders are probably still exciting to the general public, but for many there is a growing sense of fatigue and disillusionment. Current machines like the LHC are big and expensive, and proposed future colliders would be even costlier and take decades to come online, in addition to requiring a huge amount of effort from the community in terms of precise theoretical predictions and data analysis. Some argue that colliders still might uncover genuinely new physics, deviations from the standard model that might explain phenomena like dark matter, but as no such deviations have yet been conclusively observed people are increasingly skeptical. In the meantime, most people working on collider physics are focused on learning new consequences of known physics. For example, by comparing observed results with theoretical approximations, people have found that certain high-energy processes usually left out of calculations are actually needed to get a good agreement with the data, showing that these processes are more important than expected.
…ok, you see what I did there, right? Was that fair?
There are a few key differences, with implications to keep in mind:
First, collider physics is significantly more expensive than gravitational wave physics. LIGO took about $300 million to build and spends about $50 million a year. The LHC took about $5 billion to build and costs $1 billion a year to run. That cost still puts both well below several other government expenses that you probably consider frivolous (please don’t start arguing about which ones in the comments!), but it does mean collider physics demands a bit of a stronger argument.
Second, the theoretical motivation to expect new fundamental physics out of LIGO is generally considered much weaker than for colliders. A large part of the theoretical physics community thought that they had a good argument why they should see something new at the LHC. In contrast, most theorists have been skeptical of the kinds of modified gravity theories that have dramatic enough effects that one could measure them with gravitational wave telescopes, with many of these theories having other pathologies or inconsistencies that made people wary.
Third, the general public finds astrophysics cooler than particle physics. Somehow, telling people “pairs of black holes collide more often than we thought because sometimes a third star in the neighborhood nudges them together” gets people much more excited than “pairs of quarks collide more often than we thought because we need to re-sum large logarithms differently”, even though I don’t think there’s a real “principled” difference between them. Neither reveals new laws of nature, both are upgrades to our ability to model how real physical objects behave, neither is useful to know for anybody living on Earth in the present day.
With all this in mind, my advice to gravitational wave physicists is to try, as much as possible, not to lean on stories about dark matter and modified gravity. You might learn something, and it’s worth occasionally mentioning that. But if you don’t, you run a serious risk of disappointing people. And you have such a big PR advantage if you just lean on new consequences of bog standard GR, that those guys really should get the bulk of the news coverage if you want to keep the public on your side.
What’s the difference between a model and an explanation?
Suppose you cared about dark matter. You observe that things out there in the universe don’t quite move the way you would expect. There is something, a consistent something, that changes the orbits of galaxies and the bending of light, the shape of the early universe and the spiderweb of super-clusters. How do you think about that “something”?
One option is to try to model the something. You want to use as few parameters as possible, so that your model isn’t just an accident, but will actually work to predict new data. You want to describe how it changes gravity, on all the scales you care about. Your model might be very simple, like the original MOND, and just describe a modification to Newtonian gravity, since you typically only need Newtonian gravity to model many of these phenomena. (Though MOND itself can’t account for all the things attributed to dark matter, so it had to be modified.) You might have something slightly more complicated, proposing some “matter” but not going into much detail about what it is, just enough for your model to work.
If you were doing engineering, a model like that is a fine thing to have. If you were building a spaceship and wanted to figure out what its destination would look like after a long journey, you’d need a model of dark matter like this, one that predicted how galaxies move and light bends, to do the job.
But a model like that isn’t an explanation. And the reason why is that explanations generalize.
In practice, you often just need Newtonian gravity to model how galaxies move. But if you want to model more dramatic things, the movement of the whole universe or the area around a black hole, then you need general relativity as well. So to generalize to those areas, you can’t just modify Newtonian gravity. You need an explanation, one that tells you not just how Newton’s equations change, but how Einstein’s equations change.
In practice, you can get by with a simple model of dark matter, one that doesn’t tell you very much, and just adds a new type of matter. But if you want to model quantum gravity, you need to know how this new matter interacts, not just at baseline with gravity, but with everything else. You need to know how the new matter is produced, whether it gets its mass from the Higgs boson or from something else, whether it falls into the same symmetry groups as the Standard Model or totally new ones, how it arises from tangled-up strings and multi-dimensional membranes. You need not just a model, but an explanation, one that tells you not just roughly what kind of particle you need, but how it changes our models of particle physics overall.
Physics, at its best, generalizes. Newton’s genius wasn’t that he modeled gravity on Earth, but that he unified it with gravity in the solar system. By realizing that gravity was universal, he proposed an explanation that led to much more progress than the models of predecessors like Kepler. Later, Einstein’s work on general relativity led to similar progress.
We can’t always generalize. Sometimes, we simply don’t know enough. But if we’re not engineering, then we don’t need a model, and generalizing should, at least in the long-run, be our guiding hope.
Halloween is coming up, so let’s talk about the most prominent monster of the physics canon, the nightmare scenario.
Not to be confused with the D&D Nightmare, which once was a convenient source of infinite consumable items for mid-level characters.
Right now, thousands of physicists search for more information about particle physics beyond our current Standard Model. They look at data from the Large Hadron Collider to look for signs of new particles and unexpected behavior, they try to detect a wide range of possible dark matter particles, and they make very precise measurements to try to detect subtle deviations. And in the back of their minds, almost all of those physicists wonder if they’ll find anything at all.
It’s not that we think the Standard Model is right. We know it has problems, deep mathematical issues that make it give nonsense answers and an apparent big mismatch with what we observe about the motion of matter and light in the universe. (You’ve probably heard this mismatch called dark matter and dark energy.)
But none of those problems guarantee an answer soon. The Standard Model will eventually fail, but it may fail only for very difficult and expensive experiments, not a Large Hadron Collider but some sort of galactic-scale Large Earth Collider. It might be that none of the experiments or searches or theories those thousands of physicists are working on will tell them anything they didn’t already know. That’s the nightmare scenario.
I don’t know another field that has a nightmare scenario quite like this. In most fields, one experiment or another might fail, not just not giving the expected evidence but not teaching anything new. But most experiments teach us something new. We don’t have a theory, in almost any field, that has the potential to explain every observation up to the limits of our experiments, but which we still hope to disprove. Only the Standard Model is like that.
And while thousands of physicists are exposed to this nightmare scenario, the majority of physicists aren’t. Physics isn’t just the science of the reductionistic laws of the smallest constituents of matter. It’s also the study of physical systems, from the bubbling chaos of nuclear physics to the formation of planets and galaxies and black holes, to the properties of materials to the movement of bacteria on a petri dish and bees in a hive. It’s also the development of new methods, from better control of individual atoms and quantum states to powerful new tricks for calculation. For some, it can be the discovery, not of reductionistic laws of the smallest scales, but of general laws of the largest scales, of how systems with many different origins can show echoes of the same behavior.
Over time, more and more of those thousands of physicists break away from the nightmare scenario, “waking up” to new questions of these kinds. For some, motivated by puzzles and skill and the beauty of physics, the change is satisfying, a chance to work on ideas that are moving forward, connected with experiment or grounded in evolving mathematics. But if your motivation is really tied to those smallest scales, to that final reductionistic “why”, then such a shift won’t be satisfying, and this is a nightmare you won’t wake up from.
Me, I’m not sure. I’m a tool-builder, and I used to tell myself that tool-builders are always needed. But I find I do care, in the end, what my tools are used for. And as we approach the nightmare scenario, I’m not at all sure I know how to wake up.
Reading that, you might ask whether we can do better. What about a model-independent measurement of the age of the universe?
As intuitive as it might seem, we can’t actually do that. In fact, if we’re really strict about it, we can’t get a model-independent measurement of anything at all. Everything is based on a model.
Imagine stepping on your bathroom scale, getting a mass in kilograms. The number it gives you seems as objective as anything. But to get that number, you have to trust that a number of models are true. You have to model gravity, to assume that the scale’s measurement of your weight gives you the right mass based on the Earth’s surface gravity being approximately constant. You have to model the circuits and sensors in the scale, and be confident that you understand how they’re supposed to work. You have to model people: to assume that the company that made the scale tested it accurately, and that the people who sold it to you didn’t lie about where it came from. And finally, you have to model error: you know that the scale can’t possibly give you your exact weight, so you need a rough idea of just how far off it can reasonably be.
Everything we know is like this. Every measurement in science builds on past science, on our understanding of our measuring equipment and our trust in others. Everything in our daily lives comes through a network of assumptions about the world around us. Everything we perceive is filtered through instincts, our understanding of our own senses and knowledge of when they do and don’t trick us.
Ok, but when I say that the age of the universe is model-dependent, I don’t really mean it like that, right?
Everything we know is model-dependent, but only some model-dependence is worth worrying about. Your knowledge of your bathroom scale comes from centuries-old physics of gravity, widely-applied principles of electronics, and a trust in the function of basic products that serves you well in every other aspect of your life. The models that knowledge depends on aren’t really in question, especially not when you just want to measure your weight.
Some measurements we make in physics are like this too. When the experimental collaborations at the LHC measured the Higgs mass, they were doing something far from routine. But the models they based that measurement on, models of particle physics and particle detector electronics and their own computer code, are still so well-tested that it mostly doesn’t make sense to think of this as a model-dependent measurement. If we’re questioning the Higgs mass, it’s only because we’re questioning something much bigger.
The age of the universe, though, is trickier. Our most precise measurements are based on a specific model: we estimate what the universe is made of and how fast it’s expanding, plug it into our model of how the universe changes over time, and get an estimate for the age. You might suggest that we should just look out into the universe and find the oldest star, but that’s model-dependent too. Stars don’t have rings like trees. Instead, to estimate the age of a star we have to have some model for what kind of light it emits, and for how that light has changed over the history of the universe before it reached us.
These models are not quite as well-established as the models behind particle physics, let alone those behind your bathroom scale. Our models of stars are pretty good, applied to many types of stars in many different galaxies, but they do involve big, complicated systems involving many types of extreme and difficult to estimate physics. Star models get revised all the time, usually in minor ways but occasionally in more dramatic ones. Meanwhile, our model of the whole universe is powerful, but by its very nature much less-tested. We can test it on observations of the whole universe today, or on observations of the whole universe in the past (like the cosmic microwave background). And it works well for these, better than any other model. But it’s not inconceivable, not unrealistic, and above all not out of context, that another model could take its place. And if it did, many of the model-dependent measurements we’ve based on it will have to change.
So that’s why, while everything we know is model-dependent, some are model-dependent in a more important way. Some things, even if we feel they have solid backing, may well turn out to be wrong, in a way that we have reason to take seriously. The age of the universe is pretty well-established as these things go, but it still is one of those types of things, where there is enough doubt in our model that we can’t just take the measurement at face value.
“The news doesn’t come from a telescope, though, or a new observation of the sky. Instead, it comes from this press release from the University of Ottawa: “Reinventing cosmology: uOttawa research puts age of universe at 26.7 — not 13.7 — billion years”.
(If you look, you’ll find many websites copying this press release almost word-for-word. This is pretty common in science news, where some websites simply aggregate press releases and others base most of their science news on them rather than paying enough for actual journalism.)
The press release, in turn, is talking about a theory, not an observation. The theorist, Rajendra Gupta, was motivated by examples like the early galaxies observed by JWST and the Methuselah star. Since the 13.8 billion year age of the universe is based on a mathematical model, he tried to find a different mathematical model that led to an older universe. Eventually, by hypothesizing what seems like every unproven physics effect he could think of, he found one that gives a different estimate, 26.7 billion. He probably wasn’t the first person to do this, because coming up with different models to explain odd observations is a standard thing cosmologists do all the time, and until one of the models is shown to explain a wider range of observations (because our best theories explain a lot, so they’re hard to replace), they’re just treated as speculation, not newsworthy science.
This is a pretty clear case of hype, and as such most of the discussion has been about what went wrong. Should we blame the theorist? The university? The journalists? Elon Musk?
Rather than blame, I think it’s more productive to offer advice. And in this situation, the person I think could use some advice is the person who wrote the press release.
So suppose you work for a university, writing their press releases. One day, you hear that one of your professors has done something very cool, something worthy of a press release: they’ve found a new estimate for the age of the universe. What do you do?
One thing you absolutely shouldn’t do is question the science. That just isn’t your job, and even if it were you don’t have the expertise to do that. Anyone who’s hoping that you will only write articles about good science and not bad science is being unrealistic, that’s just not an option.
If you can’t be more accurate, though, you can still be more precise. You can write your article, and in particular your headline, so that you express what you do know as clearly and specifically as possible.
(I’m assuming here you write your own headlines. This is not normal in journalism, where most headlines are written by an editor, not by the writer of a piece. But university press offices are small enough that I’m assuming, perhaps incorrectly, that you can choose how to title your piece.)
Let’s take a look at the title, “Reinventing cosmology: uOttawa research puts age of universe at 26.7 — not 13.7 — billion years”, and see if we can make some small changes to improve it.
One very general word in that title is “research”. Lots of people do research: astronomers do research when they collect observations, theorists do research when they make new models. If you say “research”, some people will think you’re reporting a new observation, a new measurement that gives a radically different age for the universe.
But you know that’s not true, it’s not what the scientist you’re talking to is telling you. So to avoid the misunderstanding, you can get a bit more specific, and replace the word “research” with a more precise one: “Reinventing cosmology: uOttawa theory puts age of universe at 26.7 — not 13.7 — billion years”.
“Theory” is just as familiar a word as “research”. You won’t lose clicks, you won’t confuse people. But now, you’ve closed off a big potential misunderstanding. By a small shift, you’ve gotten a lot clearer. And you didn’t need to question the science to do it!
You can do more small shifts, if you understand a bit more of the science. “Puts” is kind of ambiguous: a theory could put an age somewhere because it computes it from first principles, or because it dialed some parameter to get there. Here, the theory was intentionally chosen to give an older universe, so the title should hint at this in some way. Instead of “puts”, then, you can use “allows”: “Reinventing cosmology: uOttawa theory allows age of universe to be 26.7 — not 13.7 — billion years”.
These kinds of little tricks can be very helpful. If you’re trying to avoid being misunderstood, then it’s good to be as specific as you can, given what you understand. If you do it carefully, you don’t have to question your scientists’ ideas or downplay their contributions. You can do your job, promote your scientists, and still contribute to responsible journalism.
If you follow astronomers on twitter, you may have heard some rumblings. For the last week or so, a few big collaborations have been hyping up an announcement of “something big”.
Those who knew who those collaborations were could guess the topic. Everyone else found out on Wednesday, when the alphabet soup of NANOGrav, EPTA, PPTA, CPTA, and InPTA announced detection of a gravitational wave background.
These guys
Who are these guys? And what have they found?
You’ll notice the letters “PTA” showing up again and again here. PTA doesn’t stand for Parent-Teacher Association, but for Pulsar Timing Array. Pulsar timing arrays keep track of pulsars, special neutron stars that spin around, shooting out jets of light. The ones studied by PTAs spin so regularly that we can use them as a kind of cosmic clock, counting time by when their beams hit our telescopes. They’re so regular that, if we see them vary, the best explanation isn’t that their spinning has changed: it’s that space-time itself has.
Because of that, we can use pulsar timing arrays to detect subtle shifts in space and time, ripples in the fabric of the universe caused by enormous gravitational waves. That’s what all these collaborations are for: the Indian Pulsar Timing Array (InPTA), the Chinese Pulsar Timing Array (CPTA), the Parkes Pulsar Timing Array (PPTA), the European Pulsar Timing Array (EPTA), and the North American Nanohertz Observatory for Gravitational Waves (NANOGrav).
For a nice explanation of what they saw, read this twitter thread by Katie Mack, who unlike me is actually an astronomer. NANOGrav, in typical North American fashion, is talking the loudest about it, but in this case they kind of deserve it. They have the most data, fifteen years of measurements, letting them make the clearest case that they are actually seeing evidence of gravitational waves. (And not, as an earlier measurement of theirs saw, Jupiter.)
We’ve seen evidence of gravitational waves before of course, most recently from the gravitational wave observatories LIGO and VIRGO. LIGO and VIRGO could pinpoint their results to colliding black holes and neutrons stars, estimating where they were and how massive. The pulsar timing arrays can’t quite do that yet, even with fifteen years of data. They expect that the waves they are seeing come from colliding black holes as well, but much larger ones: with pulsars spread over a galaxy, the effects they detect are from black holes big enough to be galactic cores. Rather than one at a time, they would see a chorus of many at once, a gravitational wave background (though not to be confused with a cosmic gravitational wave background: this would be from black holes close to the present day, not from the origin of the universe). If it is this background, then they’re seeing a bit more of the super-massive black holes than people expected. But for now, they’re not sure: they can show they’re seeing gravitational waves, but so far not much more.
With that in mind, it’s best to view the result, impressive as it is, as a proof of principle. Much as LIGO showed, not that gravitational waves exist at all, but that it is possible for us to detect them, these pulsar timing arrays have shown that it is possible to detect the gravitational wave background on these vast scales. As the different arrays pool their data and gather more, the technique will become more and more useful. We’ll start learning new things about the life-cycles of black holes and galaxies, about the shape of the universe, and maybe if we’re lucky some fundamental physics too. We’ve opened up a new window, making sure it’s bright enough we can see. Now we can sit back, and watch the universe.
Recently, someone was telling me about a book they’d read on Karl Schwarzschild. Schwarzschild is famous for discovering the equations that describe black holes, based on Einstein’s theory of gravitation. To make the story more dramatic, he did so only shortly before dying from a disease he caught fighting in the first World War. But this person had the impression that Schwarzschild had done even more. According to this person, the book said that Schwarzschild had done something to prove Einstein’s theory, or to complete it.
Another Schwarzschild accomplishment: that mustache
At first, I thought the book this person had read was wrong. But after some investigation, I figured out what happened.
The book said that Schwarzschild had found the first exact solution to Einstein’s equations. That’s true, and as a physicist I know precisely what it means. But I now realize that the average person does not.
In school, the first equations you solve are algebraic, x+y=z. Some equations, like x^2=4, have solutions. Others, like x^2=-4, seem not to, until you learn about new types of numbers that solve them. Either way, you get used to equations being like a kind of puzzle, a question for which you need to find an answer.
If you’re thinking of equations like that, then it probably sounds like Schwarzschild “solved the puzzle”. If Schwarzschild found the first solution to Einstein’s equation, that means that Einstein did not. That makes it sound like Einstein’s work was incomplete, that he had asked the right question but didn’t yet know the right answer.
Einstein’s equations aren’t algebraic equations, though. They’re differential equations. Instead of equations for a variable, they’re equations for a mathematical function, a formula that, in this case, describes the curvature of space and time.
Scientists in many fields use differential equations, but they use them in different ways. If you’re a chemist or a biologist, it might be that you’re most used to differential equations with simple solutions, like sines, cosines, or exponentials. You learn how to solve these equations, and they feel a bit like the algebraic ones: you have a puzzle, and then you solve the puzzle.
Other fields, though, have tougher differential equations. If you’re a physicist or an engineer, you’ve likely met differential equations that you can’t treat in this way. If you’re dealing with fluid mechanics, or general relativity, or even just Newtonian gravity in an odd situation, you can’t usually solve the problem by writing down known functions like sines and cosines.
That doesn’t mean you can’t solve the problem at all, though!
Even if you can’t write down a solution to a differential equation with sines and cosines, a solution can still exist. (In some cases, we can even prove a solution exists!) It just won’t be written in terms of sines and cosines, or other functions you’ve learned in school. Instead, the solution will involve some strange functions, functions no-one has heard of before.
If you want, you can make up names for those functions. But unless you’re going to classify them in a useful way, there’s not much point. Instead, you work with these functions by approximation. You calculate them in a way that doesn’t give you the full answer, but that does let you estimate how close you are. That’s good enough to give you numbers, which in turn is good enough to compare to experiments. With just an approximate solution, like this, Einstein could check if his equations described the orbit of Mercury.
Once you know you can find these approximate solutions, you have a different perspective on equations. An equation isn’t just a mysterious puzzle. If you can approximate the solution, then you already know how to solve that puzzle. So we wouldn’t think of Einstein’s theory as incomplete because he was only able to find approximate solutions: for a theory as complicated as Einstein’s, that’s perfectly normal. Most of the time, that’s all we need.
But it’s still pretty cool when you don’t have to do this. Sometimes, we can not just approximate, but actually “write down” the solution, either using known functions or well-classified new ones. We call a solution like that an analytic solution, or an exact solution.
That’s what Schwarzschild managed. These kinds of exact solutions often only work in special situations, and Schwarzschild’s is no exception. His Schwarzschild solution works for matter in a special situation, arranged in a perfect sphere. If matter happened to be arranged in that way, then the shape of space and time would be exactly as Schwarzschild described it.
That’s actually pretty cool! Einstein’s equations are complicated enough that no-one was sure that there were any solutions like that, even in very special situations. Einstein expected it would be a long time until they could do anything except approximate solutions.
(If Schwarzschild’s solution only describes matter arranged in a perfect sphere, why do we think it describes real black holes? This took later work, by people like Roger Penrose, who figured out that matter compressed far enough will always find a solution like Schwarzschild’s.)
Schwarzschild intended to describe stars with his solution, or at least a kind of imaginary perfect star. What he found was indeed a good approximation to real stars, but also the possibility that a star shoved into a sufficiently small space would become something weird and new, something we would come to describe as a black hole. That’s a pretty impressive accomplishment, especially for someone on the front lines of World War One. And if you know the difference between an exact solution and an approximate one, you have some idea of what kind of accomplishment that is.
I’ve never met someone who believed the Earth was flat. I’ve met a few who believed it was six thousand years old, but not many. Occasionally, I run into crackpots who rail against relativity or quantum mechanics, or more recent discoveries like quarks or the Higgs. But for one conclusion of modern physics, the doubters are common. For this one idea, the average person may not insist that the physicists are wrong, but they’ll usually roll their eyes a little bit, ask the occasional “really?”
That idea is dark matter.
For the average person, dark matter doesn’t sound like normal, responsible science. It sounds like cheating. Scientists try to explain the universe, using stars and planets and gravity, and eventually they notice the equations don’t work, so they just introduce some new matter nobody can detect. It’s as if a budget didn’t add up, so the accountant just introduced some “dark expenses” to hide the problem.
Part of what’s going on here is that fundamental physics, unlike other fields, doesn’t have to reduce to something else. An accountant has to explain the world in terms of transfers of money, a chemist in terms of atoms and molecules. A physicist has to explain the world in terms of math, with no more restrictions than that. Whatever the “base level” of another field is, physics can, and must, go deeper.
But that doesn’t explain everything. Physics may have to explain things in terms of math, but we shouldn’t just invent new math whenever we feel like it. Surely, we should prefer explanations in terms of things we know to explanations in terms of things we don’t know. The question then becomes, what justifies the preference? And when do we get to break it?
Imagine you’re camping in your backyard. You’ve brought a pack of jumbo marshmallows. You wake up to find a hole torn in the bag, a few marshmallows strewn on a trail into the bushes, the rest gone. You’re tempted to imagine a new species of ant, with enormous jaws capable of ripping open plastic and hauling the marshmallows away. Then you remember your brother likes marshmallows, and it’s probably his fault.
Now imagine instead you’re camping in the Amazon rainforest. Suddenly, the ant explanation makes sense. You may not have a particular species of ants in mind, but you know the rainforest is full of new species no-one has yet discovered. And you’re pretty sure your brother couldn’t have flown to your campsite in the middle of the night and stolen your marshmallows.
We do have a preference against introducing new types of “stuff”, like new species of ants or new particles. We have that preference because these new types of stuff are unlikely, based on our current knowledge. We don’t expect new species of ants in our backyards, because we think we have a pretty good idea of what kinds of ants exist, and we think a marshmallow-stealing brother is more likely. That preference gets dropped, however, based on the strength of the evidence. If it’s very unlikely our brother stole the marshmallows, and if we’re somewhere our knowledge of ants is weak, then the marshmallow-stealing ants are more likely.
Dark matter is a massive leap. It’s not a massive leap because we can’t see it, but simply because it involves new particles, particles not in our Standard Model of particle physics. (Or, for the MOND-ish fans, new fields not present in Einstein’s theory of general relativity.) It’s hard to justify physics beyond the Standard Model, and our standards for justifying it are in general very high: we need very precise experiments to conclude that the Standard Model is well and truly broken.
For dark matter, we keep those standards. The evidence for some kind of dark matter, that there is something that can’t be explained by just the Standard Model and Einstein’s gravity, is at this point very strong. Far from a vague force that appears everywhere, we can map dark matter’s location, systematically describe its effect on the motion of galaxies to clusters of galaxies to the early history of the universe. We’ve checked if there’s something we’ve left out, if black holes or unseen planets might cover it, and they can’t. It’s still possible we’ve missed something, just like it’s possible your brother flew to the Amazon to steal your marshmallows, but it’s less likely than the alternatives.
Also, much like ants in the rainforest, we don’t know every type of particle. We know there are things we’re missing: new types of neutrinos, or new particles to explain quantum gravity. These don’t have to have anything to do with dark matter, they might be totally unrelated. But they do show that we should expect, sometimes, to run into particles we don’t already know about. We shouldn’t expect that we already know all the particles.
If physicists did what the cartoons suggest, it really would be cheating. If we proposed dark matter because our equations didn’t match up, and stopped checking, we’d be no better than an accountant adding “dark money” to a budget. But we didn’t do that. When we argue that dark matter exists, it’s because we’ve actually tried to put together the evidence, because we’ve weighed it against the preference to stick with the Standard Model and found the evidence tips the scales. The instinct to call it cheating is a good instinct, one you should cultivate. But here, it’s an instinct physicists have already taken into account.