Tag Archives: PublicPerception

Confidence and Friendliness in Science

6 Replies

I’ve seen three kinds of scientific cultures.

First, there are folks who are positive about almost everyone. Ask them about someone else’s lab, even a competitor, and they’ll be polite at worst, and often downright excited. Anyone they know, they’ll tell you how cool the work they’re doing is, how it’s important and valuable and worth doing. They might tell you they prefer a different approach, but they’ll almost never bash someone’s work.

I’ve heard this comes out of American culture, and I can kind of see it. There’s an attitude in the US that everything needs to be described as positively as possible. This is especially true in a work context. Negativity is essentially a death sentence, doled out extremely rarely: if you explicitly say someone or their work is bad, you’re trying to get them fired. You don’t do that unless someone really really deserves it.

That style of scientific culture is growing, but it isn’t universal. There’s still a big cultural group that is totally ok with negativity: as long as it’s directed at other people, anyway.

This scientific culture prides itself on “telling it like it is”. They’ll happily tell you about how everything everyone else is doing is bullshit. Sometimes, they claim their ideas are the only ways forward. Others will have a small number of other people who they trust, who have gained their respect in one way or another. This sort of culture is most stereotypically associated with Russians: a “Russian-style” seminar, for example, is one where the speaker is aggressively questioned for hours.

It may sound like those are the only two options, but there is a third. While “American-style” scientists don’t criticize anyone, and “Russian-style” scientists criticize everyone else, there are also scientists who criticize almost everyone, including themselves.

With a light touch, this culture can be one of the best. There can be a real focus on “epistemic humility”, on always being clear of how much we still don’t know.

However, it can be worryingly easy to spill past that light touch, into something toxic. When the criticism goes past humility and into a lack of confidence in your own work, you risk falling into a black hole, where nothing is going well and nobody has a way out. This kind of culture can spread, filling a workplace and infecting anyone who spends too long there with the conviction that nothing will ever measure up again.

If you can’t manage that light skeptical touch, then your options are American-style or Russian-style. I don’t think either is obviously better. Both have their blind spots: the Americans can let bad ideas slide to avoid rocking the boat, while the Russians can be blind to their own flaws, confident that because everyone else seems wrong they don’t need to challenge their own worldview.

You have one more option, though. Now that you know this, you can recognize each for what it is: not the one true view of the world, but just one culture’s approach to the truth. If you can do that, you can pick up each culture as you need, switching between them as you meet different communities and encounter different things. If you stay aware, you can avoid fighting over culture and discourse, and use your energy on what matters: the science.

From Journal to Classroom

6 Replies

As part of the pedagogy course I’ve been taking, I’m doing a few guest lectures in various courses. I’ve got one coming up in a classical mechanics course (“intermediate”-level, so not Newton’s laws, but stuff the general public doesn’t know much about like Hamiltonians). They’ve been speeding through the core content, so I got to cover a “fun” topic, and after thinking back to my grad school days I chose a topic I think they’ll have a lot of fun with: Chaos theory.

Getting the obligatory Warhammer reference out of the way now

Chaos is one of those things everyone has a vague idea about. People have heard stories where a butterfly flaps its wings and causes a hurricane. Maybe they’ve heard of the rough concept, determinism with strong dependence on the initial conditions, so a tiny change (like that butterfly) can have huge consequences. Maybe they’ve seen pictures of fractals, and got the idea these are somehow related.

Its role in physics is a bit more detailed. It’s one of those concepts that “intermediate classical mechanics” is good for, one that can be much better understood once you’ve been introduced to some of the nineteenth century’s mathematical tools. It felt like a good way to show this class that the things they’ve learned aren’t just useful for dusty old problems, but for understanding something the public thinks is sexy and mysterious.

As luck would have it, the venerable textbook the students are using includes a (2000’s era) chapter on chaos. I read through it, and it struck me that it’s a very different chapter from most of the others. This hit me particularly when I noticed a section describing a famous early study of chaos, and I realized that all the illustrations were based on the actual original journal article.

I had surprisingly mixed feelings about this.

On the one hand, there’s a big fashion right now for something called research-based teaching. That doesn’t mean “using teaching methods that are justified by research” (though you’re supposed to do that too), but rather, “tying your teaching to current scientific research”. This is a fashion that makes sense, because learning about cutting-edge research in an undergraduate classroom feels pretty cool. It lets students feel more connected with the scientific community, it inspires them to get involved, and it gets them more used to what “real research” looks like.

On the other hand, structuring your textbook based on the original research papers feels kind of lazy. There’s a reason we don’t teach Newtonian mechanics the way Newton would have. Pedagogy is supposed to be something we improve at over time: we come up with better examples and better notation, more focused explanations that teach what we want students to learn. If we just summarize a paper, we’re not really providing “added value”: we should hope, at this point, that we can do better.

Thinking about this, I think the distinction boils down to why you’re teaching the material in the first place.

With a lot of research-based teaching, the goal is to show the students how to interact with current literature. You want to show them journal papers, not because the papers are the best way to teach a concept or skill, but because reading those papers is one of the skills you want to teach.

That makes sense for very current topics, but it seems a bit weird for the example I’ve been looking at, an early study of chaos from the 60’s. It’s great if students can read current papers, but they don’t necessarily need to read older ones. (At least, not yet.)

What then, is the textbook trying to teach? Here things get a bit messy. For a relatively old topic, you’d ideally want to teach not just a vague impression of what was discovered, but concrete skills. Here though, those skills are just a bit beyond the students’ reach: chaos is more approachable than you’d think, but still not 100% something the students can work with. Instead they’re learning to appreciate concepts. This can be quite valuable, but it doesn’t give the kind of structure that a concrete skill does. In particular, it makes it hard to know what to emphasize, beyond just summarizing the original article.

In this case, I’ve come up with my own way forward. There are actually concrete skills I’d like to teach. They’re skills that link up with what the textbook is teaching, skills grounded in the concepts it’s trying to convey, and that makes me think I can convey them. It will give some structure to the lesson, a focus on not merely what I’d like the students to think but what I’d like them to do.

I won’t go into too much detail: I suspect some of the students may be reading this, and I don’t want to spoil the surprise! But I’m looking forward to class, and to getting to try another pedagogical experiment.

The Folks With the Best Pictures

1 Reply

Sometimes I envy astronomers. Particle physicists can write books full of words and pages of colorful graphs and charts, and the public won’t retain any of it. Astronomers can mesmerize the world with a single picture.

NASA just released the first images from its James Webb Space Telescope. They’re impressive, and not merely visually: in twelve hours, they probe deeper than the Hubble Space Telescope managed in weeks on the same patch of sky, as well as gathering data that can show what kinds of molecules are present in the galaxies.

(If you’re curious how the James Webb images compare to Hubble ones, here’s a nice site comparing them.)

Images like this enter the popular imagination. The Hubble telescope’s deep field has appeared on essentially every artistic product one could imagine. As of writing this, searching for “Hubble” on Etsy gives almost 5,000 results. “JWST”, the acronym for the James Webb Space Telescope, already gives over 1,000, including several on the front page that already contain just-released images. Despite the Large Hadron Collider having operated for over a decade, searching “LHC” also leads to just around 1,000 results…and a few on the front page are actually pictures of the JWST!

It would be great as particle physicists to have that kind of impact…but I think we shouldn’t stress ourselves too much about it. Ultimately astronomers will always have this core advantage. Space is amazing, visually stunning and mind-bogglingly vast. It has always had a special place for human cultures, and I’m happy for astronomers to inherit that place.

Proxies for Proxies

Leave a reply

Why pay scientists?

Maybe you care about science itself. You think that exploring the world should be one of our central goals as human beings, that it “makes our country worth defending”.

Maybe you care about technology. You support science because, down the line, you think it will give us new capabilities that improve people’s lives. Maybe you expect this to happen directly, or maybe indirectly as “spinoff” inventions like the internet.

Maybe you just think science is cool. You want the stories that science tells: they entertain you, they give you a place in the world, they help distract from the mundane day to day grind.

Maybe you just think that the world ought to have scientists in it. You can think of it as a kind of bargain, maintaining expertise so that society can tackle difficult problems. Or you can be more cynical, paying early-career scientists on the assumption that most will leave academia and cheapen labor costs for tech companies.

Maybe you want to pay the scientists to teach, to be professors at universities. You notice that they don’t seem to be happy if you don’t let them research, so you throw a little research funding at them, as a treat.

Maybe you just want to grow your empire: your department, your university, the job numbers in your district.

In most jobs, you’re supposed to do what people pay you to do. As a scientist, the people who pay you have all of these motivations and more. You can’t simply choose to do what people pay you to do.

So you come up with a proxy. You sum up all of these ideas, into a vague picture of what all those people want. You have some idea of scientific quality: not just a matter of doing science correctly and carefully, but doing interesting science. It’s not something you ever articulate. It’s likely even contradictory, after all, the goals it approximates often are. Nonetheless, it’s your guide, and not just your guide: it’s the guide of those who hire you, those who choose if you get promoted or whether you get more funding. All of these people have some vague idea in their head of what makes good science, their own proxy for the desires of the vast mass of voters and decision-makers and funders.

But of course, the standard is still vague. Should good science be deep? Which topics are deeper than others? Should it be practical? Practical for whom? Should it be surprising? What do you expect to happen, and what would surprise you? Should it get the community excited? Which community?

As a practicing scientist, you have to build your own proxy for these proxies. The same work that could get you hired in one place might meet blank stares at another, and you can’t build your life around those unpredictable quirks. So you make your own vague idea of what you’re supposed to do, an alchemy of what excites you and what makes an impact and what your friends are doing. You build a stand-in in your head, on the expectation that no-one else will have quite the same stand-in, then go out and convince the other stand-ins to give money to your version. You stand on a shifting pile of unwritten rules, subtler even than some artists, because at the end of the day there’s never a real client to be seen. Just another proxy.

Gateway Hobbies

Leave a reply

When biologists tell stories of their childhoods, they’re full of trails of ants and fireflies in jars. Lots of writers start young, telling stories on the playground and making skits with their friends. And the mere existence of “chemistry sets” tells you exactly how many chemists get started. Many fields have these “gateway hobbies”, like gateway drugs for careers, ways that children and teenagers get hooked and gain experience.

Physics is a little different, though. While kids can play with magnets and electricity, there aren’t a whole lot of other “physics hobbies”, especially for esoteric corners like particle physics. Instead, the “gateway hobbies” of physics are more varied, drawing from many different fields.

First, of course, even if a child can’t “do physics”, they can always read about it. Kids will memorize the names of quarks, read about black holes, or watch documentaries about string theory. I’m not counting this as a “physics hobby” because it isn’t really: physics isn’t a collection of isolated facts, but of equations: frameworks you can use to make predictions. Reading about the Big Bang is a good way to get motivated and excited, it’s a great thing to do…but it doesn’t prepare you for the “science part” of the science.

A few efforts at physics popularization get a bit more hands-on. Many come in the form of video games. You can get experience with relativity through Velocity Raptor, quantum mechanics through Quantum Chess, or orbital mechanics through Kerbal Space Program. All of these get just another bit closer to “doing physics” rather than merely reading about it.

One can always gain experience in other fields, and that can be surprisingly relevant. Playing around with a chemistry set gives first-hand experience of the kinds of things that motivated quantum mechanics, and some things that still motivate condensed matter research. Circuits are physics, more directly, even if they’re also engineering: and for some physicists, designing electronic sensors is a huge part of what they do.

Astronomy has a special place, both in the history of physics and the pantheon of hobbies. There’s a huge amateur astronomy community, one that both makes real discoveries and reaches out to kids of all ages. Many physicists got their start looking at the heavens, using it like Newton’s contemporaries as a first glimpse into the mechanisms of nature.

More and more research in physics involves at least some programming, and programming is another activity kids have access to in spades, from Logo to robotics competitions. Learning how to program isn’t just an important skill: it’s also a way for young people to experience a world bound by clear laws and logic, another motivation to study physics.

Of course, if you’re interested in rules and logic, why not go all the way? Plenty of physicists grew up doing math competitions. I have fond memories of Oregon’s Pentagames, and the more “serious” activities go all the way up to the famously challenging Putnam Competition.

Finally, there are physics competitions too, at least in the form of the International Physics Olympiad, where high school students compete in physics prowess.

Not every physicist did these sorts of things, of course: some got hooked later. Others did more than one. A friend of mine who’s always been “Mr. Science” got almost the whole package, with a youth spent exploring the wild west of the early internet, working at a planetarium, and discovering just how easy it is to get legal access to dangerous and radioactive chemicals. There are many paths in to physics, so even if kids can’t “do physics” the same way they “do chemistry”, there’s still plenty to do!

Keeping It Colloquial

2 Replies

In the corners of academia where I hang out, a colloquium is a special kind of talk. Most talks we give are part of weekly seminars for specific groups. For example, the theoretical particle physicists here have a seminar. Each week we invite a speaker, who gives a talk on their recent work. Since they expect an audience of theoretical particle physicists, they can go into more detail.

A colloquium isn’t like that. Colloquia are talks for the whole department: theorists and experimentalists, particle physicists and biophysicists. They’re more prestigious, for big famous professors (or sometimes, for professors interviewing for jobs…). The different audience, and different context, means that the talk plays by different rules.

Recently, I saw a conference full of “colloquium-style” talks, trying to play by these rules. Some succeeded, some didn’t…and I think I now have a better idea of how those rules work.

First, in a colloquium, you’re not just speaking for yourself. You’re an ambassador for your field. For some of the audience, this might be the first time they’ve heard a talk by someone who does your kind of research. You want to give them a good impression, not just about you, but about the whole topic. So while you definitely want to mention your own work, you want to tell a full story, one that gives more than a glimpse of what others are doing as well.

Second, you want to connect to something the audience already knows. With an audience of physicists, you can assume a certain baseline, but not much more than that. You need to make the beginning accessible and start with something familiar. For the conference I mentioned, a talk that did this well was the talk on exoplanets, which started with the familiar planets of the solar system, classifying them in order to show what you might expect exoplanets to look like. In contrast, t’Hooft’s talk did this poorly. His work is exploiting a loophole in a quantum-mechanical argument called Bell’s theorem, which most physicists have heard of. Instead of mentioning Bell’s theorem, he referred vaguely to “criteria from philosophers”, and only even mentioned that near the end of the talk, instead starting with properties of quantum mechanics his audience was much less familiar with.

Moving on, then, you want to present a mystery. So far, everything in the talk has made sense, and your audience feels like they understand. Now, you show them something that doesn’t fit, something their familiar model can’t accommodate. This activates your audience’s scientist instincts: they’re curious now, they want to know the answer. A good example from the conference was a talk on chemistry in space. The speaker emphasized that we can see evidence of complex molecules in space, but that space dust is so absurdly dilute that it seems impossible such molecules could form: two atoms could go a billion years without meeting each other.

You can’t just leave your audience mystified, though. You next have to solve the mystery. Ideally, your solution will be something smart, but simple: something your audience can intuitively understand. This has two benefits. First, it makes you look smart: you described a mysterious problem, and then you show how to solve it! Second, it makes the audience feel smart: they felt the problem was hard, but now they understand how to solve it too. The audience will have good feelings about you as a result, and good feelings about the topic: in some sense, you’ve tied a piece of their self-esteem to knowing the solution to your problem. This was well-done by the speaker discussing space chemistry, who explained that the solution was chemistry on surfaces: if two atoms are on the surface of a dust grain or meteorite, they’re much more likely to react. It was also well-done by a speaker discussing models of diseases like diabetes: he explained the challenge of controlling processes with cells, when cells replicate exponentially, and showed one way they could be controlled, when the immune system kills off any cells that replicate much faster than their neighbors. (He also played the guitar to immune system-themed songs…also a good colloquium strategy for those who can pull it off!)

Finally, a picture is worth a thousand words…as long as it’s a clear one. For an audience that won’t follow most of your equations, it’s crucial to show them something visual: graphics, puns, pictures of equipment or graphs. Crucially, though, your graphics should be something the audience can understand. If you put up a graph with a lot of incomprehensible detail: parameters you haven’t explained, or just set up in a way your audience doesn’t get, then your audience gets stuck. Much like an unfamiliar word, a mysterious graph will have members of the audience scratching their heads, trying to figure out what it means. They’ll be so busy trying, they’ll miss what you say next, and you’ll lose them! So yes, put in graphs, put in pictures: but make sure that the ones you use, you have time to explain.

How Expert Is That Expert?

7 Replies

The blog Astral Codex Ten had an interesting post a while back, about when to trust experts. Rather than thinking of some experts as “trustworthy” and some as “untrustworthy”, the post suggests an approach of “bounded distrust”. Even if an expert is biased or a news source sometimes lies, there are certain things you can still expect them to tell the truth about. If you are familiar enough with their work, you can get an idea of which kinds of claims you can trust and which you can’t, in a consistent and reliable way. Knowing how to do this is a skill, one you can learn to get better at.

In my corner of science, I can’t think of anyone who outright lies. Nonetheless, some claims are worth more trust than others. Sometimes experts have solid backing for what they say, direct experience that’s hard to contradict. Other times they’re speaking mostly from general impressions, and bias could easily creep in. Luckily, it’s not so hard to tell the difference. In this post, I’ll try to teach you how.

For an example, I’ll use something I saw at a conference last week. A speaker gave a talk describing the current state of cosmology: the new tools we have to map the early universe, and the challenges in using them to their full potential. After the talk, I remember her answering three questions. In each case, she seemed to know what she was talking about, but for different reasons. If she was contradicted by a different expert, I’d use these reasons to figure out which one to trust.

First, sometimes an expert gives what is an informed opinion, but just an informed opinion. As scientists, we are expected to know a fairly broad range of background behind our work, and be able to say something informed about it. We see overview talks and hear our colleagues’ takes, and get informed opinions about topics we otherwise don’t work on. This speaker fielded a question about quantum gravity, and her answer made it clear that the topic falls into this category for her. Her answer didn’t go into much detail, mentioning a few terms but no specific scientific results, and linked back in the end to a different question closer to her expertise. That’s generally how we speak on this kind of topic: vaguely enough to show what we know without overstepping.

The second question came from a different kind of knowledge, which I might call journal club knowledge. Many scientists have what are called “journal clubs”. We meet on a regular basis, read recent papers, and talk about them. The papers go beyond what we work on day-to-day, but not by that much, because the goal is to keep an eye open for future research topics. We read papers in close-by areas, watching for elements that could be useful, answers to questions we have or questions we know how to answer. The kind of “journal club knowledge” we have covers a fair amount of detail: these aren’t topics we are working on right now, but if we spent more time on it they could be. Here, the speaker answered a question about the Hubble tension, a discrepancy between two different ways of measuring the expansion of the universe. The way she answered focused on particular results: someone did X, there was a paper showing Y, this telescope is planning to measure Z. That kind of answer is a good way to tell that someone is answering from “journal club knowledge”. It’s clearly an area she could get involved in if she wanted to, one where she knows the important questions and which papers to read, with some of her own work close enough to the question to give an important advantage. But it was also clear that she hadn’t developed a full argument on one “side” or the other, and as such there are others I’d trust a bit more on that aspect of the question.

Finally, experts are the most trustworthy when we speak about our own work. In this speaker’s case, the questions about machine learning were where her expertise clearly shone through. Her answers there were detailed in a different way than her answers about the Hubble tension: not just papers, but personal experience. They were full of phrases like “I tried that, but it doesn’t work…” or “when we do this, we prefer to do it this way”. They also had the most technical terms of any of her answers, terms that clearly drew distinctions relevant to those who work in the field. In general, when an expert talks about what they do in their own work, and uses a lot of appropriate technical terms, you have especially good reason to trust them.

These cues can help a lot when evaluating experts. An expert who makes a generic claim, like “no evidence for X”, might not know as much as an expert who cites specific papers, and in turn they might not know as much as an expert who describes what they do in their own research. The cues aren’t perfect: one risk is that someone may be an expert on their own work, but that work may be irrelevant to the question you’re asking. But they help: rather than distrusting everyone, they help you towards “bounded distrust”, knowing which claims you can trust and which are riskier.

Duality and Emergence: When Is Spacetime Not Spacetime?

21 Replies

Spacetime is doomed! At least, so say some physicists. They don’t mean this as a warning, like some comic-book universe-destroying disaster, but rather as a research plan. These physicists believe that what we think of as space and time aren’t the full story, but that they emerge from something more fundamental, so that an ultimate theory of nature might not use space or time at all. Other, grumpier physicists are skeptical. Joined by a few philosophers, they think the “spacetime is doomed” crowd are over-excited and exaggerating the implications of their discoveries. At the heart of the argument is the distinction between two related concepts: duality and emergence.

In physics, sometimes we find that two theories are actually dual: despite seeming different, the patterns of observations they predict are the same. Some of the more popular examples are what we call holographic theories. In these situations, a theory of quantum gravity in some space-time is dual to a theory without gravity describing the edges of that space-time, sort of like how a hologram is a 2D image that looks 3D when you move it. For any question you can ask about the gravitational “bulk” space, there is a matching question on the “boundary”. No matter what you observe, neither description will fail.

If theories with gravity can be described by theories without gravity, does that mean gravity doesn’t really exist? If you’re asking that question, you’re asking whether gravity is emergent. An emergent theory is one that isn’t really fundamental, but instead a result of the interaction of more fundamental parts. For example, hydrodynamics, the theory of fluids like water, emerges from more fundamental theories that describe the motion of atoms and molecules.

(For the experts: I, like most physicists, am talking about “weak emergence” here, not “strong emergence”.)

The “spacetime is doomed” crowd think that not just gravity, but space-time itself is emergent. They expect that distances and times aren’t really fundamental, but a result of relationships that will turn out to be more fundamental, like entanglement between different parts of quantum fields. As evidence, they like to bring up dualities where the dual theories have different concepts of gravity, number of dimensions, or space-time. Using those theories, they argue that space and time might “break down”, and not be really fundamental.

(I’ve made arguments like that in the past too.)

The skeptics, though, bring up an important point. If two theories are really dual, then no observation can distinguish them: they make exactly the same predictions. In that case, say the skeptics, what right do you have to call one theory more fundamental than the other? You can say that gravity emerges from a boundary theory without gravity, but you could just as easily say that the boundary theory emerges from the gravity theory. The whole point of duality is that no theory is “more true” than the other: one might be more or less convenient, but both describe the same world. If you want to really argue for emergence, then your “more fundamental” theory needs to do something extra: to predict something that your emergent theory doesn’t predict.

Sometimes this is a fair objection. There are members of the “spacetime is doomed” crowd who are genuinely reckless about this, who’ll tell a journalist about emergence when they really mean duality. But many of these people are more careful, and have thought more deeply about the question. They tend to have some mix of these two perspectives:

First, if two descriptions give the same results, then do the descriptions matter? As physicists, we have a history of treating theories as the same if they make the same predictions. Space-time itself is a result of this policy: in the theory of relativity, two people might disagree on which one of two events happened first or second, but they will agree on the overall distance in space-time between the two. From this perspective, a duality between a bulk theory and a boundary theory isn’t evidence that the bulk theory emerges from the boundary, but it is evidence that both the bulk and boundary theories should be replaced by an “overall theory”, one that treats bulk and boundary as irrelevant descriptions of the same physical reality. This perspective is similar to an old philosophical theory called positivism: that statements are meaningless if they cannot be derived from something measurable. That theory wasn’t very useful for philosophers, which is probably part of why some philosophers are skeptics of “space-time is doomed”. The perspective has been quite useful to physicists, though, so we’re likely to stick with it.

Second, some will say that it’s true that a dual theory is not an emergent theory…but it can be the first step to discover one. In this perspective, dualities are suggestive evidence that a deeper theory is waiting in the wings. The idea would be that one would first discover a duality, then discover situations that break that duality: examples on one side that don’t correspond to anything sensible on the other. Maybe some patterns of quantum entanglement are dual to a picture of space-time, but some are not. (Closer to my sub-field, maybe there’s an object like the amplituhedron that doesn’t respect locality or unitarity.) If you’re lucky, maybe there are situations, or even experiments, that go from one to the other: where the space-time description works until a certain point, then stops working, and only the dual description survives. Some of the models of emergent space-time people study are genuinely of this type, where a dimension emerges in a theory that previously didn’t have one. (For those of you having a hard time imagining this, read my old post about “bubbles of nothing”, then think of one happening in reverse.)

It’s premature to say space-time is doomed, at least as a definite statement. But it is looking like, one way or another, space-time won’t be the right picture for fundamental physics. Maybe that’s because it’s equivalent to another description, redundant embellishment on an essential theoretical core. Maybe instead it breaks down, and a more fundamental theory could describe more situations. We don’t know yet. But physicists are trying to figure it out.

The Unpublishable Dirty Tricks of Theoretical Physics

4 Replies

As the saying goes, it is better not to see laws or sausages being made. You’d prefer to see the clean package on the outside than the mess behind the scenes.

The same is true of science. A good paper tells a nice, clean story: a logical argument from beginning to end, with no extra baggage to slow it down. That story isn’t a lie: for any decent paper in theoretical physics, the conclusions will follow from the premises. Most of the time, though, it isn’t how the physicist actually did it.

The way we actually make discoveries is messy. It involves looking for inspiration in all the wrong places: pieces of old computer code and old problems, trying to reproduce this or that calculation with this or that method. In the end, once we find something interesting enough, we can reconstruct a clearer, cleaner, story, something actually fit to publish. We hide the original mess partly for career reasons (easier to get hired if you tell a clean, heroic story), partly to be understood (a paper that embraced the mess of discovery would be a mess to read), and partly just due to that deep human instinct to not let others see us that way.

The trouble is, some of that “mess” is useful, even essential. And because it’s never published or put into textbooks, the only way to learn it is word of mouth.

A lot of these messy tricks involve numerics. Many theoretical physics papers derive things analytically, writing out equations in symbols. It’s easy to make a mistake in that kind of calculation, either writing something wrong on paper or as a bug in computer code. To correct mistakes, many things are checked numerically: we plug in numbers to make sure everything still works. Sometimes this means using an approximation, trying to make sure two things cancel to some large enough number of decimal places. Sometimes instead it’s exact: we plug in prime numbers, and can much more easily see if two things are equal, or if something is rational or contains a square root. Sometimes numerics aren’t just used to check something, but to find a solution: exploring many options in an easier numerical calculation, finding one that works, and doing it again analytically.

“Ansatze” are also common: our fancy word for an educated guess. These we sometimes admit, when they’re at the core of a new scientific idea. But the more minor examples go un-mentioned. If a paper shows a nice clean formula and proves it’s correct, but doesn’t explain how the authors got it…probably, they used an ansatz. This trick can go hand-in-hand with numerics as well: make a guess, check it matches the right numbers, then try to see why it’s true.

The messy tricks can also involve the code itself. In my field we often use “computer algebra” systems, programs to do our calculations for us. These systems are programming languages in their own right, and we need to write computer code for them. That code gets passed around informally, but almost never standardized. Mathematical concepts that come up again and again can be implemented very differently by different people, some much more efficiently than others.

I don’t think it’s unreasonable that we leave “the mess” out of our papers. They would certainly be hard to understand otherwise! But it’s a shame we don’t publish our dirty tricks somewhere, even in special “dirty tricks” papers. Students often start out assuming everything is done the clean way, and start doubting themselves when they notice it’s much too slow to make progress. Learning the tricks is a big part of learning to be a physicist. We should find a better way to teach them.

Don’t Trust the Experiments, Trust the Science

2 Replies

I was chatting with an astronomer recently, and this quote by Arthur Eddington came up:

“Never trust an experimental result until it has been confirmed by theory.”

Arthur Eddington

At first, this sounds like just typical theorist arrogance, thinking we’re better than all those experimentalists. It’s not that, though, or at least not just that. Instead, it’s commenting on a trend that shows up again and again in science, but rarely makes the history books. Again and again an experiment or observation comes through with something fantastical, something that seems like it breaks the laws of physics or throws our best models into disarray. And after a few months, when everyone has checked, it turns out there was a mistake, and the experiment agrees with existing theories after all.

You might remember a recent example, when a lab claimed to have measured neutrinos moving faster than the speed of light, only for it to turn out to be due to a loose cable. Experiments like this aren’t just a result of modern hype: as Eddington’s quote shows, they were also common in his day. In general, Eddington’s advice is good: when an experiment contradicts theory, theory tends to win in the end.

This may sound unscientific: surely we should care only about what we actually observe? If we defer to theory, aren’t we putting dogma ahead of the evidence of our senses? Isn’t that the opposite of good science?

To understand what’s going on here, we can use an old philosophical argument: David Hume’s argument against miracles. David Hume wanted to understand how we use evidence to reason about the world. He argued that, for miracles in particular, we can never have good evidence. In Hume’s definition, a miracle was something that broke the established laws of science. Hume argued that, if you believe you observed a miracle, there are two possibilities: either the laws of science really were broken, or you made a mistake. The thing is, laws of science don’t just come from a textbook: they come from observations as well, many many observations in many different conditions over a long period of time. Some of those observations establish the laws in the first place, others come from the communities that successfully apply them again and again over the years. If your miracle was real, then it would throw into doubt many, if not all, of those observations. So the question you have to ask is: it it more likely those observations were wrong? Or that you made a mistake? Put another way, your evidence is only good enough for a miracle if it would be a bigger miracle if you were wrong.

Hume’s argument always struck me as a little bit too strict: if you rule out miracles like this, you also rule out new theories of science! A more modern approach would use numbers and statistics, weighing the past evidence for a theory against the precision of the new result. Most of the time you’d reach the same conclusion, but sometimes an experiment can be good enough to overthrow a theory.

Still, theory should always sit in the background, a kind of safety net for when your experiments screw up. It does mean that when you don’t have that safety net you need to be extra-careful. Physics is an interesting case of this: while we have “the laws of physics”, we don’t have any established theory that tells us what kinds of particles should exist. That puts physics in an unusual position, and it’s probably part of why we have such strict standards of statistical proof. If you’re going to be operating without the safety net of theory, you need that kind of proof.

This post was also inspired by some biological examples. The examples are politically controversial, so since this is a no-politics blog I won’t discuss them in detail. (I’ll also moderate out any comments that do.) All I’ll say is that I wonder if in that case the right heuristic is this kind of thing: not to “trust scientists” or “trust experts” or even “trust statisticians”, but just to trust the basic, cartoon-level biological theory.