Category Archives: Life as a Physicist

Science and Its Customers

In most jobs, you know who you’re working for.

A chef cooks food, and people eat it. A tailor makes clothes, and people wear them. An artist has an audience, an engineer has end users, a teacher has students. Someone out there benefits directly from what you do. Make them happy, and they’ll let you know. Piss them off, and they’ll stop hiring you.

Science benefits people too…but most of its benefits are long-term. The first person to magnetize a needle couldn’t have imagined worldwide electronic communication, and the scientists who uncovered quantum mechanics couldn’t have foreseen transistors, or personal computers. The world benefits just by having more expertise in it, more people who spend their lives understanding difficult things, and train others to understand difficult things. But those benefits aren’t easy to see for each individual scientist. As a scientist, you typically don’t know who your work will help, or how much. You might not know for years, or even decades, what impact your work will have. Even then, it will be difficult to tease out your contribution from the other scientists of your time.

We can’t ask the customers of the future to pay for the scientists of today. (At least, not straightforwardly.) In practice, scientists are paid by governments and foundations, groups trying on some level to make the future a better place. Instead of feedback from customers we get feedback from each other. If our ideas get other scientists excited, maybe they’ll matter down the road.

This is a risky thing to do, of course. Governments, foundations, and scientists can’t tell the future. They can try to act in the interests of future generations, but they might just act for themselves instead. Trying to plan ahead like this makes us prey to all the cognitive biases that flesh is heir to.

But we don’t really have an alternative. If we want to have a future at all, if we want a happier and more successful world, we need science. And if we want science, we can’t ask its real customers, the future generations, to choose whether to pay for it. We need to work for the smiles on our colleagues faces and the checks from government grant agencies. And we need to do it carefully enough that at the end of the day, we still make a positive difference.

What You Don’t Know, You Can Parametrize

In physics, what you don’t know can absolutely hurt you. If you ignore that planets have their own gravity, or that metals conduct electricity, you’re going to calculate a lot of nonsense. At the same time, as physicists we can’t possibly know everything. Our experiments are never perfect, our math never includes all the details, and even our famous Standard Model is almost certainly not the whole story. Luckily, we have another option: instead of ignoring what we don’t know, we can parametrize it, and estimate its effect.

Estimating the unknown is something we physicists have done since Newton. You might think Newton’s big discovery was the inverse-square law for gravity, but others at the time, like Robert Hooke, had also been thinking along those lines. Newton’s big discovery was that gravity was universal: that you need to know the effect of gravity, not just from the sun, but from all the other planets as well. The trouble was, Newton didn’t know how to calculate the motion of all of the planets at once (in hindsight, we know he couldn’t have). Instead, he estimated, using what he knew to guess how big the effect of what he didn’t would be. It was the accuracy of those guesses, not just the inverse square law by itself, that convinced the world that Newton was right.

If you’ve studied electricity and magnetism, you get to the point where you can do simple calculations with a few charges in your sleep. The world doesn’t have just a few charges, though: it has many charges, protons and electrons in every atom of every object. If you had to keep all of them in your calculations you’d never pass freshman physics, but luckily you can once again parametrize what you don’t know. Often you can hide those charges away, summarizing their effects with just a few numbers. Other times, you can treat materials as boundaries, and summarize everything beyond in terms of what happens on the edge. The equations of the theory let you do this, but this isn’t true for every theory: for the Navier-Stokes equation, which we use to describe fluids, it still isn’t known whether you can do this kind of trick.

Parametrizing what we don’t know isn’t just a trick for college physics, it’s key to the cutting edge as well. Right now we have a picture for how all of particle physics works, called the Standard Model, but we know that picture is incomplete. There are a million different theories you could write to go beyond the Standard Model, with a million different implications. Instead of having to use all those theories, physicists can summarize them all with what we call an effective theory: one that keeps track of the effect of all that new physics on the particles we already know. By summarizing those effects with a few parameters, we can see what they would have to be to be compatible with experimental results, ruling out some possibilities and suggesting others.

In a world where we never know everything, there’s always something that can hurt us. But if we’re careful and estimate what we don’t know, if we write down numbers and parameters and keep our options open, we can keep from getting burned. By focusing on what we do know, we can still manage to understand the world.

When and How Scientists Reach Out

You’ve probably heard of the myth of the solitary scientist. While Newton might have figured out calculus isolated on his farm, most scientists work better when they communicate. If we reach out to other scientists, we can make progress a lot faster.

Even if you understand that, you might not know what that reaching out actually looks like. I’ve seen far too many crackpots who approach scientific communication like a spammer: sending out emails to everyone in a department, commenting in every vaguely related comment section they can find. While commercial spammers hope for a few gullible people among the thousands they contact, that kind of thing doesn’t benefit crackpots. As far as I can tell, they communicate that way because they genuinely don’t know any better.

So in this post, I want to give a road map for how we scientists reach out to other scientists. Keep these steps in mind, and if you ever need to reach out to a scientist you’ll know what to do.

First, decide what you want to know. This may sound obvious, but sometimes people skip this step. We aren’t communicating just to communicate, but because we want to learn something from the other person. Maybe it’s a new method or idea, maybe we just want confirmation we’re on the right track. We don’t reach out just to “show our theory”, but because we hope to learn something from the response.

Then, figure out who might know it. To do this, we first need to decide how specialized our question is. We often have questions about specific papers: a statement we don’t understand, a formula that seems wrong, or a method that isn’t working. For those, we contact an author from that paper. Other times, the question hasn’t been addressed in a paper, but does fall under a specific well-defined topic: a particular type of calculation, for example. For those we seek out a specialist on that specific topic. Finally, sometimes the question is more general, something anyone in our field might in principle know but we happen not to. For that kind of question, we look for someone we trust, someone we have a prior friendship with and feel comfortable asking “dumb questions”. These days, we can supplement that with platforms like PhysicsOverflow that let us post technical questions and invite anyone to respond.

Note that, for all of these, there’s some work to do first. We need to read the relevant papers, bone up on a topic, even check Wikipedia sometimes. We need to put in enough work to at least try to answer our question, so that we know exactly what we need the other person for.

Finally, contact them appropriately. Papers will usually give contact information for one, or all, of the authors. University websites will give university emails. We’d reach out with something like that first, and switch to personal email (or something even more casual, like Skype or social media) only for people we already have a track record of communicating with in that way.

By posing and directing our questions well, scientists can reach out and get help when we struggle. Science is a team effort, we’re stronger when we work together.

Grants at the Other End

I’m a baby academic. Two years ago I got my first real grant, a Marie Curie Individual Fellowship from the European Union. Applying for it was a complicated process, full of Word templates and mismatched expectations. Two years later the grant is over, and I get another new experience: grant reporting.

Writing a report after a grant is sort of like applying for a grant. Instead of summarizing and justifying what you intend to do, you summarize and justify what you actually did. There are also Word templates. Grant reports are probably easier than grant applications: you don’t have to “hook” your audience or show off. But they are harder in one aspect: they highlight the different ways different fields handle uncertainty.

If you do experiments, having a clear plan makes sense. You buy special equipment and hire postdocs and even technicians to do specific jobs. Your experiments may or may not find what you hope for, but at least you can try to do them on schedule, and describe the setbacks when you can’t.

As a theorist, you’re more nimble. Your equipment are computers, your postdocs have their own research. Overall, it’s easy to pick up new projects as new ideas come in. As a result, your plans change more. New papers might inspire you to try new things. They might also discourage you, if you learn the idea you had won’t actually work. The field can move fast, and you want to keep up with it.

Writing my first grant report will be interesting. I’ll need to thread the gap between expectations and reality, to look back on my progress and talk about why. And of course, I have to do it in Microsoft Word.

The Pointy-Haired University

We all know what it looks like when office work sucks. Maybe you think of Dilbert, or The Office, or the dozens of other comics and shows with the same theme. You picture characters like Dilbert’s Pointy-Haired Boss, stupid and controlling, terrible people with far too much power.

Pictured: what you picture

What does it look like when grad school sucks?

There aren’t a lot of comics, or shows, about grad school. The main one I can think of is PHD Comics.

There are a few characters like the Pointy-Haired Boss in PHD Comics, who are just genuinely bad people, in particular the main character’s advisor Professor Smith. But for the most part, the dysfunction the comic depicts is subtler. Characters aren’t selfish so much as oblivious, they aren’t demanding out of malice but out of misplaced expectations, they’re ineffective not due to incompetence but to understandable human weaknesses.

The comic gets this mostly right. If you’re struggling in grad school, you might have a Pointy-Haired Advisor. But more likely, you’re surrounded by well-meaning, reasonable, intelligent people, who nevertheless are somehow making your life a living hell.

In that situation, it can be tempting to blame yourself. You instinctively look for someone at fault, some terrible person who’s causing the problem, and nobody knows your own faults better than you do.

But before you blame yourself, consider another possibility. Consider that there aren’t just Pointy-Haired Bosses, but Pointy-Haired Institutions. Start with the wrong rules, the wrong incentives, the wrong access to information and accountability, and those well-meaning, intelligent people will end up doing some pretty stupid things. Before deciding you aren’t good enough, ask yourself: is this the only way things could have gone? Instead of a Pointy-Haired Advisor, or a Pointy-Haired Self, maybe you’re just attending a Pointy-Haired University.

Formal Theory and Simulated Experiment

There are two kinds of theoretical physicists. Some, called phenomenologists, make predictions about the real world. Others, the so-called “formal theorists”, don’t. They work with the same kinds of theories as the phenomenologists, quantum field theories of the sort that have been so successful in understanding the subatomic world. But the specific theories they use are different: usually, toy models that aren’t intended to describe reality.

Most people get this is valuable. It’s useful to study toy models, because they help us tackle the real world. But they stumble on another point. Sure, they say, you can study toy models…but then you should call yourself a mathematician, not a physicist.

I’m a “formal theorist”. And I’m very much not a mathematician, I’m definitely a physicist. Let me explain why, with an analogy.

As an undergrad, I spent some time working in a particle physics lab. The lab had developed a new particle detector chip, designed for a future experiment: the International Linear Collider. It was my job to test this chip.

Naturally, I couldn’t test the chip by flinging particles at it. For one, the collider it was designed for hadn’t been built yet! Instead, I had to use simulated input: send in electrical signals that mimicked the expected particles, and see what happens. In effect, I was using a kind of toy model, as a way to understand better how the chip worked.

I hope you agree that this kind of work counts as physics. It isn’t “just engineering” to feed simulated input into a chip. Not when the whole point of that chip is to go into a physics experiment. This kind of work is a large chunk of what an experimental physicist does.

As a formal theorist, my work with toy models is an important part of what a theoretical physicist does. I test out the “devices” of theoretical physics, the quantum-field-theoretic machinery that we use to investigate the world. Without that kind of careful testing on toy models, we’d have fewer tools to work with when we want to understand reality.

Ok, but you might object: an experimental physicist does eventually build the real experiment. They don’t just spend their career on simulated input. If someone only works on formal theory, shouldn’t that at least make them a mathematician, not a physicist?

Here’s the thing, though: after those summers in that lab, I didn’t end up as an experimental physicist. After working on that chip, I didn’t go on to perfect it for the International Linear Collider. But it would be rather bizarre if that, retroactively, made my work in that time “engineering” and not “physics”.

Oh, I should also mention that the International Linear Collider might not ever be built. So, there’s that.

Formal theory is part of physics because it cares directly about the goals of physics: understanding the real world. It is just one step towards that goal, it doesn’t address the real world alone. But neither do the people testing out chips for future colliders. Formal theory isn’t always useful, similarly, planned experiments don’t always get built. That doesn’t mean it’s not physics.

Kicking Students Out of Their Homes During a Pandemic: A Bad Idea

I avoid talking politics on this blog. There are a few issues, though, where I feel not just able, but duty-bound, to speak out. Those are issues affecting graduate students.

This week, US Immigration and Customs Enforcement (ICE) announced that, if a university switched to online courses as a response to COVID-19, international students would have to return to their home countries or transfer to a school that still teaches in-person.

This is already pretty unreasonable for many undergrads. But think about PhD students.

Suppose you’re a foreign PhD student at a US university. Maybe your school is already planning to have classes online this fall, like Harvard is. Maybe your school is planning to have classes in person, but will change its mind a few weeks in, when so many students and professors are infected that it’s clearly unreasonable to continue. Maybe your school never changes its mind, but your state does, and the school has to lock down anyway.

As a PhD student, you likely don’t live in the dorms. More likely you live in a shared house, or an apartment. You’re an independent adult. Your parents aren’t paying for you to go to school. Your school is itself a full-time job, one that pays (as little as the university thinks it can get away with).

What happens when your school goes online? If you need to leave the country?

You’d have to find some way out of your lease, or keep paying for it. You’d have to find a flight on short notice. You’d have to pack up all your belongings, ship or sell anything you can’t store, or find friends to hold on to it.

You’d have to find somewhere to stay in your “home country”. Some could move in with their parents temporarily, many can’t. Some of those who could in other circumstances, shouldn’t if they’re fleeing from an outbreak: their parents are likely older, and vulnerable to the virus. So you have to find a hotel, eventually perhaps a new apartment, far from what was until recently your home.

Reminder: you’re doing all of this on a shoestring budget, because the university pays you peanuts.

Can you transfer instead? In a word, no.

PhD students are specialists. They’re learning very specific things from very specific people. Academics aren’t the sort of omnidisciplinary scientists you see in movies. Bruce Banner or Tony Stark could pick up a new line of research on a whim, real people can’t. This is why, while international students may be good at the undergraduate level, they’re absolutely necessary for PhDs. When only three people in the world study the thing you want to study, you don’t have the luxury of staying in your birth country. And you can’t just transfer schools when yours goes online.

It feels like the people who made this decision didn’t think about any of this. That they don’t think grad students matter, or forgot they exist altogether. It seems frustratingly common for policy that affects grad students to be made by people who know nothing about grad students, and that baffles me. PhDs are a vital part of the academic career, without them universities in their current form wouldn’t even exist. Ignoring them is like if hospital policy ignored residencies.

I hope that this policy gets reversed, or halted, or schools find some way around it. At the moment, anyone starting school in the US this fall is in a very tricky position. And anyone already there is in a worse one.

As usual, I’m going to ask that the comments don’t get too directly political. As a partial measure to tone things down, I’d like to ask you to please avoid mentioning any specific politicians, political parties, or political ideologies. Feel free to talk instead about your own experiences: how this policy is likely to affect you, or your loved ones. Please also feel free to talk more technically on the policy/legal side. I’d like to know what universities can do to work around this, and whether there are plausible paths to change or halt the policy. Please be civil, and be kind to your fellow commenters.

The Citation Motivation Situation

Citations are the bread and butter of academia, or maybe its prison cigarettes. They link us together, somewhere between a map to show us the way and an informal currency. They’re part of how the world grades us, a measure more objective than letters from our peers but that’s not saying much. It’s clear why we we want to be cited, but why do we cite others?

For more reasons than you’d expect.

First, we cite to respect priority. Since the dawn of science, we’ve kept track not only of what we know, but of who figured it out first. If we use an idea in our paper, we cite its origin: the paper that discovered or invented it. We don’t do this for the oldest and most foundational ideas: nobody cites Einstein for relativity. But if the idea is at all unusual, we make sure to give credit where credit is due.

Second, we cite to substantiate our claims. Academic papers don’t stand on their own: they depend on older proofs and prior discoveries. If we make a claim that was demonstrated in older work, we don’t need to prove it again. By citing the older work, we let the reader know where to look. If they doubt our claim, they can look at the older paper and see what went wrong.

Those two are the most obvious uses of citations, but there are more. Another important use is to provide context. Academic work doesn’t stand alone: we choose what we work on in part based on how it relates to other work. As such, it’s important to cite that other work, to help readers understand our motivation. When we’re advancing the state of the art, we need to tell the reader what that state of the art is. When we’re answering a question or solving a problem, we can cite the paper that asked the question or posed the problem. When we’re introducing a new method or idea, we need to clearly say what’s new about it: how it improves on older, similar ideas.

Scientists are social creatures. While we often have a scientific purpose in mind, citations also follow social conventions. These vary from place to place, field to field, and sub-field to sub-field. Mention someone’s research program, and you might be expected to cite every paper in that program. Cite one of a pair of rivals, and you should probably cite the other one too. Some of these conventions are formalized in the form of “citeware“, software licenses that require citations, rather than payments, to use. Others come from unspoken cultural rules. Citations are a way to support each other, something that can slightly improve another’s job prospects at no real cost to your own. It’s not surprising that they ended up part of our culture, well beyond their pure academic use.

In Defense of Shitty Code

Scientific programming was in the news lately, when doubts were raised about a coronavirus simulation by researchers at Imperial College London. While the doubts appear to have been put to rest, doing so involved digging through some seriously messy code. The whole situation seems to have gotten a lot of people worried. If these people are that bad at coding, why should we trust their science?

I don’t know much about coronavirus simulations, my knowledge there begins and ends with a talk I saw last month. But I know a thing or two about bad scientific code, because I write it. My code is atrocious. And I’ve seen published code that’s worse.

Why do scientists write bad code?

In part, it’s a matter of training. Some scientists have formal coding training, but most don’t. I took two CS courses in college and that was it. Despite that lack of training, we’re expected and encouraged to code. Before I took those courses, I spent a summer working in a particle physics lab, where I was expected to pick up the C++-based interface pretty much on the fly. I don’t think there’s another community out there that has as much reason to code as scientists do, and as little training for it.

Would it be useful for scientists to have more of the tools of a trained coder? Sometimes, yeah. Version control is a big one, I’ve collaborated on papers that used Git and papers that didn’t, and there’s a big difference. There are coding habits that would speed up our work and lead to fewer dead ends, and they’re worth picking up when we have the time.

But there’s a reason we don’t prioritize “proper coding”. It’s because the things we’re trying to do, from a coding perspective, are really easy.

What, code-wise, is a coronavirus simulation? A vector of “people”, really just simple labels, all randomly infecting each other and recovering, with a few parameters describing how likely they are to do so and how long it takes. What do I do, code-wise? Mostly, giant piles of linear algebra.

These are not some sort of cutting-edge programming tasks. These are things people have been able to do since the dawn of computers. These are things that, when you screw them up, become quite obvious quite quickly.

Compared to that, the everyday tasks of software developers, like making a reliable interface for users, or efficient graphics, are much more difficult. They’re tasks that really require good coding practices, that just can’t function without them.

For us, the important part is not the coding itself, but what we’re doing with it. Whatever bugs are in a coronavirus simulation, they will have much less impact than, for example, the way in which the simulation includes superspreaders. Bugs in my code give me obviously wrong answers, bad scientific assumptions are much harder for me to root out.

There’s an exception that proves the rule here, and it’s that, when the coding task is actually difficult, scientists step up and write better code. Scientists who want to run efficiently on supercomputers, who are afraid of numerical error or need to simulate on many scales at once, these people learn how to code properly. The code behind the LHC still might be jury-rigged by industry standards, but it’s light-years better than typical scientific code.

I get the furor around the Imperial group’s code. I get that, when a government makes a critical decision, you hope that their every input is as professional as possible. But without getting too political for this blog, let me just say that whatever your politics are, if any of it is based on science, it comes from code like this. Psychology studies, economic modeling, polling…they’re using code, and it’s jury-rigged to hell. Scientists just have more important things to worry about.

The Point of a Model

I’ve been reading more lately, partially for the obvious reasons. Mostly, I’ve been catching up on books everyone else already read.

One such book is Daniel Kahneman’s “Thinking, Fast and Slow”. With all the talk lately about cognitive biases, Kahneman’s account of his research on decision-making was quite familiar ground. The book turned out to more interesting as window into the culture of psychology research. While I had a working picture from psychologist friends in grad school, “Thinking, Fast and Slow” covered the other side, the perspective of a successful professor promoting his field.

Most of this wasn’t too surprising, but one passage struck me:

Several economists and psychologists have proposed models of decision making that are based on the emotions of regret and disappointment. It is fair to say that these models have had less influence than prospect theory, and the reason is instructive. The emotions of regret and disappointment are real, and decision makers surely anticipate these emotions when making their choices. The problem is that regret theories make few striking predictions that would distinguish them from prospect theory, which has the advantage of being simpler. The complexity of prospect theory was more acceptable in the competition with expected utility theory because it did predict observations that expected utility theory could not explain.

Richer and more realistic assumptions do not suffice to make a theory successful. Scientists use theories as a bag of working tools, and they will not take on the burden of a heavier bag unless the new tools are very useful. Prospect theory was accepted by many scholars not because it is “true” but because the concepts that it added to utility theory, notably the reference point and loss aversion, were worth the trouble; they yielded new predictions that turned out to be true. We were lucky.

Thinking Fast and Slow, page 288

Kahneman is contrasting three theories of decision making here: the old proposal that people try to maximize their expected utility (roughly, the benefit they get in future), his more complicated “prospect theory” that takes into account not only what benefits people get but their attachment to what they already have, and other more complicated models based on regret. His theory ended up more popular, both than the older theory and than the newer regret-based models.

Why did his theory win out? Apparently, not because it was the true one: as he says, people almost certainly do feel regret, and make decisions based on it. No, his theory won because it was more useful. It made new, surprising predictions, while being simpler and easier to use than the regret-based models.

This, a theory defeating another without being “more true”, might bug you. By itself, it doesn’t bug me. That’s because, as a physicist, I’m used to the idea that models should not just be true, but useful. If we want to test our theories against reality, we have a large number of “levels” of description to choose from. We can “zoom in” to quarks and gluons, or “zoom out” to look at atoms, or molecules, or polymers. We have to decide how much detail to include, and we have real pragmatic reasons for doing so: some details are just too small to measure!

It’s not clear Kahneman’s community was doing this, though. That is, it doesn’t seem like he’s saying that regret and disappointment are just “too small to be measured”. Instead, he’s saying that they don’t seem to predict much differently from prospect theory, and prospect theory is simpler to use.

Ok, we do that in physics too. We like working with simpler theories, when we have a good excuse. We’re just careful about it. When we can, we derive our simpler theories from more complicated ones, carving out complexity and estimating how much of a difference it would have made. Do this carefully, and we can treat black holes as if they were subatomic particles. When we can’t, we have what we call “phenomenological” models, models built up from observation and not from an underlying theory. We never take such models as the last word, though: a phenomenological model is always viewed as temporary, something to bridge a gap while we try to derive it from more basic physics.

Kahneman doesn’t seem to view prospect theory as temporary. It doesn’t sound like anyone is trying to derive it from regret theory, or to make regret theory easier to use, or to prove it always agrees with regret theory. Maybe they are, and Kahneman simply doesn’t think much of their efforts. Either way, it doesn’t sound like a major goal of the field.

That’s the part that bothered me. In physics, we can’t always hope to derive things from a more fundamental theory, some theories are as fundamental as we know. Psychology isn’t like that: any behavior people display has to be caused by what’s going on in their heads. What Kahneman seems to be saying here is that regret theory may well be closer to what’s going on in people’s heads, but he doesn’t care: it isn’t as useful.

And at that point, I have to ask: useful for what?

As a psychologist, isn’t your goal ultimately to answer that question? To find out “what’s going on in people’s heads”? Isn’t every model you build, every theory you propose, dedicated to that question?

And if not, what exactly is it “useful” for?

For technology? It’s true, “Thinking Fast and Slow” describes several groups Kahneman advised, most memorably the IDF. Is the advantage of prospect theory, then, its “usefulness”, that it leads to better advice for the IDF?

I don’t think that’s what Kahneman means, though. When he says “useful”, he doesn’t mean “useful for advice”. He means it’s good for giving researchers ideas, good for getting people talking. He means “useful for designing experiments”. He means “useful for writing papers”.

And this is when things start to sound worryingly familiar. Because if I’m accusing Kahneman’s community of giving up on finding the fundamental truth, just doing whatever they can to write more papers…well, that’s not an uncommon accusation in physics as well. If the people who spend their lives describing cognitive biases are really getting distracted like that, what chance does, say, string theory have?

I don’t know how seriously to take any of this. But it’s lurking there, in the back of my mind, that nasty, vicious, essential question: what are all of our models for?

Bonus quote, for the commenters to have fun with:

I have yet to meet a successful scientist who lacks the ability to exaggerate the importance of what he or she is doing, and I believe that someone who lacks a delusional sense of significance will wilt in the face of repeated experiences of multiple small failures and rare successes, the fate of most researchers.

Thinking Fast and Slow, page 264