Tag Archives: academia

Grad Students Don’t Have Majors

Leave a reply

A pet peeve of mine:

Suppose you’re writing a story, and one of your characters is studying for a PhD in linguistics. You could call them a grad student or a PhD student, a linguistics student or even just a linguist. But one thing you absolutely shouldn’t call them is a linguistics major.

Graduate degrees, from the PhD to medical doctors to masters degrees, don’t have majors. Majors are a very specific concept, from a very specific system: one that only applies to undergraduate degrees, and even there is uncommon to unheard of in most of the world.

You can think of “major” as short for “major area of study”. In many universities in the US, bachelor’s degree students enter not as students of a particular topic, but as “undecided” students. They then have some amount of time to choose a major. Majors define some of your courses, but not all of them. You can also have “minors”, minor areas of study where you take a few courses from another department, and you typically have to take some number of general courses from other departments as well. Overall, the US system for bachelor’s students is quite flexible. The idea is that students can choose from a wide range of courses offered by different departments at a university, focusing on one department’s program but sampling from many. The major is your major focus, but not your only focus.

Basically no other degree works this way.

In Europe, bachelor’s degree students sign up as students of a specific department. By default, all of their courses will be from that department. If you have to learn more math, or writing skills, then normally your department will have its own math or writing course, focused on the needs of their degree. It can be possible to take courses from other departments, but it’s not common and it’s often not easy, sometimes requiring special permission. You’re supposed to have done your general education as a high school student, and be ready to focus on a particular area.

Graduate degrees in the US also don’t work this way. A student in medical school or law school isn’t a medicine major or a law major, they’re a med student or a law student. They typically don’t take courses from the rest of the university at that point, just from the med school or the law school. A student studying for an MBA (Master’s in Business Administration) is similarly a business student, not the business major they might have been during their bachelor’s studies. And a student studying for a PhD is a PhD student, a student of a specific department. They might still have the option of taking classes outside of that department (for example, I took classes in science communication). But these are special exceptions. A linguistics PhD student will take almost all of their classes from the linguistics department, a physics PhD student will take almost all of their classes from the physics department. They don’t have majors.

So the next time you write a story with people with advanced degrees, keep this in mind. Majors are a thing for US bachelor’s degrees, and a few similar systems. Anything else, don’t call it a major!

Toy Models

7 Replies

In academia, scientists don’t always work with what they actually care about. A lot of the time, they use what academics call toy models. A toy model can be a theory with simpler mathematics than the theories that describe the real world, but it can also be something that is itself real, just simpler or easier to work with, like nematodes, fruit flies, or college students.

Some people in industry seem to think this is all academics do. I’ve seen a few job ads that emphasize experience dealing with “real-world data”, and a few people skeptical that someone used to academia would be able to deal with the messy challenges of the business world.

There’s a grain of truth to this, but I don’t think industry has a monopoly on mess. To see why, let’s think about how academics write computer code.

There are a lot of things that one is in-principle supposed to do to code well, and most academics do none of them. Good code has test suites, so that if you change something you can check whether it still works by testing it on all the things that could go wrong. Good code is modular, with functions doing specific things and re-used whenever appropriate. Good code follows shared conventions, so that others can pick up your code and understand how you did it.

Some academics do these things, for example those who build numerical simulations on supercomputers. But for most academics, coding best-practices range from impractical to outright counterproductive. Testing is perhaps the clearest example. To design a test suite, you have to have some idea what kinds of things your code will run into, what kind of input you expect what the output is supposed to be. Many academic projects, though, are the first of their kind. Academics code up something to do a calculation nobody has done before, not knowing the result, or they make code to analyze a dataset nobody has worked with before. By the time they understand the problem well enough to write a test suite, they’ve already solved the problem, and they’re on to the next project, which may need something totally different.

From the perspective of these academics, if you have a problem well-defined enough that you can build a test suite, well enough that you can have stable conventions and reusable functions…then you have a toy model, not a real problem from the real world.

…and of course, that’s not quite fair either, right?

The truth is, academics and businesspeople want to work with toy models. Toy models are well-behaved, and easy, and you can do a lot with them. The real world isn’t a toy model…but it can be, if you make it one.

This means planning your experiments, whether in business or in science. It means making sure the data you gather is labeled and organized before you begin. It means coming up with processes, and procedures, and making as much of the work as possible a standardized, replicable thing. That’s desirable regardless, whether you’re making a consistent product instead of artisanal one-offs or a well-documented scientific study that another team can replicate.

Academia and industry both must handle mess. They handle different kinds of mess in different circumstances, and manage it in different ways, and this can be a real challenge for someone trying to go from one world to another. But neither world is intrinsically messier or cleaner. Nobody has a monopoly on toy models.

The “Who” of Fixing Academic Publishing

2 Replies

I was on the debate team in high school. There’s a type of debate, called Policy, where one team proposes a government policy and the other team argues the policy is bad. The rules of Policy debate don’t say who the debaters are pretending to be: they could be congresspeople, cabinet members, or staff at a think tank. This creates ambiguity, and nerds are great at exploiting ambiguity. A popular strategy was to argue that the opponents had a perfectly good policy, but were wrong about who should implement it. This had reasonable forms (no, congress does not have the power to do X) but could also get very silly (the crux of one debate was whether the supreme court or the undersecretary of the TSA was the best authority to usher in a Malthusian dictatorship). When debating policy, “who” could be much more important than “what”.

Occasionally, when I see people argue that something needs to be done, I ask myself this question. Who, precisely, should do it?

Recently, I saw a tweet complaining about scientific publishing. Physicists put their work out for free on arXiv.org, then submit that work to journals, which charge huge fees either to the scientists themselves or to libraries that want access to the work. It’s a problem academics complain about frequently, but usually we act like it’s something we should fix ourselves, a kind of grassroots movement to change our publication and hiring culture.

This tweet, surprisingly, didn’t do that. Instead, it seemed to have a different “who” in mind. The tweet argued that the stranglehold of publishers like Elsevier on academic publishing is a waste of taxpayer money. The implication, maybe intended maybe not, is that the problem should be fixed by the taxpayers: that is, by the government.

Which in turn got me thinking, what could that look like?

I could imagine a few different options, from the kinds of things normal governments do to radical things that would probably never happen.

First, the most plausible strategy: collective negotiation. Particle physicists don’t pay from our own grants to publish papers, and we don’t pay to read them. Instead, we have a collective agreement, called SCOAP3, where the big institutions pay together each year to guarantee open access. The University of California system tried to negotiate a similar agreement a few years back, not just for physicists but for all fields. You could imagine governments leaning on this, with the university systems of whole countries negotiating a fixed payment. The journals would still be getting paid, but less.

Second, less likely but not impossible: governments could use the same strategies against the big publishers that they use against other big companies. This could be antitrust action (if you have to publish in Nature to get hired, are they really competing with anybody?), or even some kind of price controls. The impression I get is that when governments do try to change scientific publishing they usually do it via restrictions on the scientists (such as requiring them to publish open-access), while this would involve restrictions on the publishers.

Third, governments could fund alternative institutions to journals. They could put more money into websites like arXiv.org and its equivalents in other fields or fund an alternate review process to vet papers like journal referees do. There are existing institutions they could build on, or they could create their own.

Fourth, you could imagine addressing the problem on the job market side, with universities told not to weigh the prestige of journals when considering candidates. This seems unlikely to happen, and that’s probably a good thing, because it’s very micromanagey. Still, I do think that both grants and jobs could do with less time and effort spent attempting to vet candidates and more explicit randomness.

Fifth, you could imagine governments essentially opting out of the game altogether. They could disallow spending any money from publicly funded grants or universities on open-access fees or subscription fees, pricing most scientists out of the journal system. Journals would either have to radically lower their prices so that scientists could pay for them out of pocket, or more likely go extinct. This does have the problem that if only some countries did it, their scientists would have a harder time in other countries’ job markets. And of course, many critics of journals just want the journals to make less obscene profits, and not actually go extinct.

Most academics I know agree that something is deeply wrong with how academic journals work. While the situation might be solved at the grassroots level, it’s worth imagining what governments might do. Realistically, I don’t expect them to do all that much. But stranger things have gotten political momentum before.

Musing on Application Fees

Leave a reply

A loose rule of thumb: PhD candidates in the US are treated like students. In Europe, they’re treated like employees.

This does exaggerate things a bit. In both Europe and the US, PhD candidates get paid a salary (at least in STEM). In both places, PhD candidates count as university employees, if sometimes officially part-time ones, with at least some of the benefits that entails.

On the other hand, PhD candidates in both places take classes (albeit more classes in the US). Universities charge both for tuition, which is in turn almost always paid by their supervisor’s grants or department, not by them. Both aim for a degree, capped off with a thesis defense.

But there is a difference. And it’s at its most obvious in how applications work.

In Europe, PhD applications are like job applications. You apply to a particular advisor, advertising a particular kind of project. You submit things like a CV, cover letter, and publication list, as well as copies of your previous degrees.

In the US, PhD applications are like applications to a school. You apply to the school, perhaps mentioning an advisor or topic you are interested in. You submit things like essays, test scores, and transcripts. And typically, you have to pay an application fee.

I don’t think I quite appreciated, back when I applied for PhD programs, just how much those fees add up to. With each school charging a fee in the $100 range, and students commonly advised to apply to ten or so schools, applying to PhD programs in the US can quickly get unaffordable for many. Schools do offer fee waivers under certain conditions, but the standards vary from school to school. Most don’t seem to apply to non-Americans, so if you’re considering a US PhD from abroad be aware that just applying can be an expensive thing to do.

Why the fee? I don’t really know. The existence of application fees, by itself, isn’t a US thing. If you want to get a Master’s degree from the University of Copenhagen and you’re coming from outside Europe, you have to pay an application fee of roughly the same size that US schools charge.

Based on that, I’d guess part of the difference is funding. It costs something for a university to process an application, and governments might be willing to cover it for locals (in the case of the Master’s in Copenhagen) or more specifically for locals in need (in the US PhD case). I don’t know whether it makes sense for that cost to be around $100, though.

It’s also an incentive, presumably. Schools don’t want too many applicants, so they attach a fee so only the most dedicated people apply.

Jobs don’t typically have an application fee, and I think it would piss a lot of people off if they did. Some jobs get a lot of applicants, enough that bigger and more well-known companies in some places use AI to filter applications. I have to wonder if US PhD schools are better off in this respect. Does charging a fee mean they have a reasonable number of applications to deal with? Or do they still have to filter through a huge pile, with nothing besides raw numbers to pare things down? (At least, because of the “school model” with test scores, they have some raw numbers to use.)

Overall, coming at this with a “theoretical physicist mentality”, I have to wonder if any of this is necessary. Surely there’s a way to make it easy for students to apply, and just filter them down to the few you want to accept? But the world is of course rarely that simple.

Amplitudes 2024, Continued

Leave a reply

I’ve now had time to look over the rest of the slides from the Amplitudes 2024 conference, so I can say something about Thursday and Friday’s talks.

Thursday was gravity-focused. Zvi Bern’s review talk was actually a review, a tour of the state of the art in using amplitudes techniques to make predictions for gravitational wave physics. Bern emphasized that future experiments will require much more precision: two more orders of magnitude, which in our lingo amounts to two more “loops”. The current state of the art is three loops, but they’ve been hacking away at four, doing things piece by piece in a way that cleverly also yields publications (for example, they can do just the integrals needed for supergravity, which are simpler). Four loops here is the first time that the Feynman diagrams involve Calabi-Yau manifolds, so they will likely need techniques from some of the folks I talked about last week. Once they have four loops, they’ll want to go to five, since that is the level of precision you need to learn something about the material in neutron stars. The talk covered a variety of other developments, some of which were talked about later on Thursday and some of which were only mentioned here.

Of that day’s other speakers, Stefano De Angelis, Lucile Cangemi, Mikhail Ivanov, and Alessandra Buonanno also focused on gravitational waves. De Angelis talked about the subtleties that show up when you try to calculate gravitational waveforms directly with amplitudes methods, showcasing various improvements to the pipeline there. Cangemi talked about a recurring question with its own list of subtleties, namely how the Kerr metric for spinning black holes emerges from the math of amplitudes of spinning particles. Gravitational waves were the focus of only the second half of Ivanov’s talk, where he talked about how amplitudes methods can clear up some of the subtler effects people try to take into account. The first half was about another gravitational application, that of using amplitudes methods to compute the correlations of galaxy structures in the sky, a field where it looks like a lot of progress can be made. Finally, Buonanno gave the kind of talk she’s given a few times at these conferences, a talk that puts these methods in context, explaining how amplitudes results are packaged with other types of calculations into the Effective-One-Body framework which then is more directly used at LIGO. This year’s talk went into more detail about what the predictions are actually used for, which I appreciated. I hadn’t realized that there have been a handful of black hole collisions discovered by other groups from LIGO’s data, a win for open science! Her slides had a nice diagram explaining what data from the gravitational wave is used to infer what black hole properties, quite a bit more organized than the statistical template-matching I was imagining. She explained the logic behind Bern’s statement that gravitational wave telescopes will need two more orders of magnitude, pointing out that that kind of precision is necessary to be sure that something that might appear to be a deviation from Einstein’s theory of gravity is not actually a subtle effect of known physics. Her method typically is adjusted to fit numerical simulations, but she shows that even without that adjustment they now fit the numerics quite well, thanks in part to contributions from amplitudes calculations.

Of the other talks that day, David Kosower’s was the only one that didn’t explicitly involve gravity. Instead, his talk focused on a more general question, namely how to find a well-defined basis of integrals for Feynman diagrams, which turns out to involve some rather subtle mathematics and geometry. This is a topic that my former boss Jake Bourjaily worked on in a different context for some time, and I’m curious whether there is any connection between the two approaches. Oliver Schlotterer gave the day’s second review talk, once again of the “actually a review” kind, covering a variety of recent developments in string theory amplitudes. These include some new pictures of how string theory amplitudes that correspond to Yang-Mills theories “square” to amplitudes involving gravity at higher loops and progress towards going past two loops, the current state of the art for most string amplitude calculations. (For the experts: this does not involve taking the final integral over the moduli space, which is still a big unsolved problem.) He also talked about progress by Sebastian Mizera and collaborators in understanding how the integrals that show up in string theory make sense in the complex plane. This is a problem that people had mostly managed to avoid dealing with because of certain simplifications in the calculations people typically did (no moduli space integration, expansion in the string length), but taking things seriously means confronting it, and Mizera and collaborators found a novel solution to the problem that has already passed a lot of checks. Finally, Tobias Hansen’s talk also related to string theory, specifically in anti-de-Sitter space, where the duality between string theory and N=4 super Yang-Mills lets him and his collaborators do Yang-Mills calculations and see markedly stringy-looking behavior.

Friday began with Kevin Costello, whose not-really-a-review talk dealt with his work with Natalie Paquette showing that one can use an exactly-solvable system to learn something about QCD. This only works for certain rather specific combinations of particles: for example, in order to have three colors of quarks, they need to do the calculation for nine flavors. Still, they managed to do a calculation with this method that had not previously been done with more traditional means, and to me it’s impressive that anything like this works for a theory without supersymmetry. Mina Himwich and Diksha Jain both had talks related to a topic of current interest, “celestial” conformal field theory, a picture that tries to apply ideas from holography in which a theory on the boundary of a space fully describes the interior, to the “boundary” of flat space, infinitely far away. Himwich talked about a symmetry observed in that research program, and how that symmetry can be seen using more normal methods, which also lead to some suggestions of how the idea might be generalized. Jain likewise covered a different approach, one in which one sets artificial boundaries in flat space and sees what happens when those boundaries move.

Yifei He described progress in the modern S-matrix bootstrap approach. Previously, this approach had gotten quite general constraints on amplitudes. She tries to do something more specific, and predict the S-matrix for scattering of pions in the real world. By imposing compatibility with knowledge from low energies and high energies, she was able to find a much more restricted space of consistent S-matrices, and these turn out to actually match pretty well to experimental results. Mathieu Giroux addresses an important question for a variety of parts of amplitudes research, how to predict the singularities of Feynman diagrams. He explored a recursive approach to solving Landau’s equations for these singularities, one which seems impressively powerful, in one case being able to find a solution that in text form is approximately the length of Harry Potter. Finally, Juan Maldacena closed the conference by talking about some progress he’s made towards an old idea, that of defining M theory in terms of a theory involving actual matrices. This is a very challenging thing to do, but he is at least able to tackle the simplest possible case, involving correlations between three observations. This had a known answer, so his work serves mostly as a confirmation that the original idea makes sense at at least this level.

Beyond Elliptic Polylogarithms in Oaxaca

1 Reply

Arguably my biggest project over the last two years wasn’t a scientific paper, a journalistic article, or even a grant application. It was a conference.

Most of the time, when scientists organize a conference, they do it “at home”. Either they host the conference at their own university, or rent out a nearby event venue. There is an alternative, though. Scattered around the world, often in out-of-the way locations, are places dedicated to hosting scientific conferences. These places accept applications each year from scientists arguing that their conference would best serve the place’s scientific mission.

One of these places is the Banff International Research Station in Alberta, Canada. Since 2001, Banff has been hosting gatherings of mathematicians from around the world, letting them focus on their research in an idyllic Canadian ski resort.

If you don’t like skiing, though, Banff still has you covered! They have “affiliate centers” elsewhere, with one elsewhere in Canada, one in China, two on the way in India and Spain…and one, that particularly caught my interest, in Oaxaca, Mexico.

Back around this time of year in 2022, I started putting a proposal together for a conference at the Casa Mathemática Oaxaca. The idea would be a conference discussing the frontier of the field, how to express the strange mathematical functions that live in Feynman diagrams. I assembled a big team of co-organizers, five in total. At the time, I wasn’t sure whether I could find a permanent academic job, so I wanted to make sure there were enough people involved that they could run the conference without me.

Followers of the blog know I did end up finding that permanent job…only to give it up. In the end, I wasn’t able to make it to the conference. But my four co-organizers were (modulo some delays in the Houston airport). The conference was this week, with the last few talks happening over the next few hours.

I gave a short speech via Zoom at the beginning of the conference, a mix of welcome and goodbye. Since then I haven’t had the time to tune in to the talks, but they’re good folks and I suspect they’re having good discussions.

I do regret that, near the end, I wasn’t able to give the conference the focus it deserved. There were people we really hoped to have, but who couldn’t afford the travel. I’d hoped to find a source of funding that could support them, but the plan fell through. The week after Amplitudes 2024 was also a rough time to have a conference in this field, with many people who would have attended not able to go to both. (At least they weren’t the same week, thanks to some flexibility on the part of the Amplitudes organizers!)

Still, it’s nice to see something I’ve been working on for two years finally come to pass, to hopefully stir up conversations between different communities and give various researchers a taste of one of Mexico’s most beautiful places. I still haven’t been to Oaxaca yet, but I suspect I will eventually. Danish companies do give at minimum five weeks of holiday per year, so I should get a chance at some point.

Does Science Require Publication?

7 Replies

Seen on Twitter:

As is traditional, twitter erupted into dumb arguments over this. Some made fun of Yann LeCun for implying that Elon Musk will be forgotten, which despite any other faults of his seems unlikely. Science popularizer Sabine Hossenfelder pointed out that there are two senses of “publish” getting confused here: publish as in “make public” and publish as in “put in a scientific journal”. The latter tends to be necessary for scientists in practice, but is not required in principle. (The way journals work has changed a lot over just the last century!) The former, Sabine argued, is still 100% necessary.

Plenty of people on twitter still disagreed (this always happens). It got me thinking a bit about the role of publication in science.

When we talk about what science requires or doesn’t require, what are we actually talking about?

“Science” is a word, and like any word its meaning is determined by how it is used. Scientists use the word “science” of course, as do schools and governments and journalists. But if we’re getting into arguments about what does or does not count as science, then we’re asking about a philosophical problem, one in which philosophers of science try to understand what counts as science and what doesn’t.

What do philosophers of science want? Many things, but a big one is to explain why science works so well. Over a few centuries, humanity went from understanding the world in terms of familiar materials and living creatures to decomposing them in terms of molecules and atoms and cells and proteins. In doing this, we radically changed what we were capable of, computers out of the reach of blacksmiths and cures for diseases that weren’t even distinguishable. And while other human endeavors have seen some progress over this time (democracy, human rights…), science’s accomplishment demands an explanation.

Part of that explanation, I think, has to include making results public. Alchemists were interested in many of the things later chemists were, and had started to get some valuable insights. But alchemists were fearful of what their knowledge would bring (especially the ones who actually thought they could turn lead into gold). They published almost only in code. As such, the pieces of progress they made didn’t build up, didn’t aggregate, didn’t become overall progress. It was only when a new scientific culture emerged, when natural philosophers and physicists and chemists started writing to each other as clearly as they could, that knowledge began to build on itself.

Some on twitter pointed out the example of the Manhattan project during World War II. A group of scientists got together and made progress on something almost entirely in secret. Does that not count as science?

I’m willing to bite this bullet: I don’t think it does! When the Soviets tried to replicate the bomb, they mostly had to start from scratch, aside from some smuggled atomic secrets. Today, nations trying to build their own bombs know more, but they still must reinvent most of it. We may think this is a good thing, we may not want more countries to make progress in this way. But I don’t think we can deny that it genuinely does slow progress!

At the same time, to contradict myself a bit: I think you can think of science that happens within a particular community. The scientists of the Manhattan project didn’t publish in journals the Soviets could read. But they did write internal reports, they did publish to each other. I don’t think science by its nature has to include the whole of humanity (if it does, then perhaps studying the inside of black holes really is unscientific). You probably can do science sticking to just your own little world. But it will be slower. Better, for progress’s sake, if you can include people from across the world.

Peer Review in Post-scarcity Academia

3 Replies

I posted a link last week to a dialogue written by a former colleague of mine, Sylvain Ribault. Sylvain’s dialogue is a summary of different perspectives on academic publishing. Unlike certain more famous dialogues written by physicists, Sylvain’s account doesn’t have a clear bias: he’s trying to set out the concerns different stakeholders might have and highlight the history of the subject, without endorsing one particular approach as the right one.

The purpose of such a dialogue is to provoke thought, and true to its purpose, the dialogue got me thinking.

Why do peer review? Why do we ask three or so people to read every paper, comment on it, and decide whether it should be published? While one can list many reasons, they seem to fall into two broad groups:

  1. We want to distinguish better science from worse science. We want to reward the better scientists with jobs and grants and tenure. To measure whether scientists are better, we want to see whether they publish more often in the better journals. We then apply those measures on up the chain, funding universities more when they have better scientists, and supporting grant programs that bring about better science.
  2. We want published science to be true. We want to make sure that when a paper is published that the result is actually genuine, free both from deception and from mistakes. We want journalists and the public to know which scientific results are valid, and we want scientists to know what results they can base their own research on.

The first set of goals is a product of scarcity. If we could pay every scientist and fund every scientific project with no cost, we wouldn’t need to worry so much about better and worse science. We’d fund it all and see what happens. The second set of goals is more universal: the whole point of science is to find out the truth, and we want a process that helps to achieve that.

My approach to science is to break problems down. What happens if we had only the second set of concerns, and not the first?

Well, what happens to hobbyists?

I’ve called hobby communities a kind of “post-scarcity academia”. Hobbyists aren’t trying to get jobs doing their hobby or get grants to fund it. They have their day jobs, and research their hobby as a pure passion project. There isn’t much need to rank which hobbyists are “better” than others, but they typically do care about whether what they write is true. So what happens when it’s not?

Sometimes, not much.

My main hobby community was Dungeons and Dragons. In a game with over 50 optional rulebooks covering multiple partially compatible-editions, there were frequent arguments about what the rules actually meant. Some were truly matters of opinion, but some were true misunderstandings, situations where many people thought a rule worked a certain way until they heard the right explanation.

One such rule regarded a certain type of creature called a Warbeast. Warbeasts, like Tolkien’s Oliphaunts, are “upgraded” versions of more normal wild animals, bred and trained for war. There were rules to train a Warbeast, and people interpreted these rules differently: some thought you could find an animal in the wild and train it to become a Warbeast, others thought the rules were for training a creature that was already a Warbeast to fight.

I supported the second interpretation: you can train an existing Warbeast, you can’t train a wild animal to make it into a Warbeast. As such, keep in mind, I’m biased. But every time I explained the reasoning (pointing out that the text was written in the context of an earlier version of the game, and how the numbers in it matched up with that version), people usually agreed with me. And yet, I kept seeing people use the other interpretation. New players would come in asking how to play the game, and get advised to go train wild animals to make them into Warbeasts.

Ok, so suppose the Dungeons and Dragons community had a peer review process. Would that change anything?

Not really! The wrong interpretation was popular. If whoever first proposed it got three random referees, there’s a decent chance none of them would spot the problem. In good science, sometimes the problems with an idea are quite subtle. Referees will spot obvious issues (and not even all of those!), but only the most diligent review (which sometimes happens in mathematics, and pretty much nowhere else) can spot subtle flaws in an argument. For an experiment, you sometimes need more than that: not just a review, but an actual replication.

What would have helped the Dungeons and Dragons community? Not peer review, but citations.

Suppose that, every time someone suggested you could train a wild animal to make it a Warbeast, they had to link to the first post suggesting you could do this. Then I could go to that first post, and try to convince the author that my interpretation was correct. If I succeeded, the author could correct their post, and then every time someone followed one of these citation links it would tell them the claim was wrong.

Academic citations don’t quite work like this. But the idea is out there. People have suggested letting anyone who wants to review a paper, and publishing the reviews next to the piece like comments on a blog post. Sylvain’s dialogue mentions a setup like this, and some of the risks involved.

Still, a setup like that would have gone a long way towards solving the problem for the Dungeons and Dragons community. It has me thinking that something like that is worth exploring.

No Unmoved Movers

2 Replies

Economists must find academics confusing.

When investors put money in a company, they have some control over what that company does. They vote to decide a board, and the board votes to hire a CEO. If the company isn’t doing what the investors want, the board can fire the CEO, or the investors can vote in a new board. Everybody is incentivized to do what the people who gave the money want to happen. And usually, those people want the company to increase its profits, since most of them people are companies with their own investors).

Academics are paid by universities and research centers, funded in the aggregate by governments and student tuition and endowments from donors. But individually, they’re also often funded by grants.

What grant-givers want is more ambiguous. The money comes in big lumps from governments and private foundations, which generally want something vague like “scientific progress”. The actual decision of who gets the money are made by committees made up of senior scientists. These people aren’t experts in every topic, so they have to extrapolate, much as investors have to guess whether a new company will be profitable based on past experience. At their best, they use their deep familiarity with scientific research to judge which projects are most likely to work, and which have the most interesting payoffs. At their weakest, though, they stick with ideas they’ve heard of, things they know work because they’ve seen them work before. That, in a nutshell, is why mainstream research prevails: not because the mainstream wants to suppress alternatives, but because sometimes the only way to guess if something will work is raw familiarity.

(What “works” means is another question. The cynical answers are “publishes papers” or “gets citations”, but that’s a bit unfair: in Europe and the US, most funders know that these numbers don’t tell the whole story. The trivial answer is “achieves what you said it would”, but that can’t be the whole story, because some goals are more pointless than others. You might want the answer to be “benefits humanity”, but that’s almost impossible to judge. So in the end the answer is “sounds like good science”, which is vulnerable to all the fads you can imagine…but is pretty much our only option, regardless.)

So are academics incentivized to do what the grant committees want? Sort of.

Science never goes according to plan. Grant committees are made up of scientists, so they know that. So while many grants have a review process afterwards to see whether you achieved what you planned, they aren’t all that picky about it. If you can tell a good story, you can explain why you moved away from your original proposal. You can say the original idea inspired a new direction, or that it became clear that a new approach was necessary. I’ve done this with an EU grant, and they were fine with it.

Looking at this, you might imagine that an academic who’s a half-capable storyteller could get away with anything they wanted. Propose a fashionable project, work on what you actually care about, and tell a good story afterwards to avoid getting in trouble. As long as you’re not literally embezzling the money (the guy who was paying himself rent out of his visitor funding, for instance), what could go wrong? You get the money without the incentives, you move the scientific world and nobody gets to move you.

It’s not quite that easy, though.

Sabine Hossenfelder told herself she could do something like this. She got grants for fashionable topics she thought were pointless, and told herself she’d spend time on the side on the things she felt were actually important. Eventually, she realized she wasn’t actually doing the important things: the faddish research ended up taking all her time. Not able to get grants doing what she actually cared about (and, in one of those weird temporary European positions that only lasts until you run out of grants), she now has to make a living from her science popularization work.

I can’t speak for Hossenfelder, but I’ve also put some thought into how to choose what to research, about whether I could actually be an unmoved mover. A few things get in the way:

First, applying for grants doesn’t just take storytelling skills, it takes scientific knowledge. Grant committees aren’t experts in everything, but they usually send grants to be reviewed by much more appropriate experts. These experts will check if your grant makes sense. In order to make the grant make sense, you have to know enough about the faddish topic to propose something reasonable. You have to keep up with the fad. You have to spend time reading papers, and talking to people in the faddish subfield. This takes work, but also changes your motivation. If you spend time around people excited by an idea, you’ll either get excited too, or be too drained by the dissonance to get any work done.

Second, you can’t change things that much. You still need a plausible story as to how you got from where you are to where you are going.

Third, you need to be a plausible person to do the work. If the committee looks at your CV and sees that you’ve never actually worked on the faddish topic, they’re more likely to give a grant to someone who’s actually worked on it.

Fourth, you have to choose what to do when you hire people. If you never hire any postdocs or students working on the faddish topic, then it will be very obvious that you aren’t trying to research it. If you do hire them, then you’ll be surrounded by people who actually care about the fad, and want your help to understand how to work with it.

Ultimately, to avoid the grant committee’s incentives, you need a golden tongue and a heart of stone, and even then you’ll need to spend some time working on something you think is pointless.

Even if you don’t apply for grants, even if you have a real permanent position or even tenure, you still feel some of these pressures. You’re still surrounded by people who care about particular things, by students and postdocs who need grants and jobs and fellow professors who are confident the mainstream is the right path forward. It takes a lot of strength, and sometimes cruelty, to avoid bowing to that.

So despite the ambiguous rules and lack of oversight, academics still respond to incentives: they can’t just do whatever they feel like. They aren’t bound by shareholders, they aren’t expected to make a profit. But ultimately, the things that do constrain them, expertise and cognitive load, social pressure and compassion for those they mentor, those can be even stronger.

I suspect that those pressures dominate the private sector as well. My guess is that for all that companies think of themselves as trying to maximize profits, the all-too-human motivations we share are more powerful than any corporate governance structure or org chart. But I don’t know yet. Likely, I’ll find out soon.

Making More Nails

1 Reply

They say when all you have is a hammer, everything looks like a nail.

Academics are a bit smarter than that. Confidently predict a world of nails, and you fall to the first paper that shows evidence of a screw. There are limits to how long you can delude yourself when your job is supposed to be all about finding the truth.

You can make your own nails, though.

Suppose there’s something you’re really good at. Maybe, like many of my past colleagues, you can do particle physics calculations faster than anyone else, even when the particles are super-complicated hypothetical gravitons. Maybe you know more than anyone else about how to make a quantum computer, or maybe you just know how to build a “quantum computer“. Maybe you’re an expert in esoteric mathematics, who can re-phrase anything in terms of the arcane language of category theory.

That’s your hammer. Get good enough with it, and anyone with a nail-based problem will come to you to solve it. If nails are trendy, then you’ll impress grant committees and hiring committees, and your students will too.

When nails aren’t trendy, though, you need to try something else. If your job is secure, and you don’t have students with their own insecure jobs banging down your door, then you could spend a while retraining. You could form a reading group, pick up a textbook or two about screwdrivers and wrenches, and learn how to use different tools. Eventually, you might find a screwdriving task you have an advantage with, something you can once again do better than everyone else, and you’ll start getting all those rewards again.

Or, maybe you won’t. You’ll get less funding to hire people, so you’ll do less research, so your work will get less impressive and you’ll get less funding, and so on and so forth.

Instead of risking that, most academics take another path. They take what they’re good at, and invent new problems in the new trendy area to use that expertise.

If everyone is excited about gravitational waves, you turn a black hole calculation into a graviton calculation. If companies are investing in computation in the here-and-now, then you find ways those companies can use insights from your quantum research. If everyone wants to know how AI works, you build a mathematical picture that sort of looks like one part of how AI works, and do category theory to it.

At first, you won’t be competitive. Your hammer isn’t going to work nearly as well as the screwdrivers people have been using forever for these problems, and there will be all sorts of new issues you have to solve just to get your hammer in position in the first place. But that doesn’t matter so much, as long as you’re honest. Academic research is expected to take time, applications aren’t supposed to be obvious. Grant committees care about what you’re trying to do, as long as you have a reasonably plausible story about how you’ll get there.

(Investors are also not immune to a nice story. Customers are also not immune to a nice story. You can take this farther than you might think.)

So, unlike the re-trainers, you survive. And some of the time, you make it work. Your hammer-based screwdriving ends up morphing into something that, some of the time, actually does something the screwdrivers can’t. Instead of delusionally imagining nails, you’ve added a real ersatz nail to the world, where previously there was just a screw.

Making nails is a better path for you. Is it a better path for the world? I’m not sure.

If all those grants you won, all those jobs you and your students got, all that money from investors or customers drawn in by a good story, if that all went to the people who had the screwdrivers in the first place, could they have done a better job?

Sometimes, no. Sometimes you happen upon some real irreproducible magic. Your hammer is Thor’s hammer, and when hefted by the worthy it can do great things.

Sometimes, though, your hammer was just the hammer that got the funding. Now every screwdriver kit has to have a space for a little hammer, when it could have had another specialized screwdriver that fit better in the box.

In the end, the world is build out of these kinds of ill-fitting toolkits. We all try to survive, both as human beings and by our sub-culture’s concept of the good life. We each have our hammers, and regardless of whether the world is full of screws, we have to convince people they want a hammer anyway. Everything we do is built on a vast rickety pile of consequences, the end-results of billions of people desperate to be wanted. For those of us who love clean solutions and ideal paths, this is maddening and frustrating and terrifying. But it’s life, and in a world where we never know the ideal path, screw-nails and nail-screws are the best way we’ve found to get things done.