It Only Counts When AI Gets to My Field

It’s a meme at this point.

When Deep Blue beat Kasparov, Go players could say that their game, unlike Chess, was too complex to fall to a computer program. Then AlphaGo showed they were wrong. It just took a better approach.

When AlphaFold leaped ahead of human experts in predicting how proteins fold, it was due to a mountain of carefully labeled protein structure data. Other scientists and mathematicians could argue that nothing like that existed in their field, so a similar success was unlikely. But LLMs can now navigate scientific literature, and loosely imitate the reasoning process of a mathematical proof. And increasingly, the math and computer science results coming out of AI labs are ones that humans find impressive.

Now, experts argue about how far AI can really go. Will AI mathematics only be good at finding counterexamples and solving cute puzzles, not introducing new concepts and frameworks? Will AI only be meaningfully good at fields like mathematics with clear rules and carefully collected conjectures, not fuzzier fields like physics? Will AI-powered labs only manage to optimize specific procedures, and not carry out entire experimental programs? Each time, the pattern seems to be that scholars are skeptical, until AI gets to their field.

I’m aware of this pattern. But I’m willing to take the risk.

I think my old field, scattering amplitudes, is special. And when AI can do something meaningful there, I’ll really start to worry.

By “something meaningful”, I don’t mean the student-level results that have come out so far. I mean tackling some of the field’s big outstanding problems: determining whether N=8 supergravity diverges at seven loops, or finding the six-particle amplitude in N=4 super Yang-Mills to nine loops. Getting another loop past the state of the art for gravitational wave physics or collider physics would also count.

These problems are difficult not just because people haven’t had the right ideas, but because they’re hard in a computational sense. Each loop, a rough measure of the precision of the end result, represents an increase in complexity, in calculations that typically scale exponentially or even factorially in the number of loops. In principle, amplitudes researchers could do any of these with no new ideas, just using known methods. They’d just need access to a lot more computing power.

See, while everyone else is preoccupied with whether AI can come up with genuinely new ideas, I think the real measure is what those ideas accomplish. And the most important measure of accomplishment, if you’re worried about how scared to be about AI, is whether it can do things that seem like they would take too much computing power.

In the past, when people dreamed up the scariest hypothetical things AI could achieve, critics argued they were impossible due to a lack of computing power. Apocalypse scenarios often involve designing self-replicating nanobots based on computer models of molecules, or unstoppable social manipulation based on simulating the minds of the humans the AI interacts with. If AI is going to manage these things, or something like them, it will take an approach that somehow bypasses that need for more computers than we can build.

So if AI companies want to impress people like me (or scare us, for that matter), then they need to tackle my old field. Show that an AI can take the kinds of computer resources an academic has access to, and solve one of the scattering amplitudes field’s big outstanding problems. Show that a computational limit everyone expected to be a problem doesn’t actually matter. Give us N=8 supergravity to seven loops, or N=4 super Yang-Mills to nine loops.

Leave a comment! If it's your first time, it will go into moderation.