An AI Apocalypse, but Just for You

For Halloween, I like to talk about spooky scenarios. And these days, the most trendy one is an AI apocalypse.

You can run into a wild range of ideas for how AI could go wrong, of varying plausibility. They go from hacks shutting down key infrastructure or engineered viruses, to taking over nuclear arsenals and sending out armies of killer robots, all the way to nanotech that devours the world and flawlessly manipulative plans that end up filling space with computers simulating hell at the speed of light.

And of course, on the other end, the possibility that banks suddenly become convinced that the investments underpinning AI companies’ trillion dollar evaluations aren’t going to pay, collapsing the economy.

So, lots of things to be scared of this year!

But in all of those scenarios, at least we suffer together. If civilization cracks like an egg, everyone who survives has to adapt and figure out a new path forward. And if everyone’s dead, well, everyone’s dead. There’s something oddly comforting about that.

That’s not guaranteed, though.

I occasionally see quotes from “100+ reactions to 100+ solutions”, a compilation of thoughts by mathematicians about OpenAI’s recent claim to have solved hundreds of difficult open problems in mathematics. For the most part, these people aren’t writing about an AI apocalypse that sweeps civilization into a new form. They’re writing about what AI seems to be capable of in mathematics, right now, and wondering how they personally will withstand it.

Picture a world where you’ve defined yourself around something. Maybe you’re competitive, someone who wants to be the best in the world. Maybe you just want something of your own, accomplishments that came from you, not anyone else. Either way, it’s something you love, something you chose a strange and difficult path to pursue.

Then, that identity falls apart. For the competitive, you’re not the best anymore, not at the kind of thing you were doing, you’ll never be the best at that again. For the rest, your sense of ownership is less secure. Someone could hear about that problem you’re tinkering with, decide to tackle it on a whim, and jump so far ahead that you’d never be able to catch up on your own merits.

And if there isn’t a wider collapse, you’re facing this on your own. You might have to find a new job, though in a world where universities still need math classes that seems unlikely. But you will have to find a new purpose. You’d have to become a different kind of mathematician, at minimum, one more dedicated to something the AI can’t do and less dedicated to the things that happened to define your style of mathematics. Your friends in other fields will never quite understand. Your friends outside of academia will understand even less.

If we thread the needle between the apocalypses of powerful AI and an economic bubble that pops so hard that companies can’t host the models anymore, we end up with a series of quieter, more personal AI apocalypses, people who feel like their own world is ending, and have to figure out how to go on. For now, that’s the possibility many mathematicians are considering. We’ll see how many others consider it in the years to come.

1 thought on “An AI Apocalypse, but Just for You”

  1. James Cross's avatarJames Cross

    “The first thing we do, let’s kill all the lawyers” from William Shakespeare’s Henry VI, Part 2.

    So, the first thing AI does is kill all the mathematicians.

    How devious could that be?

    Like

    Reply ↓

Leave a comment! If it's your first time, it will go into moderation.