In academia, scientists don’t always work with what they actually care about. A lot of the time, they use what academics call toy models. A toy model can be a theory with simpler mathematics than the theories that describe the real world, but it can also be something that is itself real, just simpler or easier to work with, like nematodes, fruit flies, or college students.
Some people in industry seem to think this is all academics do. I’ve seen a few job ads that emphasize experience dealing with “real-world data”, and a few people skeptical that someone used to academia would be able to deal with the messy challenges of the business world.
There’s a grain of truth to this, but I don’t think industry has a monopoly on mess. To see why, let’s think about how academics write computer code.
There are a lot of things that one is in-principle supposed to do to code well, and most academics do none of them. Good code has test suites, so that if you change something you can check whether it still works by testing it on all the things that could go wrong. Good code is modular, with functions doing specific things and re-used whenever appropriate. Good code follows shared conventions, so that others can pick up your code and understand how you did it.
Some academics do these things, for example those who build numerical simulations on supercomputers. But for most academics, coding best-practices range from impractical to outright counterproductive. Testing is perhaps the clearest example. To design a test suite, you have to have some idea what kinds of things your code will run into, what kind of input you expect what the output is supposed to be. Many academic projects, though, are the first of their kind. Academics code up something to do a calculation nobody has done before, not knowing the result, or they make code to analyze a dataset nobody has worked with before. By the time they understand the problem well enough to write a test suite, they’ve already solved the problem, and they’re on to the next project, which may need something totally different.
From the perspective of these academics, if you have a problem well-defined enough that you can build a test suite, well enough that you can have stable conventions and reusable functions…then you have a toy model, not a real problem from the real world.
…and of course, that’s not quite fair either, right?
The truth is, academics and businesspeople want to work with toy models. Toy models are well-behaved, and easy, and you can do a lot with them. The real world isn’t a toy model…but it can be, if you make it one.
This means planning your experiments, whether in business or in science. It means making sure the data you gather is labeled and organized before you begin. It means coming up with processes, and procedures, and making as much of the work as possible a standardized, replicable thing. That’s desirable regardless, whether you’re making a consistent product instead of artisanal one-offs or a well-documented scientific study that another team can replicate.
Academia and industry both must handle mess. They handle different kinds of mess in different circumstances, and manage it in different ways, and this can be a real challenge for someone trying to go from one world to another. But neither world is intrinsically messier or cleaner. Nobody has a monopoly on toy models.
4 gravitons 
“From the perspective of these academics, if you have a problem well-defined enough that you can build a test suite, well enough that you can have stable conventions and reusable functions…then you have a toy model, not a real problem from the real world.”
Hmm, I don’t agree with this. Definitely there exists bespoke code is written for a particular project, never meant to see any other users, and then discarded when moving on to the next thing. But there are also codes that are used for collider event simulation or analysis that are extremely modular, have dozens (hundreds?) of test suites, and are definitely based on a real problem from the real world. The paragon examples in particle physics are Pythia (for leading-order collider event simulation and parton showering), MadGraph (for complex fixed-order event generation at leading or next-to-leading order), and FastJet (for event analysis and in particular jet clustering). These codes are used by the majority of particle physicists (other than GEANT, Pythia is the most highly-cited program in ATLAS and CMS papers) and run extremely efficiently on a laptop.
LikeLike
Yeah, I mentioned numerical simulation, but of course sufficiently reliable analytics à la MadGraph falls in the same bucket, as do a lot of data analysis codes on the experimental side. When the science gets “big” enough, one has to standardize methods to get anything done at all, and that requires many of the same coding best-practices as industry.
At the same time, I do think it’s fair to say that part of the reason any of that works is because the problem has been intentionally squished into a toy model-esque context. We’re talking about collider physics, to begin with, where our collision is happening at the middle of a giant pile of calorimeters that we’ve spent a huge amount of time testing and modeling, where there are widely separated scales so that there is a well-defined “hard part” of the process that we can treat with the toy model that is the Minkowski space S-matrix.
And yeah, getting to that point involves a huge amount of work, both experimental and theoretical, and a lot of that work is about handling messes. Which seems roughly comparable to how this works in industry too: to go from the real world of people’s complicated desires and the extremely varied things that you could make for them to a standardized product with a test suite requires a huge amount of work, and shouldn’t be underestimated, but the beneficiaries of that work are still going to end up unused to other kinds of open-ended problems.
LikeLike
Fair point, definitely agree with this. I guess as another commenter mentioned, I think there is rather a distinction between “toy model” and “systematically improvable approximation”. In my opinion, N=4 SYM is a toy model, full stop, that does not describe the universe, but nevertheless can lead to insights into real problems. By contrast, we may never solve QCD, but I wouldn’t call leading-order perturbation theory a “toy model”. We know precisely how to improve the description of QCD in perturbation theory, and then it just becomes a problem with how strong we are. If we know absolutely everything in N=4, the complete, non-perturbative path integral say, there’s not a systematic way to improve that description to account for QCD such that subsequent terms in the expansion are indeed smaller and smaller in a controlled way.
On the other hand, I take your point well, and have myself used several codes in research that are absolute state-of-the-art but copies were only passed down from grad student to grad student. Perhaps the best example of that is EVENT2 for NLO evaluation of jets in lepton colliders, which has minimal documentation and written in Fortran, but is by far the most efficient tool for this.
LikeLike
Yeah, that’s an important distinction! I guess what I’d say is that “happens inside a collider” is itself a kind of a toy model. You can improve on it in that things you see happening inside a collider let you fix parameters in the SM, which describes the rest of the world, but it’s not a systematic approximation like LO vs NLO is, you can’t go “next-to-collider” and “next-to-next-to-collider”. It’s actually a lot more like N=4, where you gain insights working in that simpler context (calculate some basis integrals, for example in the N=4 case) which help you in the non-toy model case.
LikeLike
Hmm, interesting. I mean, I guess the “inside a collider” is a toy model, but the approximation framework for understanding it, the S-matrix or Fermi’s Golden Rule or the like, has a well-defined regime of applicability. Relately, factorization theorems that ensure that collisions can be described by parton distributions and hard matrix elements are accurate up to non-perturbative power corrections. In all these cases, we don’t have a good understanding of how to actually do the expansion and calculate higher-order corrections systematically, but we can nevertheless assign an uncertainty to the known unknowns. But I also suppose that at this point one is splitting hairs over what precisely could be a “toy model”.
LikeLike
Great example of coding in academia. I have stood behind this line all my programming life – “By the time they understand the problem well enough to write a test suite, they’ve already solved the problem, ⋯”. It also being the reason why most of my code is uncommented, leading to impracticality AND outright counterproductivity after 6 months. 😅
LikeLike
When we don’t know, will most likely never know and probably can’t know if the universe we live in isn’t somebody’s toy model I find them principledly dubious. At the most I’ll give them double-edged sword status.
Let’s look at it through an example. We want to figure out moving of an object down the slope when both are in a gravitational well. So we start with ideal slope and an ideal object shaped in mathematically ideal way. We draw immutable forces and their component vectors and get a static picture which we can then translate into an equation. Or we attack the problem from energy point of view in much the same way.
Then we add materials for both, labor over friction; idealising it of course, then add idealised medium in which the system resides and so on. We are basically doing path integral except instead of probabilities we use contributions. Each ‘improvement’ changes the result less and less and at some point we call it quits. At least until next post-grad figures out it could make decent PhD thesis and adds another. I call the process scientific accounting. Most call it progress. And yes, if the right metric is chosen, we definitely progressed.
Was the whole endeavor useful? Absolutely. Those equations come handy in countless situations. Good job. Did we figure out the bloody thing? Not really. We’ve made elaborate approximation and while in principle there’s nothing wrong with approximations, we have this evolutionary predisposition to take an approximation and while fully knowing it is such promote it to ‘the truth’. Not even going into how we then often go and fight to death on that hill.
Ye remember how your ‘1 over n when n goes to zero’ changed during your education? How many had their first explanation stated as ‘undefined’? Preferably with accompanying definition of mathematical meaning of undefined? Now maybe starting it basically wrong but simple and then steering it towards correct only for all those interested is the better pedagogical approach. But that leaves significant portion of populous at the ‘wrong’ stage doesn’t it?
LikeLike