The blog Slate Star Codex had an interesting post last month, titled Socratic Grilling. The post started with a dialogue, a student arguing with a teacher about germ theory.

Student: Hey, wait. If germs are spread from person to person on touch, why doesn’t the government just mandate one week when nobody is allowed to touch anyone else? Then all the germs will die and we’ll never have to worry about germs again.

Out of context, the student looks like a crackpot. But in context, the student is just trying to learn, practicing a more aggressive version of Socratic questioning which the post dubbed “Socratic grilling”.

The post argued that Socratic grilling is normal and unavoidable, and that experts treat it with far more hostility than they should. Experts often reject this kind of questioning as arrogant, unless the non-expert doing the grilling is hilariously deferential. (The post’s example: “I know I am but a mere student, and nowhere near smart enough to actually challenge you, so I’m sure I’m just misunderstanding this, but the thing you just said seems really confusing to me, and I’m not saying it’s not true, but I can’t figure out how it possibly could be true, which is my fault and not yours, but could you please try to explain it differently?”)

The post made me think a bit about my own relationship with crackpots. I’d like to say that when a non-expert challenges me I listen to them regardless of their tone, that you don’t need to be so deferential around me. In practice, though…well, it certainly helps.

What I want (or at least what I want to want) is not humility, but intellectual humility. You shouldn’t have to talk about how inexperienced you are to get me to listen to you. But you should make clear what you know, how you know it, and what the limits of that evidence are. If I’m right, it helps me understand what you’re misunderstanding. If you’re right, it helps me get why your argument works.

I’ve referred to both non-experts and crackpots in this post. To be clear, I think of one as a subgroup of the other. When I refer to crackpots, I’m thinking of a specific sort of non-expert: one with a very detailed idea they have invested a lot of time and passion into, which the mainstream considers impossible. If you’re just skeptical of general relativity or quantum mechanics, you’re not a crackpot. But if you’ve come up with your own replacement to general relativity or quantum mechanics, you probably are. Note also that, no matter how dumb their ideas, I don’t think of experts in a topic as crackpots on that topic. Garrett Lisi is silly, and probably wrong, but he’s not a crackpot.

A result of this is that crackpots (as I define them) rarely do actual Socratic grilling. For a non-expert who hasn’t developed their own theory, Socratic grilling can be a good way to figure out what the heck those experts are thinking. But for a crackpot, the work they have invested in their ideas means they’re often much less interested in what the experts have to say.

This isn’t always the case. I’ve had some perfectly nice conversations with crackpots. I remember an email exchange with a guy who had drawn what he thought were Feynman diagrams without really knowing what they were, and wanted me to calculate them. While I quit that conversation out of frustration, it was my fault, not his.

Sometimes, though, it’s clear from the tactics that someone isn’t trying to learn. There’s a guy who has tried to post variations of the same comment on this blog sixteen times. He picks a post that mentions math, and uses that as an excuse to bring up his formula for the Hubble constant (“you think you’re so good at math, then explain this!”). He says absolutely nothing about the actual post, and concludes by mentioning that his book is available on Kindle.

It’s pretty clear that spammers like that aren’t trying to learn. They aren’t doing Socratic grilling, they’re just trying (and failing) to get people to buy their book.

It’s less clear how to distinguish Socratic grilling from trolling. Sometimes, someone asks an aggressive series of questions because they think you’re wrong, and want to clarify why. Sometimes, though, someone asks an aggressive series of questions because they want to annoy you.

How can you tell if someone is just trolling? Inconsistency is one way. A Socratic grill-er will have a specific position in mind, even if you can’t quite tell what it is. A troll will say whatever they need to to keep arguing. If it becomes clear that there isn’t any consistent picture behind what the other person is saying, they’re probably just a troll.

In the end, no-one is a perfect teacher. If you aren’t making headway explaining something, if an argument just keeps going in circles, then you probably shouldn’t continue. You may be dealing with a troll, or it might just be honest Socratic grilling, but either way it doesn’t matter: if you’re stuck, you’re stuck, and it’s more productive to back off than to get in a screaming match.

That’s been my philosophy anyway. I engage with Socratic grilling as long as it’s productive, whether or not you’re a crackpot. But if you spam, I’ll block your comments, while if I think you’re trolling or not listening I’ll just stop responding. It’s not worth my time at that point, and it’s not worth yours either.