Tag Archives: PublicPerception

Adversarial Collaborations for Physics

Sometimes physics debates get ugly. For the scientists reading this, imagine your worst opponents. Think of the people who always misinterpret your work while using shoddy arguments to prop up their own, where every question at a talk becomes a screaming match until you just stop going to the same conferences at all.

Now, imagine writing a paper with those people.

Adversarial collaborations, subject of a recent a contest on the blog Slate Star Codex, are a proposed method for resolving scientific debates. Two scientists on opposite sides of an argument commit to writing a paper together, describing the overall state of knowledge on the topic. For the paper to get published, both sides have to sign off on it: they both have to agree that everything in the paper is true. This prevents either side from cheating, or from coming back later with made-up objections: if a point in the paper is wrong, one side or the other is bound to catch it.

This won’t work for the most vicious debates, when one (or both) sides isn’t interested in common ground. But for some ongoing debates in physics, I think this approach could actually help.

One advantage of adversarial collaborations is in preventing accusations of bias. The debate between dark matter and MOND-like proposals is filled with these kinds of accusations: claims that one group or another is ignoring important data, being dishonest about the parameters they need to fit, or applying standards of proof they would never require of their own pet theory. Adversarial collaboration prevents these kinds of accusations: whatever comes out of an adversarial collaboration, both sides would make sure the other side didn’t bias it.

Another advantage of adversarial collaborations is that they make it much harder for one side to move the goalposts, or to accuse the other side of moving the goalposts. From the sidelines, one thing that frustrates me watching string theorists debate whether the theory can describe de Sitter space is that they rarely articulate what it would take to decisively show that a particular model gives rise to de Sitter. Any conclusion of an adversarial collaboration between de Sitter skeptics and optimists would at least guarantee that both parties agreed on the criteria. Similarly, I get the impression that many debates about interpretations of quantum mechanics are bogged down by one side claiming they’ve closed off a loophole with a new experiment, only for the other to claim it wasn’t the loophole they were actually using, something that could be avoided if both sides were involved in the experiment from the beginning.

It’s possible, even likely, that no-one will try adversarial collaboration for these debates. Even if they did, it’s quite possible the collaborations wouldn’t be able to agree on anything! Still, I have to hope that someone takes the plunge and tries writing a paper with their enemies. At minimum, it’ll be an interesting read!

Journalists Need to Adapt to Preprints, Not Ignore Them

Nature has an article making the rounds this week, decrying the dangers of preprints.

On the surface, this is a bit like an article by foxes decrying the dangers of henhouses. There’s a pretty big conflict of interest when a journal like Nature, that makes huge amounts of money out of research scientists would be happy to publish for free, gets snippy about scientists sharing their work elsewhere. I was expecting an article about how “important” the peer review process is, how we can’t just “let anyone” publish, and the like.

Instead, I was pleasantly surprised. The article is about a real challenge, the weakening of journalistic embargoes. While this is still a problem I think journalists can think their way around, it’s a bit subtler than the usual argument.

For the record, peer review is usually presented as much more important than it actually is. When a scientific article gets submitted to a journal, it gets sent to two or three experts in the field for comment. In the best cases, these experts read the paper carefully and send criticism back. They don’t replicate the experiments, they don’t even (except for a few heroic souls) reproduce the calculations. That kind of careful reading is important, but it’s hardly unique: it’s something scientists do on their own when they want to build off of someone else’s paper, and it’s what good journalists get when they send a paper to experts for comments before writing an article. If peer review in a journal is important, it’s to ensure that this careful reading happens at least once, a sort of minimal evidence that the paper is good enough to appear on a scientist’s CV.

The Nature article points out that peer review serves another purpose, specifically one of delay. While a journal is preparing to publish an article they can send it out to journalists, after making them sign an agreement (an embargo) that they won’t tell the public until the journal publishes. This gives the journalists a bit of lead time, so the more responsible ones can research and fact-check before publishing.

Open-access preprints cut out the lead time. If the paper just appears online with no warning and no embargoes, journalists can write about it immediately. The unethical journalists can skip fact-checking and publish first, and the ethical ones have to follow soon after, or risk publishing “old news”. Nobody gets the time to properly vet, or understand, a new paper.

There’s a simple solution I’ve seen from a few folks on Twitter: “Don’t be an unethical journalist!” That doesn’t actually solve the problem though. The question is, if you’re an ethical journalist, but other people are unethical journalists, what do you do?

Apparently, what some ethical journalists do is to carry on as if preprints didn’t exist. The Nature article describes journalists who, after a preprint has been covered extensively by others, wait until a journal publishes it and then cover it as if nothing had happened. The article frames this as virtuous, but doomed: journalists sticking to their ethics even if it means publishing “old news”.

To be 100% clear here, this is not virtuous. If you present a paper’s publication in a journal as news, when it was already released as a preprint, you are actively misleading the public. I can’t count the number of times I’ve gotten messages from readers, confused because they saw a scientific result covered again months later and thought it was new. It leads to a sort of mental “double-counting”, where the public assumes that the scientific result was found twice, and therefore that it’s more solid. Unless the publication itself is unexpected (something that wasn’t expected to pass peer review, or something controversial like Mochizuki’s proof of the ABC conjecture) mere publication in a journal of an already-public result is not news.

What science journalists need to do here is to step back, and think about how their colleagues cover stories. Current events these days don’t have embargoes, they aren’t fed through carefully managed press releases. There’s a flurry of initial coverage, and it gets things wrong and misses details and misleads people, because science isn’t the only field that’s complicated, real life is complicated. Journalists have adapted to this schedule, mostly, by specializing. Some journalists and news outlets cover breaking news as it happens, others cover it later with more in-depth analysis. Crucially, the latter journalists don’t present the topic as new. They write explicitly in the light of previous news, as a response to existing discussion. That way, the public isn’t misled, and their existing misunderstandings can be corrected.

The Nature article brings up public health, and other topics where misunderstandings can do lasting damage, as areas where embargoes are useful. While I agree, I would hope many of these areas would figure out embargoes on their own. My field certainly does: the big results of scientific collaborations aren’t just put online as preprints, they’re released only after the collaboration sets up its own journalistic embargoes, and prepares its own press releases. In a world of preprints, this sort of practice needs to happen for important controversial public health and environmental results as well. Unethical scientists might still release too fast, to keep journalists from fact-checking, but they could do that anyway, without preprints. You don’t need a preprint to call a journalist on the phone and claim you cured cancer.

As open-access preprints become the norm, journalists will have to adapt. I’m confident they will be able to, but only if they stop treating science journalism as unique, and start treating it as news. Science journalism isn’t teaching, you’re not just passing down facts someone else has vetted. You’re asking the same questions as any other journalist: who did what? And what really happened? If you can do that, preprints shouldn’t be scary.

Citations Are Reblogs

Last week we had a seminar from Nadav Drukker, a physicist who commemorates his papers with pottery.

At the speaker dinner we got to chatting about physics outreach, and one of my colleagues told an amusing story. He was explaining the idea of citations to someone at a party, and the other person latched on to the idea of citations as “likes” on Facebook. She was then shocked when he told her that a typical paper of his got around fifty citations.

“Only fifty likes???”

Ok, clearly the metaphor of citations as “likes” is more than a little silly. Liking a post is easy and quick, while citing a paper requires a full paper of your own. Obviously, citations are not “likes”.

No, citations are reblogs.

Citations are someone engaging with your paper, your “post” in this metaphor, and building on it, making it part of their own work. That’s much closer to a “reblog” (or in Facebook terms a “share”) than a “like”. More specifically, it’s a “reblog-with-commentary”, taking someone’s content and adding your own, in a way that acknowledges where the original idea came from. And while fifty “likes” on a post may seem low, fifty reblogs with commentary (not just “LOL SMH”, but actual discussion) is pretty reasonable.

The average person doesn’t know much about academia, but there are a lot of academia-like communities out there. People who’ve never written a paper might know what it’s like to use characters from someone else’s fanfiction, or sew a quilt based on a friend’s pattern. Small communities of creative people aren’t so different from each other, whether they’re writers or gamers or scientists. Each group has traditions of building on each other’s work, acknowledging where your inspiration came from, and using that to build standing in the community. Citations happen to be ours.

Seeing the Wires in Science Communication

Recently, I’ve been going to Science and Cocktails, a series of popular science lectures in Freetown Christiania. The atmosphere is great fun, but I’ve been finding the lectures themselves a bit underwhelming. It’s mostly my fault, though.

There’s a problem, common to all types of performing artists. Once you know the tricks that make a performance work, you can’t un-see them. Do enough theater and you can’t help but notice how an actor interprets their lines, or how they handle Shakespeare’s dirty jokes. Play an instrument, and you think about how they made that sound, or when they pause for breath. Work on action movies, and you start to see the wires.

This has been happening to me with science communication. Going to the Science and Cocktails lectures, I keep seeing the tricks the speaker used to make the presentation easier. I notice the slides that were probably copied from the speaker’s colloquiums, sometimes without adapting them to the new audience. I notice when an example doesn’t really fit the narrative, but is wedged in there anyway because the speaker wants to talk about it. I notice filler, like a recent speaker who spent several slides on the history of electron microscopes, starting with Hooke!

I’m not claiming I’m a better speaker than these people. The truth is, I notice these tricks because I’ve been guilty of them myself! I reuse slides, I insert pet topics, I’ve had talks that were too short until I added a long historical section.

And overall, it doesn’t seem to matter. The audience doesn’t notice our little shortcuts, just like they didn’t notice the wires in old kung-fu movies. They’re there for the magic of the performance, they want to be swept away by a good story.

I need to reconnect with that. It’s still important to avoid using blatant tricks, to cover up the wires and make things that much more seamless. But in the end, what matters is whether the audience learned something, and whether they had a good time. I need to watch not just the tricks, but the magic: what makes the audience’s eyes light up, what makes them laugh, what makes them think. I need to stop griping about the wires, and start seeing the action.

By Any Other Author Would Smell as Sweet

I was chatting with someone about this paper (which probably deserves a post in its own right, once I figure out an angle that isn’t just me geeking out about how much I could do with their new setup), and I referred to it as “Claude’s paper”. This got me chided a bit: the paper has five authors, experts on Feynman diagrams and elliptic integrals. It’s not just “Claude’s paper”. So why do I think of it that way?

Part of it, I think, comes from the experience of reading a paper. We want to think of a paper as a speech act: someone talking to us, explaining something, leading us through a calculation. Our brain models that as a conversation with a single person, so we naturally try to put a single face to a paper. With a collaborative paper, this is almost never how it was written: different sections are usually written by different people, who then edit each other’s work. But unless you know the collaborators well, you aren’t going to know who wrote which section, so it’s easier to just picture one author for the whole thing.

Another element comes from how I think about the field. Just as it’s easier to think of a paper as the speech of one person, it’s easier to think of new developments as continuations of a story. I at least tend to think about the field in terms of specific programs: these people worked on this, which is a continuation of that. You can follow those kinds of threads though the field, but in reality they’re tangled together: collaborations are an opportunity for two programs to meet. In other fields you might have a “first author” to default to, but in theoretical physics we normally write authors alphabetically. For “Claude’s paper”, it just feels like the sort of thing I’d expect Claude Duhr to write, like a continuation of the other things he’s known for, even if it couldn’t have existed without the other four authors.

You’d worry that associating papers with people like this takes away deserved credit. I don’t think it’s quite that simple, though. In an older post I described this paper as the work of Anastasia Volovich and Mark Spradlin. On some level, that’s still how I think about it. Nevertheless, when I heard that Cristian Vergu was going to be at the Niels Bohr Institute next year, I was excited: we’re hiring one of the authors of GSVV! Even if I don’t think of him immediately when I think of the paper, I think of the paper when I think of him.

That, I think, is more important for credit. If you’re a hiring committee, you’ll start out by seeing names of applicants. It’s important, at that point, that you know what they did, that the authors of important papers stand out, that you assign credit where it’s due. It’s less necessary on the other end, when you’re reading a paper and casually classify it in your head.

Nevertheless, I should be more careful about credit. It’s important to remember that “Claude Duhr’s paper” is also “Johannes Broedel’s paper” and “Falko Dulat’s paper”, “Brenda Penante’s paper” and “Lorenzo Tancredi’s paper”. It gives me more of an appreciation of where it comes from, so I can get back to having fun applying it.

We Didn’t Deserve Hawking

I don’t usually do obituaries. I didn’t do one when Joseph Polchinksi died, though his textbook is sitting an arm’s reach from me right now. I never collaborated with Polchinski, I never met him, and others were much better at telling his story.

I never met Stephen Hawking, either. When I was at Perimeter, I’d often get asked if I had. Visitors would see his name on the Perimeter website, and I’d have to disappoint them by explaining that he hadn’t visited the institute in quite some time. His health, while exceptional for a septuagenarian with ALS, wasn’t up to the travel.

Was his work especially relevant to mine? Only because of its relevance to everyone who does gravitational physics. The universality of singularities in general relativity, black hole thermodynamics and Hawking radiation, these sharpened the questions around quantum gravity. Without his work, string theory wouldn’t have tried to answer the questions Hawking posed, and it wouldn’t have become the field it is today.

Hawking was unique, though, not necessarily because of his work, but because of his recognizability. Those visitors to Perimeter were a cross-section of the Canadian public. Some of them didn’t know the name of the speaker for the lecture they came to see. Some, arriving after reading Lee Smolin’s book, could only refer to him as “that older fellow who thinks about quantum gravity”. But Hawking? They knew Hawking. Without exception, they knew Hawking.

Who was the last physicist the public knew, like that? Feynman, at the height of his popularity, might have been close. You’d have to go back to Einstein to find someone who was really solidly known like that, who you could mention in homes across the world and expect recognition. And who else has that kind of status? Bohr might have it in Denmark. Go further back, and you’ll find people know Newton, they know Gaileo.

Einstein changed our picture of space and time irrevocably. Newton invented physics as we know it. Galileo and Copernicus pointed up to the sky and shouted that the Earth moves!

Hawking asked questions. He told us what did and didn’t make sense, he showed what we had to take into account. He laid the rules of engagement, and the rest of quantum gravity came and asked alongside him.

We live in an age of questions now. We’re starting to glimpse the answers, we have candidates and frameworks and tools, and if we’re feeling very optimistic we might already be sitting on a theory of everything. But we haven’t turned that corner yet, from asking questions to changing the world.

These ages don’t usually get a household name. Normally, you need an Einstein, a Newton, a Galileo, you need to shake the foundations of the world.

Somehow, Hawking gave us one anyway. Somehow, in our age of questions, we put a face in everyone’s mind, a figure huddled in a wheelchair with a snarky, computer-generated voice. Somehow Hawking reached out and reminded the world that there were people out there asking, that there was a big beautiful puzzle that our field was trying to solve.

Deep down, I’m not sure we deserved that. I hope we deserve it soon.

Of Grad Students and Money

I usually avoid talking politics on this blog. In part, that’s because I usually don’t have something worth saying.

When the US House of Representatives voted on a tax bill that included a tax on grad student tuition waivers, though, I was tempted. Grad school wasn’t so long ago for me, and combining my friends’ experiences with mine I thought I knew enough for a post.

I still had questions, though. So I asked around, and tried to learn more.

In the end, the tax on tuition waivers was dropped from the bill. I’m not going to comment on the rest of the bill, I really don’t have any relevant expertise there.

I do want to say a bit about what I learned, though.

First, the basics:

In the US, PhD students don’t typically pay tuition. Instead, they get paid a stipend, which gets taxed just like any other income. In exchange, they work for their department at the university, as Teaching Assistants and Research Assistants.

PhD tuition isn’t zero, though. Their tuition (often comparable to undergraduate tuition at the same university) is waived, but someone still pays it. Sometimes that “someone” is the department, paying tuition alongside wages as part of the cost of a Teaching Assistant. Sometimes it’s a grant held by a professor, as part of the cost of that professor hiring a Research Assistant. Sometimes it’s another organization: the National Science Foundation or the Fulbright Program, paying for a student who showed their worth in an application process.

 

My first question, then, was this: what determines PhD student tuition?

I know a fair number of professors, many of whom have worked with university administrations, so I thought this would be simple to answer. Then I started asking people, and everyone I asked said something different.

Some thought it was mostly set by comparing to other universities. Others had the impression it was tied to undergrad tuition, that the university had a standard price it charges per course. Others pointed out that at many places, the cost of funding a grad student is the same as the cost of a postdoc. Since postdoc salaries are at least somewhat competitive, this implies that the total of grad student tuition plus stipend is set by the postdoc market, and then the university takes as much of it for tuition as they can before the stipend becomes unreasonably low.

What no one claimed, even after I asked them directly, was that grad student tuition represented the cost of educating a grad student. Grad education does cost money, in professor salaries and campus resources. But I couldn’t find anyone who would claim that this cost was anywhere near what universities charged in PhD tuition.

Rather, grad tuition seems to be part of the bulk of mysterious “overhead” that universities take out of grants. “Overhead” varies from grant to grant and situation to situation, with universities taking less out of some places and more out of others. Either way, it isn’t really overhead in the conventional sense: rather than being the cost to the university of administering that grant or educating that grad student, it’s treated as a source of money for the university to funnel elsewhere, to fund everything else they do.

 

If grad tuition waivers had ended up taxed, couldn’t universities just pay their grad students’ tuition some other way?

Yes, but you probably wouldn’t like it.

Waiving tuition is only one way to let grad students go tuition-free. Another way, which would not have been taxed under the proposed bill, is scholarships.

There are already some US universities that cover grad student tuition with scholarships, and I get the impression it’s a common setup in Canada. But from what I’ve seen, it doesn’t work very well.

The problem, as far as I can tell, is that once a university decides that something is a “scholarship”, it wants to pay it like a scholarship. For some reason, this appears to mean randomly, over the course of the year, rather than at the beginning of the year. This isn’t a huge problem when it’s just tuition, since usually universities are sensible enough to wait until you’ve gotten your scholarship to charge you. But often, universities that are already covering tuition with a scholarship will cover a significant chunk of stipend with it too.

The end result, as I’ve seen happen in several places, is that students show up and are told they’ll be paid a particular stipend. They sign rental contracts, they make plans assuming that money will be there. And then several months pass, and it turns out most of the stipend they were promised is a “scholarship”, and that scholarship won’t actually be paid until the university feels like it. So for the first few months, those students have to hope they have forgiving landlords, because it’s not like they can get the university to pay them on time just because they said they were going to.

 

Of course, I should mention that even without scholarships, there are universities that pay their students late, which leads into my overall point: this system is a huge mess. Grad students are in a weird in-between place, treated like employees part of the time and students part of the time, with the actual rationale in each case frustratingly opaque. In some places, with attentive departments or savvy grad student unions, the mess gets kept to a minimum. Others aren’t so lucky. What’s worse is that this kind of system is often the sort where, if you put it under any pressure, it shuffles the problem around until it ends up with someone who can’t complain. And chances are, that person is a grad student.

I don’t know how to fix this. It seems like the sort of thing where you have to just reform the system all in one go, in a way that takes everything into account. I don’t know of any proposed plans that do that.

 

One final note: I usually have a ban on politics in the comments. That would be more than a little hypocritical to enforce here. I’d still like to prevent the more vicious arguments, to keep the discussion civil and informative. As such, the following rules are intended as conversational speed bumps, with the hope that in writing around them you take a bit more time to think about what you have to say.

For the comments here, please: do not mention specific politicians, political parties, or ideologies. Please avoid personal insults, especially towards your fellow commenters. Please try to avoid speculation about peoples’ motives, and focus as much as possible on specifics: specific experiences you’ve had, specific rules and regulations, specific administrative practices, specific economic studies. If at all possible, try to inform, not just vent, and maybe we can learn something from each other.