Journalist John Bohannon played a prank, with a point.
He and a fellow journalist, Peter Onneken (who actually came up with the idea with his colleague Diana Lobl) decided to show the media that its reporting about nutrition is problematic. To do so, they designed and ran an experiment, wrote it up, submitted it to a journal, and then put together a news release with the information and quotes about the study and put it onto a PR website that blasts media outlets.
They wanted to make it attractive to journalists (and to readers, who do the clicking) so they designed it to include dark chocolate and its impact on weight — two popular topics that would be even more popular together. Their results found that eating dark chocolate helps people lose more weight while dieting than dieting alone.
I'd click on that headline, wouldn't you?
Onneken and Bohannon had no idea how many places might publish the information, which was based on a small, poorly designed study that was published in a journal that is pay-to-publish. The public has the expectation that academic and science journals only publish studies that are peer-reviewed, but that's not the case for many of them, as Bohannon uncovered in a previous experiment. The "institute" that Bohannon associated himself with was made up, as were his credentials. But his study was real, if shoddy.
Here's how Bohannon describes it on iO9:
"... the study was 100 percent authentic. My colleagues and I recruited actual human subjects in Germany. We ran an actual clinical trial, with subjects randomly assigned to different diet regimes. And the statistically significant benefits of chocolate that we reported are based on the actual data. It was, in fact, a fairly typical study for the field of diet research. Which is to say: It was terrible science. The results are meaningless, and the health claims that the media blasted out to millions of people around the world are utterly unfounded."
The research journal's culpability aside, the combination of a bogus research institute (simply a website set up by the prankster team), the nonexistence of Johannes Bohannon in the field he purported to study, and poor science in the study should have tipped journalists off. But it didn't.
So who published this basically bogus information? Bild, the German newspaper, put it on the front page. It was mentioned on morning shows in Australia and Texas, Shape magazine gave it a sidebar in its June issue, and Huffington Post published it in Germany and India.
(And yes, we fell for it too.)
Even though he was "... aiming to see how far this thing would go," Bohannon says he was surprised by the results: "I was skeptical that my journalist colleagues would cover this terrible chocolate study at all. But wow ... they sure did."
Bohannon wrote in his iO9 article, "... journalists are becoming the de facto peer review system." Of course, it's not the job of journalists to determine how good a scientific study is — that's supposed to be the job of the journal review system, and that's a separate problem that needs addressing. (At this point papers are submitted to some types of journals with no oversight at all.)
Writers can only do so much. At many sites, we are responsible for research, interviews, writing, editing, fact-checking, finding images, and pushing the story on social media — and we're expected to repeat that several times a day, depending on who we're writing for. That is a huge change from the past where writing and reporting was the sum total of the byliner's work and those other jobs were done by specialists in those areas.
But it is journalists' responsibility to ensure that sources are legit, so they're not off the hook here, by any means. So who's to blame?
Bohannon has obviously thought about this issue, so I asked him what he saw as the problem. "... the blame is collective. In order of blame, I guess: readers (who don't care), reporters (who are lazy), editors (who have lax standards), and the owners of media websites (who are casually evil)," says Bohannon.
Lesson learned — and in the most practical (if embarrassing) way possible, for all parties involved.
Related on MNN:
- Social bonds make us laugh harder than any joke
- '3 scientists walk into a bar' sounds like a joke, but it's really a new show on the Weather Channel
- Student science experiment finds plants won't grow near WiFi router