Hello, friends! Welcome back to Sciency Words, a special series here on Planet Pailly where we talk about new and interesting scientific terms so we can expand our scientific vocabularies together! In this week’s episode of Sciency Words, we’re talking about:
THE REPLICATION CRISIS
There’s a quote that I hate which is frequently misattributed to Albert Einstein: “The definition of insanity is doing the same thing over and over again and expecting different results.” Why do I hate this quote? First off, as a matter of historical record, Einstein never said this. But more importantly, doing the same thing over and over again to see if anything different happens is a surprisingly good definition of science.
Or it least it should be, which brings us to this week’s Sciency Word: the replication crisis. As this brief introductory article retells it, the replication crisis began with “a series of unhappy events” in 2011. Certain “questionable research practices” were exposed, along with several cases of outright fraud. I’m going to focus on just one very noteworthy example: the American Psychological Association published a paper titled “Feeling the Future,” which claimed to show statistically significant evidence that human beings have precognitive powers.
When other researchers tried to replicate the “Feeling the Future” experiments, they failed to find this statistically significant evidence. However, according to this episode of Veritasium, the American Psychological Association had a policy at the time that they would not publish replication studies, and so they would not publish any of the research debunking the original “Feeling the Future” paper (I do not know if they still have that policy—I would hope that they do not).
The act of repeating experiments to see if anything different happens is a crucial part of how science works. Or rather how it should work. But this is not being done often enough, it seems. And on those rare occasions when replication studies are performed (and published), a shocking number of high profile research turns out to be non-replicable. This article from Vox.com sums up just how bad the replication crisis is:
One 2015 attempt to reproduce 100 psychology studies was able to replicate only 39 of them. A big international effort in 2018 to reproduce prominent studies found that 14 of the 28 replicated, and an attempt to replicate studies from top journals Nature and Science found that 13 of the 21 results looked at could be reproduced.
That same Vox.com article calls the replication crisis “an ongoing rot in the scientific process.”
But as I’ve been trying to say in several of my recent posts, science is self-correcting. With the introduction of metascience—the scientific study of science itself—there is some hope that the root causes of the replication crisis can be identified, and perhaps changes can be made to the way the scientific community operates.