What is Replication Crisis?
And what can be done to fix it?
Psychology has a replication problem. Since 2010, scientists conducting replications of hundreds of studies have discovered that a dismal amount of published results can be reproduced. This realization by psychologists has come to be known as “replication crisis”.
For me, this story all started with ego-depletion, and the comics I had drawn about it in 2014. The idea is that your self-control is a resource that can be diminished with use. When you think about all the times you’ve been slowly worn down by temptation, it seems obvious. When I drew the comics, there had been new research pointing to blood sugar levels as the font of self-control from which we all drew from. It also made sense—people get cranky when they’re hungry. We even made up a word for it. We call it being “hangry”.
Fast forward to July of this year when Martin Hagger and Nikos Chatzisarantis published the results of their multi-lab effort to replicate the results of a seminal ego-depletion study. The verdict? Over two thousand participants across twenty-three labs were not able to produce a statistically significant result. Alarmed by the implications, and looking to set the record straight on the comics I had drawn about ego-depletion, I reached out to Martin. Originally, I had planned to draw an erratum of sorts in comic form, but as I spoke with Martin about the issue, it became clear that there was something bigger that I had been missing.
I spoke with several scientists, and found out that this wasn’t just an isolated event in one field. Reproducibility problems were arising wherever they were studied, and they all pointed to various systematic problems and creeping biases within the publishing process itself.
You can read my comic about replication crisis, Repeat After Me: Psychology’s Reproducibility Problem, on The Nib.
But it hardly stops there. I had more material than I knew what to do with, and several interviews happened too late in the process to make it into the comic. As such, I’m going to start publishing the unused material here on Boxplot. Hopefully, together with my Nib piece, it will give people a greater understanding of this current problem in science, as well as how scientists are working to fix it.