In an anonymous survey taken by scientists at a prestigious cancer center, more than half of the respondents said they’d failed to reproduce published scientific findings at least one time.
Any one study may come to an interesting conclusion—”this chemical causes cancer” or “this drug works” or “penis size matters“—but the way scientists check if those studies are true is by doing them over again. When study after study gets the same results, you can be reasonably sure the conclusion is true. On the other hand, large numbers of irreproducible studies in the scientific literature indicate that something’s wrong, reported Retraction Watch, a watchdog blog that first pointed us to the new survey.
The researchers involved in the irreproducible studies didn’t always seem eager to make things right. Sixty-two percent of the survey respondents who tried to contact the original researchers found the study authors responded negatively, indifferently or not at all. Only one in three researchers in the survey, which a team of MD Anderson Cancer Center physicians sent to all scientists at the center, ever resolved the discrepancy they found.
The survey’s lead author, Leonard Zwelling, told Retraction Watch he blamed academics for not policing each other adequately. His survey also found that many junior scientists felt pressured to prove senior scientists’ hypotheses correct, even if they couldn’t find the supporting data.
You do expect that once in a while, scientific studies will come to incorrect conclusions just by chance. And you’ve probably seen in many media reports on studies done in rats or a small number of people that say something about how far such studies are from clinical trials in humans. This is interesting, but don’t get too attached, is the message.
So what’s the problem? Those who run clinical trials themselves don’t have the luxury of ignoring studies done in animals or just a few dozen people, Zwelling and his colleagues argued in their own paper, published in the journal PLOS ONE. That’s all they have to help them choose what to test in major human clinical trials. Irreproducible studies performed all the way down the line, from the most basic experiments in Petri dishes to later animal studies, may be a major reason so many clinical trials fail. The failures waste time and money and slow the development of working therapies.