A study claiming hydroxychloroquine increased risk of death used dubious data

Hydroxychloroquine still hasn’t proven itself useful, but research on its dangers has now been called into question.
a hospital room with empty beds
This isn't the final word on the drug, but it looks like another nail in the coffin. Unsplash

Follow all of PopSci’s COVID-19 coverage here, including news on federal policies, the latest on immune-response research, and a state-by-state breakdown of confirmed cases.

This post has been updated.

For weeks, hydroxychloroquine has been making headlines as a hotly contested therapy for treating patients with COVID-19. President Donald Trump publicly praised the antimalarial drug on numerous occasions, and several weeks ago he claimed he’d been taking it himself. But the Food and Drug Administration (FDA) warned back in April that the medication could cause heart problems, and most experts agree that taking it as protection against COVID-19 is a bad idea. Meanwhile, research continues to suggest that the drug, if effective at all against COVID-19, is far from a miraculous treatment.

A study published in May in The Lancet appeared to make an even more pressing case against hydroxychloroquine and the related drug chloroquine. In what was presented as the largest such analysis to date, scientists found that in addition to showing no benefit to patients hospitalized for COVID-19, people who took the medications were at higher risk of suffering heart problems while hospitalized—and perhaps even more likely to die, especially when the drugs were combined with antibiotics.

On Thursday—after indications that the study’s data may have come from dubious sources—The Lancet retracted the paper, which is an indication that the journal can no longer be certain that the findings are scientifically sound. Another study in the New England Journal of Medicine, which focused on blood pressure medications and COVID-19 and used the same contentious dataset, has also been retracted.

As PopSci previously reported, the study in The Lancet was not without its flaws—even before its data sources were publicly contested. Instead of a randomized controlled study, where groups of patients are either given a medication or a placebo, the paper merely analyzed the health outcomes of COVID-19 patients who’d already been hospitalized and treated. While a randomized controlled trial (like the one currently being run in Thailand and Great Britain) can ensure that a large variety of patients are receiving the drug in question, a study that looks back on existing data may be skewed by the fact that doctors were more likely to prescribe it to severely ill patients. This leaves out valuable information on how people with mild or moderate symptoms fare on the therapy. But with some 96,000 patients at 671 different hospitals across six continents included, the new results were still compelling to many outside experts, especially given previous indications that hydroxychloroquine might have dangerous side effects.

We can not yet be certain just how misleading the study’s findings might be, because the private company responsible for collecting patient data and providing it to the researchers—the Chicago-based Surgisphere—has refused to transfer the complete dataset for independent review. One expert told The New York Times that it was completely possible for a small, unknown company to have amassed the quantity of patient data Surgisphere supposedly provided for these studies. But the fact that such a robust bank of patient information had not previously been tapped—and that the folks behind Surgisphere are not cooperating with attempts to verify those numbers—raises enough suspicion to render the studies useless.

The uncertainty now surrounding this study’s results add to a growing heap of questions about hydroxychloroquine as a potential COVID-19 treatment. The findings had initially prompted the World Health Organization to halt a massive study on hydroxychloroquine, but this research has now started back up.

Because COVID-19 is a new disease with sweeping global implications, researchers are working at unprecedented speed to develop vaccines and find treatments. But while this quick thinking may be the only way for us to return to a semblance of normal within the next year or two, it also increases the likelihood of mistakes—and the unscrupulous use of subpar data.

“I’m concerned that the usual standards, both at the level of the journals and at the level of authors and faculty rushing to get high-impact work published, has meant that our usual standards have fallen,” Steven Joffe, a medical ethicist at the University of Pennsylvania, told The Washington Post.

Retail sales of hydroxychloroquine in March—after Trump began bringing up the drug at press conferences—were twice as high as they’d been at the same time a year before. While figures for April and May aren’t available yet, the drug has been hard to come by for those taking it for Lupus and other autoimmune diseases, for which it has FDA approval.