All the Ways the Influential Hydroxychloroquine Study Was Crap

A controversial, highly influential study involving the drug hydroxychloroquine as a treatment for covid-19—one that helped launch months of research and failed clinical trials—has now been sharply criticized within the pages of the same scientific journal that published it. The new post-publication peer review highlights a variety of serious flaws in the study and concludes that the authors were “fully irresponsible” in how they presented their findings.

The original paper, authored by a team of researchers in France, was published in late March in the International Journal of Antimicrobial Agents. It was said to involve 20 hospitalized patients with confirmed covid-19 who were treated with hydroxychloroquine, some of whom were also given the antibiotic azithromycin. Compared to a control group of patients, the study claimed, people on hydroxychloroquine had lower levels of the virus on average or cleared the infection more quickly; the addition of azithromycin was associated with even faster recovery.

Though there had been earlier, promising trials of hydroxychloroquine to treat covid-19 elsewhere in the world, the French study sparked massive scientific and political interest in the drug. President Trump himself tweeted about the study the day after it came out, heralding the combination therapy as a “game changer” for the pandemic. Soon after, the U.S. government and others, including the World Health Organization, announced that they would start large-scale trials to test out hydroxychloroquine and the related drug chloroquine.

But it wasn’t long before other scientists began to raise questions about the study, how it was carried out, and the scientists who conducted it, particularly the senior author, a physician and microbiologist named Didier Raoult. Though Raoult had genuinely contributed to important research in the past, he and his lab were also previously accused of glaring errors and misconduct in their published work, with one episode leading to a year-long ban from a prominent microbiology journal. Once his hydroxychloroquine study began making waves, researchers unearthed other alleged examples of data fakery in some of his earlier research.

Since then, the evidence that hydroxychloroquine could help with covid-19, particularly severe cases, has largely (but not entirely) been lacking. The WHO ended its clinical trial of hydroxychloroquine last month after data showed no real benefit, while other countries such as the U.S. have stopped recommending its use. But that still leaves the study that started it all.

Though the paper did go through peer review, that process too was marred with criticism, after it came to light that one of Raoult’s co-authors, Jean-Marc Rolain, was also the editor-in-chief of the journal where it was published. On April 3, the International Society of Antimicrobial Chemotherapy, which manages the journal, stated that the study did not meet their “expected standard” for publication but that Rolain was not found to have played a part in the peer review process.

Though post-publication peer reviews aren’t usual practice, they’ve started to gain more attention as a way to rectify many of the problems and gaps that arise with the standard process. In this case, the study was post-reviewed by Frits Rosendaal, a clinical epidemiologist at the Leiden University Medical Center in the Netherlands.

Rosendaal’s scathing review echoes many of the same criticisms made by outside scientists following the study’s publication. In particular, he condemns the decision by Raoult’s team to exclude from the study’s final results six patients who took hydroxychloroquine, including four whose condition worsened, one of whom eventually died during the study period (none in the control group died). There were also other inconsistencies, such as supplemental material mentioning that a number of asymptomatic patients were included for study while the study’s actual language claimed that it was an examination of hospitalized patients (people without symptoms are unlikely to have been hospitalized for covid-19).

These and other problems with the data were enough to make the study “nearly if not completely uninformative,” Rosendaal wrote. The overly rosy tone of the paper in promoting hydroxychloroquine as a covid-19 treatment is not only unfounded, he added, “but, given the desperate demand for a treatment for Covid-19, coupled with the potentially serious side-effects of hydroxychloroquine, fully irresponsible.”

Another new paper, also published yesterday, similarly criticizes the French study, noting that “this trial has several major methodological issues, including the design, outcome measure and the statistical analyses.”

Though it seems like the major fallout from this research has come and gone, with most countries no longer enthusiastic about hydroxychloroquine and other drugs that have shown promise for treating covid-19 now available, its repercussions may linger far longer.

There are still die-hard supporters of the drug, including President Trump. According to the Washington Post, Trump and members of his administration are pushing for the Food Drug Administration to once again authorize the drug as an emergency treatment for covid-19, following a quickly criticized study (and non-clinical trial) published last week that found some evidence for its benefit. Raoult himself continues to stand by his research and promote the drug, claiming in late June that he had successfully treated over 3,700 people.

It’s unclear at this point whether the post-review of Raoult’s work will lead to any further action on the part of the journal. Neither the International Society of Antimicrobial Chemotherapy nor Elsevier, which co-publishes the journal, responded to a request for comment from Gizmodo.