How a Preprint Lie Is Maintained

On January 1, 2022, a preprint was posted claiming that COVID vaccines were “unlikely to protect against infection by Omicron,” then the latest strain of the virus:

Link text on v1:

I wrote about this when it happened, and how the authors were sowing doubts on social media and Substack. It was described well by the CBC at the time:

A Canadian study that vastly underestimated the protection COVID-19 vaccines provide against the Omicron variant is being revised — but not before it spread widely on social media by anti-vaxxers, academics, and even the creators of the Russian Sputnik V vaccine.

The authors were claiming that any three doses of mRNA Covid-19 vaccines were just 37% effective against Omicron infection, while claiming two doses actually showed negative protection:

. . . receipt of 2 doses of COVID-19 vaccines was not protective against Omicron infection at any point in time, and VE was –38% (95%CI, –61%, –18%) 120-179 days and –42% (95%CI, –69%, –19%) 180-239 days after the second dose. VE against Omicron was 37% (95%CI, 19-50%) ≥7 days after receiving an mRNA vaccine for the third dose.

It looked like real science and real math, but the authors acknowledged they didn’t have the right data and didn’t use the right methods to examine it. Yet, up went the preprint, thanks to our friends at Yale, BMJ, Cold Spring Harbor Labs, and CZI LLC, in addition to Dr. JACC and Mr. Rxiv — lest we forget who is enabling so much of this junk.

On January 28, 2022, the promised revision — v2 of the preprint — was posted on medRxiv. The conclusion had already shifted:

Link text in v2:

On September 22, 2022, the paper was published in JAMA Network Open, boiled down to this:

The conclusions are much more attenuated toward healthcare professionals, with nuance unlikely to be hijacked by anti-vax weirdos:

My complaints are multiple here, but the poor design of preprint platforms remains key to it. Here are my gripes:

• Neither v1 or v2 are what was published in JAMA Open Network. It is deceptive to make it seem this artifact matches what’s made it through peer review and editorial work. There are stark differences.
• Why is there a v1 or v2 still available at all? They’ve been superseded by a final paper. Archive them somewhere, or let them go away. They don’t belong to the scholarly record except as drafts, and if we’re just scrapbooking science now, that’s sad.
• Why is the medRxiv statement so ham-handed and templated? v1 was not published in JAMA Network Open. v2 was not published in JAMA Network Open. These kinds of misdirected nods to authority are leveraged by bad players all the time.
  • Alternative: This version has been superseded. Do not cite or link to it. A final version of the paper is available at: . . .
    • If you’re going to run a preprint server, do it well. Unless that’s not possible, in which case shut it down.
• Also, the evidence that preprints deviate in important ways beyond wording changes is everywhere, so please data scientists stop being lazy by doing text-comparison research but instead actually read the history of a few key preprints and you’ll see the problems. Be serious.

The scientific record is a mess. Here’s just one of hundreds of examples. And let’s not forget medRxiv’s culpability in the resurgence of measles and perpetuating Wakefield’s vaccine injury fraud.

Can we move fast and fix things? We hope so.

See you in Charleston next week.