LLM Policies Prove a Distraction

This one got weird fast, and in ways I never anticipated.

Early last week, I accidentally stumbled into an active Nature News investigation into LLM use among authors writing for “The Scholarly Kitchen” and was amused to learn this is being treated as an issue of such gravity that very serious policies are being refined, reinforced, and revised.

Basic question: To what end?

The nature of posts on “The Scholarly Kitchen” and similar outlets — this one included — made it clearer to me that such policies don’t address the core issue that faces every publisher, reviewer, and editor:

• What do you choose to publish?

“The Scholarly Kitchen” Gets Played

Editors can get played when they let anything through, whether AI use is involved or disclosed — or not.

It turns out “The Scholarly Kitchen” accidentally participated in a pseudoscience grift last week. And, yes, the use of an LLM was uniformly disclosed by the author involved.

The disclosures didn’t matter.

Friday’s guest post on “The Scholarly Kitchen” was attributed to Julia Romano, who started it off inauspiciously with “I am not a paper mill” and then discussed how hard it was to get a preprint posted on PsyRxiv because the screener “found no record of previous relevant peer-reviewed publications by the author.” This forms the basis for a complaint about gatekeepers screening credentials, making it harder for certified yoga instructors with fresh ideas to get citable preprints.

• Fresh ideas? More like tried-and-true wellness grifts . . . now with an assist from the scientific publishing industry, where any barrier is viewed as bad, and fraud and “exclusion” are portrayed as equivalent.
  • Fraud is fraud. “Exclusion” is a head-fake word to smear selectivity.
    • Next thing, they’re going to go all the way to Cyberlibertarian City claiming that choosing not to publish something is “censorship.”

But being bounced from PsyRxiv was only a blip in our porous publishing system. There are other preprint servers using the same OSF Preprints infrastructure.

So she turned to SocRxiv.

That worked. The preprint on SocRxiv was submitted in April, and has already been through three revisions (“I am not a paper mill,” she said). Version 1 has a file name containing “Final.” Version 2 has “Final-Submission.” Version 3 has “Revised_v3.” So, yeah, there’s lots going on here.

The initial version upload coincided with Romano’s posting on a related Substack pushing her business as a counselor and yoga therapist.

What does the paper deal with? Per the abstract:

This paper introduces the Comprehensive Postpartum Assessment Index (CPAI), a novel multi-domain screening instrument grounded in the panchamayakosha model from the yoga therapy tradition — a five-layered framework that understands the human being as physical body, breath and energetic body, psycho-emotional realm, discernment body, and capacity for awe.

This reflects a common output in the AI-generated review article sphere called the “Paradigm Layering Framework.” Put some keywords into an LLM and you can get a hierarchical output like the one Romano describes. So it’s no surprise Romano also discloses that:

Claude AI (Anthropic), a large language model, was used as a research assistant in the development of this paper. Claude assisted with literature identification and synthesis, structural drafting, and editorial refinement across multiple working sessions.

Pushing this “novel screening tool” may help her earn some new clients, but whether or not an LLM was used, it’s still snake oil on a yoga mat — a blatantly commercial preprint pushing a yoga business and offering nothing that advances scientific insights.

Romano is participating in science cosplay, whether an LLM was involved or not. And “The Scholarly Kitchen” is helping her cook the science books, as is SocRxiv.

All of this hasn’t cost Romano a dime . . . yet.

Of course, the preprint is under review at a hybrid OA journal, where Romano can pay US$4,320 to get it published. Given how harmless and ignorable the content is, I’m betting she will have her AI slop paper accepted, Taylor & Francis will happily collect the fee, and she will make hay of it as best she can.

• Pseudoscience and OA publishing are highly compatible on many levels, especially if the consequences of a junk paper are limited to more downward-facing dogs and lotus positions.

Best of all, it will be “peer reviewed” at that point!

Does it matter that she disclosed AI usage in both writing the paper and “The Scholarly Kitchen” post?

Not at all.

What matters is that SocRxiv let her get the DOI she wanted so badly, and that a Taylor & Francis OA journal will probably accept payment to publish her pablum. In between, the SSP’s official blog — which I founded nearly two decades ago, and left about a decade ago — further enabled her by platforming her thoughts about publishing, even though she has no credentials, experience, or relationship to scholarly publishing except as someone trying to exploit it for her business.

Tech Isn’t Incentives

The problem with the chain of events above wasn’t AI or LLM usage. It was that there are marketing incentives to post pseudoscience gobbledygook in the scientific publishing system while further eroding the idea of expert, rigorous scientific publishing in order to clear the path for more gobbledygook and grifting.

• It’s weird that an apparent incentive for the official blog of SSP is to advance posts that dump on editors, reviewers, and selectivity writ large. If you read the comments on the post, these people need support, not this nonsense.

This points to a deeper problem in how we ensure science is working for a specific community.

If publishers, editors, and journal executives were focused on epistemic value while being held accountable by paying readers rather than relying on technology and — more and more — authors as revenue sources, this discussion would be very different.

Without feudalism providing the fuel, SocRxiv probably wouldn’t exist as a dumping ground for tawdry commercial pseudoscience slop. Without APCs, Taylor & Francis would have likely already desk-rejected her submission.

But we are so inured to articles posted on platforms funded by philanthrocapitalists or published on a “cash on the barrelhead” basis, we are unable to remember a world where publishing businesses only made money if sophisticated, expert consumers were happy, in that way ensuring the market reflected their needs and wants.

• It’s another way AI and OA align — neither one is connected to consumers, but instead exploits them by extracting value from producers first.
  • Just like much of Silicon Valley . . .
    • We have to click a box to attest that we are not robots, and now robots are watermarking their text to attest that they are robots. It’s insanity.

From BMJ to JAMA and others — most with, not coincidentally, growing concerns in Gold OA revenues — AI-use disclosure provides cloud cover for more of the same problems that have gripped scientific and scholarly publishing for decades now.

Rather than addressing the fundamental corruption of pay-to-play or funder influence, we just paper over it with whatever policy-shaped ideas we need to in order to keep the money flowing. LLMs are new. Let’s policy those up for a while to distract from the deeper corruptions . . .

To see why such policies don’t actually matter to the core functions of journals, let’s talk about how scientific articles at top journals like BMJ and JAMA are typically handled.

• That is, does the use of an LLM by an author to write and/or refine a submission ultimately matter?