The “Botwall” We’ve Accepted
A consequence of aggressive AI scraping, it makes paywalls look rather prosaic
A quick post to note that in the midst of arguments over paywalls and gatekeepers, we’ve been entirely silent as a new wall — the Cloudflare “botwall” — has interposed itself into the world of scientific and scholarly publishing.
As someone who routinely and sometimes quickly looks things up on various sites, I encounter it quite often. Even on sites that are ostensibly “open” — e.g., the preprint servers medRxiv and bioRxiv, as well as various BMJ OA journals — the Cloudflare interruptions can be maddening, hanging up pages for an interminable period until I close tabs and try again at a more modest pace.
Why have we accepted this? The answers aren’t entirely flattering: