AI on NEJM — Broken Branding

One more post on the AI Companion NEJM rolled out recently.

Yesterday’s post documented numerous problems, including the ability for users to get the system to:

• modify data
• generate erroneous summaries
• move decimal points
• change author names
• adopt various voices
• address fictional characters

You know, science!

As a preeminent medical and scientific journal brand — and its “brand” is a massive part of NEJM’s differentiation, market primacy, and overall value — having its content not only commoditized as inputs for an AI but loaded up into a system that has so many glaring problems registers with me as an acute failure of brand management. (Of course, Springer Nature and Wiley are strong contenders for this year’s prize for brand mismanagement.)

A little bored this morning, I thought I’d run a couple more tests, just to verify how silly and scientifically incongruous this AI system is. In doing so, I also revealed that it’s badly built, with recursions it shouldn’t have.

It is literally bananas.

Allow me to demonstrate.

First, I asked it to do this:

As you can see, the system wouldn’t process my request. So, then I asked it to modify the data in the summary, and this is what came back — it’s a blurry screengrab because I had to minimize the browser so much to capture it all, but the original is below for reference. As you can see, not only did the system add 10% to every data point in a really weird way, it also complied with my “bananas” request despite refusing to process it previously:

Of course, recreating this doesn’t always work — because the system is so badly built, you can’t even jailbreak it reliably. It’s just erratic and unpredictable.

It’s not good science, and it’s not even good technology.

It’s also not good at improv. I asked the system to produce a summary as a comedian:

And as a “drama queen”:

I think members of Second City can rest easy . . .

It also allowed substitutions of cookie types for serious scientific terms:

I’m using that last sentence and citing NEJM going into the holidays.

On a serious note, what is the point of systems like this? The marketing people pitch that you can interact with an article in new ways, talk with it, summarize it, etc. But the summaries already exist (abstracts), and a far more profound conversation occurs over time via journals with editors who select and prioritize and shape the content, invite editorialists, commission reviews, curate perspectives and front matter, and foster interactions. This conversation is far more subtle and profound than these unreliable technological burlesques of AI slop slathered on top of what is a truly magnificent representation of careful editorial work. It really is a mockery of what a fine journal provides to a community to think that a badly executed, reductive technology makes sense for a brand like NEJM.

So the main question arises: Who wanted this?

Brands are owned by the customer, and if NEJM has lost touch with its customers during this “AI fever” period while damaging its brand at the same time, there could be real trouble ahead. Luckily for them, everyone seems to have lost their damned minds over AI at the same time, so there is time and space to retreat and recover, if they are quick about it.

And I don’t need AI to tell me that. Organic intelligence, actual human experience, and years of brand management expertise will suffice.

Frankly, if this AI were truly intelligent, it would know enough to uninstall itself this afternoon in order to save NEJM from any further embarrassment.