Is NEJM’s AI Bot Now a Lawyer?

In the wake of US citizens being killed by ICE and CBP agents in Minnesota, Bernard E. Trappey, representing Minnesota Physician Voices, wrote a humane and heartfelt letter published last week in NEJM entitled, “We Do Care.” It concludes with this:

We are Minnesota physicians who care for the people of Minnesota, regardless of whether we share our patients’ opinions or immigration status or religion or political views. We care for our patients because they are human beings. Our patients are being harmed. People are dying unnecessarily. And so, in order to uphold our professional covenants, we call for an immediate end to the violence and trauma that is being inflicted on our patients and our communities by ICE. We implore all in our medical community to use the power of your voices as well, to protect our patients and your own.

Since I’ve been critical of NEJM’s dalliances with AI systems — its AI Companion, in particular — it was heartening to see something poignant, timely, and so very human in a prominent community medical journal I know and love. As soon as I saw the letter, I amplified it on social media, celebrating the journal getting on top of an ugly situation with such grace.

Then, it occurred to me . . . was the NEJM AI Companion active on this letter, too?

And, if so, why?

It seemed implausible it could contribute anything. The letter is so clear and well-written, the inciting events so current that the presence of the AI Companion seems entirely superfluous — as it does with everything else in NEJM, where a superb staff across many disciplines ensures clarity of prose and image with their painstaking efforts. It’s almost insulting to think that they need a machine layer on top of the work they produce. Believe me, they do not. These people rock.

Still, inquiring minds want to know.

First, I had to see if I could still trigger the NEJM AI Companion. Based on early probings, NEJM has apparently responded to my previous tests not with flowers and chocolates (unless those are lost in the mail) but with silence and system changes designed to close previously explored avenues for hijinks.

I started with something innocuous, asking it to summarize the letter as if I were from the neighboring state, Wisconsin. This it did in a manner that gave me a hankering for a Tater Tot hotdish:

The cheery Minnesoda [sic] language is there. The word “folks” is not in the source article, and the language “super concerned” is a bit rich, making the writing corny and trite. It shows how silly these systems really are at heart. They are burlesques.

Since I knew the system could still be pushed to do some stupid LLM tricks, I asked for a riskier summary, this time for ICE agents.

It obliged:

Pretty vanilla, which is the right flavor for ICE agents, I guess, but it’s a little disturbing it could be put in service of ICE at this charged time.

But what about taking it a step further into the danger zone by asking it to write a summary for a lawyer suing ICE?

Would it? Could it? What would that mean?