Runtime · Governance

When the chatbot should have stopped talking

In July 2026, Reuters and The New York Times reported a lawsuit filed by a Florida pastor against OpenAI, alleging that ChatGPT gave him inaccurate medical guidance that kept him from seeking treatment for recurring dizzy spells, and that he subsequently suffered a bilateral pulmonary embolism. The claims are allegations and have not been tested in court. What is not in dispute is the shape of the exposure: a general-purpose assistant answered a question that belonged to a clinician.

It is not an isolated filing. The Washington Post published an analysis of a teenager’s conversations with a chatbot showing the system continued to engage on self-harm across a long exchange rather than decisively escalating. And NBC News reported tests in which AI-powered children’s toys produced sexual content and political talking points, alongside consumer-group warnings about the category generally.

The common failure is not accuracy

It is tempting to read these as model-quality problems — the answer was wrong, so build a better model. That reading leads to the wrong roadmap. In each case the system did what a helpful assistant does: it stayed in the conversation. The missing control was a boundary that recognises certain topics as out of scope regardless of how confidently the model can respond.

Helpfulness is the default behaviour of every assistant you deploy. In a high-risk domain, that default is the hazard.

A refusal is also not sufficient on its own. ‘I cannot help with that’ leaves a user in a medical or mental-health situation with nothing, which is precisely the pressure that pushes product teams to soften the boundary until it stops holding. The control that works is refusal plus handoff: stop generating, and route to a real path — an emergency number, a clinician, a human agent, a documented escalation.

Long conversations erode guardrails

Most safety evaluation happens on single prompts. Most harm in these reports accumulated over long, trusting exchanges — where context drifts, rapport builds, and the boundary that held at turn three does not hold at turn ninety. If your evaluation suite is a list of prompts, it is not measuring the failure mode that produced these headlines. Test multi-turn, test with a sympathetic user, and test what happens after the model has already been helpful for an hour.

Argorix Control applies domain policy at runtime rather than in the system prompt — high-risk topics trigger a defined escalation path, and every decision is logged as evidence you can show a regulator or a court.

If you run a customer-facing assistant

  1. Name your out-of-scope domains explicitly. Medical, legal, financial, mental health, safety-critical. Write the list down; an undocumented boundary is not a control.
  2. Enforce it outside the prompt. System-prompt instructions are guidance the model may lose across a long conversation. Classification and routing at the runtime layer are not.
  3. Pair every refusal with a route. A boundary users experience as abandonment is a boundary the business will eventually remove.
  4. Log the boundary events. How often it triggers, what users did next, where it failed — that record is both a product signal and your defence.

Sources

Test your assistant where it actually fails
Start with a 2–4 week assessment.
Start Assessment