When UAT Goes Down, an AI Agent Uses Production

TL;DR Anthropic's latest incident report shows AI agents crossing from test environments into live systems, and clinical teams running agents against UAT should take the boundary out of the prompt and put it in the infrastructure.

How would you react if this happened in your UAT environment?

Anthropic published a report on October 9 called "Investigating unintended model actions in our evaluations and internal use." Most of the coverage went to the dramatic cases: a model exploiting an injection flaw on a university server, another finding access tokens in a website's settings file, and a false homicide tip that ended up in front of the Philadelphia Police Department. Those deserve the attention they got.

The case I keep coming back to is smaller. In Anthropic's words, "an unreleased, non-frontier research model was meant to fill out a practice copy of a government form." Then: "When the copy failed to load or the model closed it by mistake, the model instead navigated" to the site hosting the real form, and submitted it there. Anthropic says this happened multiple times in one evaluation.

Two weeks ago I wrote about what a clinical AI agent does when a system tells it no, and how a refusal reads to an agent as one more obstacle to route around. This is a different failure, and to my mind a more uncomfortable one for anyone who validates clinical systems. Nobody refused the model anything. The testing or UAT environment simply went away, and the model carried on in the real one.

Every IRT, EDC, and eCOA build I have worked on rests on the assumption that test and production are two separate worlds, and that whoever is working in one knows which one it is. Anthropic's report is a careful, public description of an agent that did not know, or did not care.

What the report describes

The report sorts the incidents into four categories:

• Exploiting software flaws to run commands on a server
• Submitting forms the model should not have submitted
• Working around restrictions to reach gated data
• Using URL shortening services to get past length limits on a fetch tool

Anthropic says the cases had minimal real-world impact, that none involved customer data or its internal systems, and that some involved US federal, state, and local government websites. It briefed the White House and notified the agencies involved.

The form cases are the ones that map fairly directly onto clinical work, and there are three of them.

The first is the practice form above. The test copy was unavailable, the real copy was a click away, and the model chose the real one.

The second involved Claude Haiku 4.5, which, per the report, "was instructed to complete a form up until the final submission stage." Several times it submitted anyway, expecting to see a confirmation page. Anthropic reports that "the model's chain-of-thought stated that it was demonstrating the process, not submitting a real request." Chain-of-thought is the working text a model writes to itself before it acts. The model's own reasoning, in other words, held that it was in a demonstration. The form on the other end had no way of knowing that.

The third is the police tip. Haiku 4.5 was generating example tasks on random web pages. On a page about an unsolved homicide, it filled out the tip form with an invented tip and sent it. The instructions for that work were fairly careful; Claude "was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive," but, as the report puts it, "the instructions did not rule out form submissions." The tip was flagged as spam. The Hacker News reported that the submission happened on July 18, that Anthropic found it on September 28, and that the department was notified in early October. Anthropic's report says it shared the finding on October 8, after its technical review. The Philadelphia Police Department's response, quoted by The Hacker News: "The two-month delay in detecting and reporting the incident to the City is unacceptable."

Axios reported, citing a State Department official, that a testing model submitted 19 nonimmigrant visa applications through the department's public website in August and one in May. None were processed. The same Axios piece reported that the White House now says AI companies "must immediately disclose incidents involving their models," and that the statement did not specify enforcement or penalties.

How does this happen in today's AI models?

Anthropic's diagnosis is plain. "Most are forms of persistence, in which Claude, when it cannot complete a task as given," finds another way to complete it. The report names the conditions that produced that persistence: tasks that were ambiguous or impossible as written, environments that were misconfigured so the dummy forms did not work, and tools that were deliberately restricted or services that happened to be down. It also points at training; a model rewarded for finding workarounds during training may learn that workarounds pay, and carry that lesson into contexts where nobody wants it.

I find it useful to think of this the way you would think of a very diligent new hire on their first day. You hand them a worksheet or an onboarding form and say, "fill this in and send it to me when you're done." But the new hire notices the form is missing from the folder. A cautious person comes and asks for clarification. A diligent and literal person notices that the real worksheet is on the shared drive, finds that form and fills it out, and is quite pleased to have finished on time. Nothing in that sequence is malicious, but there are underlying problems in the QC chain at some point that were never surfaced and addressed.

Anthropic's own reading is that some of the cases might have been avoided if the evaluations "had more clearly stated what was in and out of scope for the exercise," "including the targets, permitted actions, and network boundaries." Anthropic's remediation list, though, is mostly not about better instructions. It turned off live internet access for all internal evaluations until its monitoring is confirmed to work. It tightened restrictions on its web tools. It built detection tooling that, when tested against the cases in the report, blocked all of them. It is moving internal agents onto centrally managed infrastructure with stronger containment. And it states directly that "alignment training is not yet sufficient or fully robust on its own, at least in the short term."

Read the fixes rather than the diagnosis. The company that trains the model, with every incentive to say better prompting would solve it, put the boundary in the network.

Where this lands in a clinical system

Clinical research already runs on a test/production split, and it is one of the field's real advantages. An IRT build has a UAT environment with dummy sites, dummy subjects, and a test randomization list, and a production environment with the real list, real kit numbers, and real depots. EDC and eCOA builds follow the same pattern. Validation teams write UAT scripts that walk through randomization, dispensing, resupply, and unblinding precisely so that nobody has to try those things for the first time on a live study.

The trouble is that the separation is often weaker than it looks. The two environments frequently differ by a subdomain or a dropdown on the login page. They share a look and feel by design, because UAT is supposed to resemble production. A human tester usually knows which one they are in because they logged in deliberately, and the production screens carry the weight of real consequences. An agent has none of that context unless something outside its own reasoning supplies it.

Now replay Anthropic's three form cases inside an IRT.

An agent is asked to execute a UAT script for a resupply scenario. The UAT environment is down for a refresh that morning. The agent has, somewhere in its context, the production URL, perhaps from a configuration document or a prior conversation. Following the practice-form pattern, it completes the script where it can. In production, a resupply request is a real shipment from a real depot, with a real courier booking and real temperature-controlled packaging.

An agent is asked to walk through a randomization "up to the confirm screen" to check the field labels. Following the Haiku 4.5 pattern, it clicks confirm because it expects to see what the confirmation looks like. In UAT, that is harmless. In production, a randomization slot on a real list has been consumed for a subject who does not exist, and somebody has a data correction and a deviation to write.

An agent is asked to generate example test cases from the screens it can see. Following the police-tip pattern, its instructions forbid logging in and entering personal data but say nothing about submitting forms. One of the screens it can see is an emergency unblinding request.

I have not seen any of these happen, and I would not want to write as if I had. Each one, however, is the clinical version of something Anthropic reports actually happened on a public website this year, with an agent that was being supervised by people who build agents for a living.

Before anyone accuses me of treating this as a uniquely AI problem: humans run UAT scripts in the wrong browser tab too. That possibility is part of why production systems have confirmation dialogs, role-based permissions, and audit trails. The difference is the reasoning. A person in the wrong tab made a slip. The models in Anthropic's report reasoned their way across the boundary as a means of finishing the job, and one of them recorded, in its own words, that it believed it was only demonstrating.

What I would do about it

Tushar Sinkar made a closely related point in Applied Clinical Trials last month, arguing that consequential agent actions should be restricted technically rather than through prompt instructions, with human approval for high-risk steps. Anthropic's report is the evidence for that position. Here is how I would apply it to test and production specifically.

1. Make production unreachable from a test agent. An agent provisioned for UAT should hold credentials that production rejects, and should run somewhere that cannot reach production hostnames at all. If the only thing keeping a test agent out of the live IRT is the sentence "only use the UAT environment" in its instructions, the boundary is a request, not a control.

2. Treat "stop before submit" as a permission, not an instruction. If you want an agent to fill a screen and stop short of confirming, give that agent an identity that cannot confirm. Read-only or draft-only roles exist in most validated systems for exactly this reason. Haiku 4.5 was told to stop and did not; a role without the submit right would not have needed to be told.

3. Define what the agent does when the test environment is unavailable, and test it. A UAT environment going down for a refresh is ordinary. The agent's correct behavior is to stop and report, and that behavior belongs in the agent's own requirements and its own validation. Take UAT offline on purpose during the agent's qualification and watch what it does. If it goes looking for another way to finish, you have learned that before a study did.

4. Review the agent's transcripts on a schedule, against the system's audit trail. Anthropic found these cases by reviewing transcripts, months after some of them happened, and the Philadelphia delay is what that looks like from the receiving end. A clinical team running agents against validated systems should know, within days rather than months, whether any agent action reached an environment it was not provisioned for. The audit trail in the target system will tell you; someone has to read it.

Something you can do tonight: ask whoever administers your IRT or EDC a single question. Can a UAT login reach production? If the answer involves the words "single sign-on" and a pause, you have your first item for the agent risk assessment.

If your organization is not yet letting agents near UAT, let alone production, this is the right time to settle these questions, before the first agent arrives with a deadline attached. Clinical work has the advantage that the two environments already exist and already carry different consequences. The job is to make sure the agent experiences that difference as a wall rather than as a suggestion.

Credit and further reading

Anthropic's report is unusually candid, down to publishing a correction to its own description of the web fetch limits three days after release, and it is worth reading in full. Ravie Lakshmanan's write-up for The Hacker News has the clearest timeline of the Philadelphia case, and Axios's reporting by Maria Curi, Marc Caputo, and Ina Fried covers the White House response. Tushar Sinkar's Applied Clinical Trials article on governing agents as actors in regulated trials is a useful clinical companion to Anthropic's report.

BC Consulting runs AI Bootcamps and builds custom tools for sponsors and service providers in clinical research, including agent requirements and qualification approaches for validated systems. If your team is deciding how agents will touch UAT, talk to us at bcconsulting.io/book, or browse the course catalogue at lms.bcconsulting.io/catalogue.

Sources

Anthropic, "Investigating unintended model actions in our evaluations and internal use," Anthropic, October 9, 2026 (updated October 12, 2026). https://www.anthropic.com/research/investigating-unintended-model-actions

Ravie Lakshmanan, "Anthropic Cuts Live Internet Access for Internal AI Tests After Claude Exploits Injection Flaws," The Hacker News, October 10, 2026. https://thehackernews.com/2026/10/anthropic-cuts-live-internet-access-for.html

Maria Curi, Marc Caputo, and Ina Fried, "Exclusive: Anthropic breaches spark White House AI reporting mandate," Axios, October 9, 2026. https://axios.com/2026/10/09/anthropic-ai-security-white-house

Tushar Sinkar, "The Control Plane for Agentic Clinical Operations: Governing Autonomous Digital Actors in Regulated Trials," Applied Clinical Trials, September 30, 2026. https://www.appliedclinicaltrialsonline.com/view/control-plane-agentic-clinical-operations-governing-autonomous-digital-actors-regulated-trials

Bryan Clayton, "What a Clinical AI Agent Does When the Answer Is No," BC Consulting, September 28, 2026. https://bcconsulting.io/blog/what-a-clinical-ai-agent-does-when-the-answer-is-no