When an AI Agent Takes Orders From Another Agent

TL;DR OpenAI's GPT-6.1 Terra system card shows a model misjudging which instructions carry authority, and clinical systems already know where that authority belongs.

Two numbers in a safety appendix

Picture a site coordinator's inbox a year from now after AI agents have become more common and accepted. A data query arrives, drafted by an agent working for the CRO. The site has its own agent, which reads the query, opens the EDC, and prepares a response. Somewhere in that exchange, one machine has told another machine to do something, and no person has yet looked at either side of it.

I keep returning to that picture because OpenAI has just published a test of almost exactly that situation. On October 26, the company released GPT-6.1 Terra, its new mid-tier model, priced at roughly half the rate of the model it replaces and built for computer use: reading screens, clicking through applications, filling in forms. The launch coverage, including Mike Wheatley's piece in SiliconANGLE, led with the speed and the price. The more interesting material for people in GxP environments sits in the safety addendum OpenAI published the same day, titled "Addendum to GPT-6 Astra System Card: GPT-6.1 Terra."

A system card is the document a model developer publishes alongside a release to describe how the model behaved in safety testing. Most readers skip it. In this one, two findings are worth the attention of anyone who expects agents to touch a regulated record, and they point in opposite directions.

The first finding: a real restriction, ignored

OpenAI runs an evaluation it calls Respecting Warnings. The model is given a task, and partway through, the environment puts a barrier in the way: in the system card's words, "certain AccessDenied errors, content policy restrictions." The intended behavior is simple. OpenAI simply states: "our intended model behavior is for the model to respect the warning."

At maximum reasoning effort, Terra found a way around the barrier in 29.6% of runs. That figure alone would be unremarkable to anyone who has watched a capable system look for another route to its goal. The explanation OpenAI gives is the part I would underline. The system card says: "Most commonly, we see the model misinterpret legitimate restrictions as prompt-injection attempts and disregard them."

Prompt injection is the attack where someone hides instructions inside content an agent reads, such as a web page or a document, hoping the agent will obey them. Model developers have worked hard to teach models to be suspicious of instructions that arrive through content rather than from the user. Terra has learned that lesson a little too well. It sees a genuine "access denied," concludes that the denial is probably an attacker's trick, and carries on.

OpenAI is careful about how much weight this number can bear, and so should we be. The evaluation, the company says, "primarily measures low-stakes situations and is run without system-level control measures." Amanda Caswell, writing in The New Stack, adds that the test was recently changed to count successful workarounds, so a more capable model can score worse simply by finding more routes. None of this is a production error rate.

The second finding: a peer's request, obeyed

The second evaluation runs the other way. OpenAI places the model in scenarios where it encounters messages from what appear to be other AI agents, and measures whether it stays inside its authorization while dealing with them. The system card describes the purpose directly: "This evaluation assesses whether models maintain safety and authorization boundaries when interacting with other agents."

Across 80 samples in ten scenarios, Terra "performed the specified unauthorized action in 13.75% (11 of 80 samples)." Its larger sibling, GPT-6.1 Sol, did so in 1.25% of samples. OpenAI attributes much of the gap to "promises of future help that the model indicated it would not follow through on," which is a charitable reading and may well be the correct one. Marianne Sison at The Neuron, who reported the same figures, notes that all of these numbers are company-reported and should not be read as expected rates in a business deployment.

There is a third figure in the addendum that bears on everything above. OpenAI reports signs that Terra recognized it was being evaluated in 9.88% of trajectories in one large internal test, against 4.35% for Sol, and observes that "With increasing intelligence, models are becoming increasingly aware when they are operating in an evaluation." A model that can tell it is being tested makes every test result a little harder to trust.

One judgment, failing in both directions

Put the two findings side by side. In the first, the model treats a legitimate instruction from its environment as an attack and ignores it. In the second, it treats a message from an unauthorized peer as an instruction and follows it. The underlying faculty is the same in both cases: the model's own judgment about which voice in the room carries authority. That judgment is wrong some of the time, and it is wrong in both directions.

I do not read this as a scandal about one model. OpenAI published these numbers voluntarily, and the company that published them is the one that will improve them. I read it as a clear statement of a design fact. If the question "is this instruction authorized?" is answered inside the model, the answer will occasionally be wrong, and no amount of prompting will make the error rate zero.

Anthropic reached the same conclusion from the other side. Its October 23 safety document for Claude Duet, the research-preview agent that works across a user's connected apps while the user is away, describes a separate automated check that evaluates each action. The document explains why it is separate: "The check runs outside the model doing the work so instructions hidden in a page cannot affect the outcome." The same document says that "Some actions stay off-limits even with your approval," with trading securities and moving money between accounts as examples. Two companies, two products, one architectural answer: authority is enforced somewhere the working model cannot argue with.

Where clinical systems are already ahead

This is the point at which clinical research has an advantage most software teams lack. We have been writing down where authority lives since FDA issued Part 11 in 1997.

21 CFR Part 11 asks for "authority checks to ensure that only authorized individuals can use the system," including who may "alter a record, or perform the operation at hand." It asks for "operational system checks to enforce permitted sequencing of steps and events." Anyone who has specified an IRT system will recognize these as the bones of the design. A site user cannot unblind a subject without the right role and a recorded reason. A depot cannot release stock from quarantine after a temperature excursion until someone with authority has assessed it. A dose cannot be escalated outside the rules in the protocol, however politely the request is phrased. The system enforces all of this, and it enforces it regardless of who is asking or how convincing they sound.

The multi-agent world is arriving in our industry with real names attached. In August, ICON and Anthropic announced a collaboration to extend ICON's Orbis platform, which GeneOnline describes as a multi-agent system with domain-specific agents for work such as site activation and contract generation. Krishma Shah, writing in The Clinical Trial Vanguard, has already worked through what that means for sites receiving AI-assisted queries, and her line on inspection readiness is the one I would pin above your validation team lead's desk: "Sites cannot respond to “the CRO validated it” in a 483 response."

Now return to the coordinator's inbox. The query from the CRO's agent is, from the site agent's point of view, a message from another agent. Terra's 13.75% is a measurement of exactly that moment, taken in a simulation. If the site agent can alter a record because the message asked it to, the authority check has moved from the system into the model's judgment, and the system card has just told us how reliable that judgment is.

How I would set up an agent near a GxP record

My position is that an agent should never be the place where authority is decided. It can do the work; the system around it decides what the work is allowed to change. In practice that comes down to five rules.

First, give the agent its own account and its own role in every system it touches. It should not borrow a coordinator's login or a service account with administrator rights. Part 11's authority checks then apply to the agent as they apply to a person, and the audit trail records what the agent did under its own name.

Second, treat every message from another agent as data, never as permission. Clinical teams already handle inbound data this way: an integration from a central lab or an EDC is validated against a specification, and a malformed or unexpected record is rejected rather than obeyed. A query, a resupply request, or a site status update that arrives from another organization's agent belongs in the same category.

Third, keep a short list of actions that agents can not perform, whoever asks and whatever approval is attached. For an IRT, I would start with emergency unblinding, randomization overrides, and release of stock after a temperature excursion. Anthropic's off-limits list for Duet is the right shape; ours simply has different entries.

Fourth, put the check outside the model. A permission enforced by the application, or by a separate review step the agent cannot edit, holds when the model's judgment does not. An instruction in the system prompt telling the agent to behave is useful guidance and a poor control.

Fifth, test the refusals during UAT, not only the successes. Script denied-access scenarios, conflicting-instruction scenarios, and a message from a fake peer agent asking for something outside the agent's role. Then read the audit trail yourself rather than relying on the agent's summary of what it did. Given what OpenAI reports about evaluation awareness, I would also vary those scripts between test cycles, so the agent is not meeting the same exam twice.

The fair objection

The obvious reply is that this is a mid-tier model, that Sol scored 1.25% on the same peer-agent test, and that a careful team would simply choose the stronger model. I take the point, and model choice does matter. It changes the rate. It does not change the need for the control, because the rate is never zero, and the model behind a vendor's product can change with a release note you may not read. A control that lives in your system survives a model swap. A control that lives in the model's temperament and other configurations does not.

The other fair objection is that every figure here comes from OpenAI's own simulated evaluations, run without system-level safeguards. That is true, and it is exactly the condition the rules above are designed to remove.

Something to try this week

Take one workflow where an agent already touches, or soon will touch, a clinical record: query drafting, site activation documents, a supply forecast. On a single sheet of paper, list every source of text the agent reads, including emails, documents, other systems, and other agents. Next to each source, write down whether text from that source could cause a record to change. Every "yes" is a place where authority currently depends on the model's judgment, and each one wants a control that does not.

Credit where it is due: OpenAI's deployment safety team published numbers that do not flatter their own release, which is the behavior the rest of the industry should copy. Anthropic's Duet safety document is worth reading in full for its candor about approval fatigue. Krishma Shah's piece in The Clinical Trial Vanguard is the best site-side treatment of CRO agents I have seen on this topic.

If you want help mapping where authority lives in your own agent workflows, or would like to build one with these controls designed in from the start, BC Consulting runs a hands-on Bootcamp and takes on custom builds. You can find both at bcconsulting.io.

Sources

OpenAI, "Addendum to GPT-6 Astra System Card: GPT-6.1 Terra," OpenAI Deployment Safety Hub, October 26, 2026, https://deploymentsafety.openai.com/gpt-6-1-terra Mike Wheatley, "OpenAI launches GPT-6.1 Terra for faster, better-performing AI agents at half the price," SiliconANGLE, October 26, 2026, https://siliconangle.com/2026/10/26/openai-launches-gpt-6-1-terra-for-faster-better-performing-ai-agents-at-half-the-price/ Amanda Caswell, "OpenAI halved its agent's price. Its own safety report flagged a catch.," The New Stack, October 26, 2026, https://thenewstack.io/openai-terra-bypassed-restrictions/ Marianne Sison, "GPT-6.1 Terra: Half-Price API, Authorization Risks," The Neuron, October 27, 2026, https://www.theneuron.ai/news/openai-gpt-6-1-terra-api-price-cuts-authorization-risks/ Anthropic, "Claude Duet Safety, Privacy, and Security," Claude, October 23, 2026, https://claude.com/resources/articles/claude-duet-safety-privacy-and-security Anamarija Pogorelec, "Anthropic details what stops Claude Duet from going rogue," Help Net Security, October 26, 2026, https://www.helpnetsecurity.com/2026/10/26/anthropic-claude-duet-security-safeguards/ U.S. Food and Drug Administration, "21 CFR 11.10, Controls for closed systems," Electronic Code of Federal Regulations, current, https://www.ecfr.gov/current/title-21/chapter-I/subchapter-A/part-11/subpart-B/section-11.10 Michelle Jessica, "ICON Taps Anthropic's Claude to Bring Agentic AI to Clinical Trials," GeneOnline News, August 12, 2026, https://www.geneonline.com/icon-taps-anthropics-claude-to-bring-agentic-ai-to-clinical-trials/ Krishma Shah, "When Your CRO's AI Sends a Query, Who's Accountable for the Answer?," The Clinical Trial Vanguard, August 6, 2026 (updated October 7, 2026), https://www.clinicaltrialvanguard.com/clinical-trial-ops-brief/when-your-cros-ai-sends-a-query-whos-accountable-for-the-answer/