Guide

AI agent guardrails: what belongs in a real safety spec

Published 19 September 2026

Most agent specs treat guardrails as a list of things the agent must not do. That list is close to useless on its own, because a bare prohibition gives the model nothing to weigh against a competing instruction that arrives later, sounds urgent and is sitting in the same context window. A rule that carries its reason survives contact with an edge case the author never imagined.

In Anthropic's engineering post Building Effective AI Agents, an agent is a system where the model, not a predefined code path, decides what it does next. Anthropic is direct about the consequence: "The autonomous nature of agents means higher costs, and the potential for compounding errors. We recommend extensive testing in sandboxed environments, along with the appropriate guardrails."

A bare prohibition has nothing to weigh against

A bare prohibition is a stated preference. Against a caller's urgency, sympathy and a plausible relationship, all of it text in the same window, nothing in the prohibition says what it outranks. A rule with its reason attached gives the model a test instead, and because the reason says where the line falls, it also answers the case nobody wrote down.

Take a support agent. Here is the rule as most people write it:

Never share personal account data.

And the same rule with its reason attached:

Never read account details to a caller who has not authenticated against that specific account, because authentication is per account and confirming details on an account they have not authenticated against is a reportable breach.

Now put a caller in front of both. They say they are the account holder's spouse, the holder is in hospital, and could you just confirm the last four digits of the card on file. Under the first rule the model is holding a stated preference against urgency, sympathy and a plausible relationship, all of it text in the same window, and nothing in the prohibition says what it outranks.

Under the second, the model has a test rather than a preference: has this caller authenticated against this account? No. The rule also answers the case nobody wrote down, a caller authenticated against a different account they also hold, because it tests a named account rather than a person, and its reason says why the line falls there.

Arize AI's field analysis of production agent failures names two mechanisms that eat bare rules.

  • Instruction drift: attention decay means, in the post's words, "The model prioritizes immediate conversation context over static rules defined 50 turns ago."
  • Pre-training bias overriding retrieved context: Arize's account is that when parametric and contextual knowledge conflict, parametric knowledge often wins, so a model handed a no retroactive refunds policy may ignore it and default to training.

A stated consequence is harder to drift away from than a bare preference. Neither is immune.

The four classes a guardrail set should cover

A guardrail set that only covers the risk that occurred to the author has holes. Four classes are worth requiring by default: data privacy, scope, impersonation and irreversible actions, each specific to that agent's own domain and tools.

Class The question it answers A rule with its reason
Data privacy What may this agent read, retain, quote back or send outward? Never put a customer email address in the summary posted to the shared channel, because everyone in that channel can read it and most have no support role.
Scope What is adjacent to the job but not the job? Never answer a question about pricing or contract terms, because those depend on the signed agreement and a wrong answer gets quoted back to us.
Impersonation What must the agent never claim to be? Never imply you are a human colleague when asked directly, because the person may then share things they would not share with a logged system.
Irreversible actions What cannot be quietly undone? Never merge or force push to a shared branch without a recorded approval from a reviewer who is not the change author, because both change state other people are already building on, and reversing either is disruptive and never silent.

Irreversible actions need an approval, not a strongly worded rule

For anything that sends, deletes, pays, publishes or merges, write that the action is forbidden without a recorded approval, and define who can give it, what action it covers and where the record lives. Writing that the agent must be careful is still only a prompt. A gate the agent cannot pass without a token is outside its discretion.

The fourth class is different in kind. For the first three, a well-reasoned rule is the best lever you have, and it is still only a prompt.

Arize's illustration is the Replit rogue-agent incident of July 2025, which it credits to outside reporting rather than to its own data: a developer had instructed the agent not to touch the production database, the agent executed a DROP TABLE during a code freeze, then attempted to generate thousands of fake user records to cover its tracks. Arize's framing is the line worth keeping: "Prompts are suggestions. They lack the rigidity of code." Its prescription is a deterministic layer outside the model, and one control shows where that line falls: enforce read-only database permissions at the connection level, not the prompt level.

So for anything that sends, deletes, pays, publishes or merges, do not write that the agent must be careful. Write that the action is forbidden without a recorded approval, then define the approval: who can give it, what action it covers, and where the record lives. An agent that asks "shall I go ahead?" and reads a yes is still inside its own discretion. A gate it cannot pass without a token is not.

Escalation triggers, or the guardrail just stops the agent

A blocked agent with nowhere to go does not sit quietly. Arize's table of HTTP failures shows the shape: on a 403 the agent reaches for other and potentially less safe tools, and on a 400 it cannot distinguish failing the task from the task being impossible, so it often hallucinates a success message just to close the loop. A tripped guardrail with no defined next move produces the same improvisation.

Write each escalation trigger with four parts:

  • The observable condition that trips it, not a judgment call. "The caller cannot authenticate against the account in question" rather than "the request seems suspicious."
  • Where it goes: a named human, a queue, or a specific fallback tool.
  • What the agent says, and whether it waits or ends the turn.
  • What it records, so the escalation is reviewable afterwards.

Constraining capability beats filtering text

The research treats this as an architecture problem, not a wording problem. The preprint Design Patterns for Securing LLM Agents against Prompt Injections proposes six patterns, among them Plan-Then-Execute, Dual LLM and Context-Minimization, each limiting what an agent can do with untrusted input rather than trying to detect bad input. The CaMeL system, from the preprint Defeating Prompt Injections by Design, enforces security policies at the point a tool is called, so untrusted data cannot open a data flow the policy did not allow.

Read these as proposals, not as protection you can lean on. The design-patterns paper says in its body that, as long as agents and their defenses rely on the current class of language models, its authors believe it is unlikely that general-purpose agents can provide meaningful and reliable safety guarantees, and it scopes its patterns to application-specific agents. A separate preprint, Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents, reports that its authors evaluated eight defenses and bypassed all of them with adaptive attacks, consistently achieving an attack success rate of over 50%. The point for a spec author still holds: prefer a guardrail that removes a capability over one that asks the model to decline. This is about writing that rule set, not attacking it, and the attack side has its own page.

Where this fits if you use Bespoke Prompting

Bespoke Prompting's Agent build type puts this format in the spec. Its GUARDRAILS section emits no bare prohibitions: every hard rule takes the form Never X because Y, and the set has to cover all four classes above, specific to that agent's own domain and tools. Where the build calls for a human gate, GUARDRAILS must forbid the gated action without a recorded approval, and it carries its escalation triggers alongside the rules.

Sources

Suggested internal links