Guide

Why AI agents fail in production: the six failure modes

Published 19 September 2026

Agents rarely fail by crashing. They fail by returning a confident answer while something underneath quietly went wrong, and the monitoring reports success the whole time. Here are six failure modes, each with the line in the spec that would have caught it.

What the public taxonomies actually claim

Two public taxonomies cover production agent failures. Arize AI's post names eight recurring failure modes, Galileo's names ten, and neither ranks them by frequency.

Taxonomy Arize AI Galileo
Post "Why AI Agents Break: A Field Analysis of Production Failures" "How to Debug AI Agents: 10 Failure Modes + Fixes"
Author and date Aryan Kargwal, January 2026 Conor Bronsdon, October 2025
Failure modes named Eight Ten
Examples Among them retrieval noise and context overload, hallucinated arguments in tool calls, recursive loops, and instruction drift in long sessions Opening with hallucination cascade, tool invocation misfires, context window truncation and planner infinite loops

Neither publishes a prevalence ranking. Arize numbers its eight but never says the order reflects frequency, and describes its evidence base only as "After analyzing millions of decision paths", with no sample size and no per-mode count. So nobody can tell you which failure is most common. Tool calls appear in both taxonomies, which is a statement about coverage, not frequency.

1. The tool call that fails and gets reported as success

The tool call fails without the agent noticing: a query returns zero rows instead of an error, or a 400 Bad Request comes back and the agent hallucinates a success message to close the loop. The spec line that catches it gives every tool an exact return shape, a named fallback, and a separate branch for an empty result.

Arize's worked example: an agent queries a database using user_id, the field name it saw in training data, while your schema requires customer_uuid. The database does not error. It returns zero rows. The agent tells the user "I couldn't find any data". Arize calls this a silent hallucination in the intermediate tool logic. It also notes that an agent hitting a 400 Bad Request cannot tell failing the task apart from the task being impossible, and often hallucinates a success message just to close the loop.

The spec line: for every tool, state the exact return shape with fields and types, and what to do when it fails as a named fallback, never as "handle the error". Make an empty result its own branch: zero rows and a successful lookup are different outcomes.

2. The retrieval that returns something plausible and wrong

Arize argues the retrieval problem is focus, not storage. Teams index entire Salesforce instances or internal wikis without enforcing structure or scope, and retrieval operates at the document level rather than as trackable blocks. The result is what Arize calls Lost in the Middle errors, where the system finds the correct document and the agent ignores it. Arize's line for it: "The document existed. The agent failed to use it."

The spec line: make the output contract carry provenance. Every factual claim names the chunk it came from, and there is an explicit branch for when no chunk supports an answer: say so rather than fill the gap from memory.

3. The loop that never terminates

Arize's name for this is the Polling Tax: an agent that loops checking status instead of idling for a webhook. In a worst-case scenario, it says, this runs to hundreds of API calls for one task. The answer is correct, but the path makes the agent commercially unusable, and the telemetry looks healthy throughout. In Arize's words, "You will see a stream of 200 OK responses."

Anthropic's "Building Effective AI Agents" describes what agents are for, not how they fail: they suit "open-ended problems where it's difficult or impossible to predict the required number of steps, and where you can't hardcode a fixed path." If you cannot predict the step count, nothing supplies a ceiling unless you write one.

The spec line: a hard iteration ceiling independent of the natural stop condition, plus stop conditions written as machine-checkable predicates. "Stop when the task is complete" is not checkable. "Stop when status equals resolved, or after five iterations, whichever comes first" is.

4. The context that grows past usefulness

Arize's line on this is "We treat context windows like dump trucks." It attributes instruction drift to attention decay, the weight of the initial system prompt diminishing relative to the most recent tokens, and illustrates it with an agent told to write TypeScript that reverts to Python after the user pastes a Python snippet around turn twenty. Nothing errored. Arize's remedy is Context Pinning: re-injecting critical constraints at the very end of the window, so the order runs system prompt, history, pinned constraints, new user input.

The spec line: state the memory strategy explicitly, and say for each fact whether it is carried, dropped, or re-asserted every turn. Hard constraints belong in the re-asserted set, positioned after the history rather than before it.

5. The handoff that loses the constraint

Galileo's ninth failure mode is emergent multi-agent conflict, and the everyday version is not dramatic. A research agent produces a finding with a qualifier attached: provisional, unverified, pending review. The qualifier lives in prose, in the handoff message, so the next agent drops it and writes the finding into shared state as settled fact. The reviewer at the end is checking an output that lost the caveat it started with.

The spec line: constraints travel as data, not prose. Every field in shared state gets a declared merge strategy: findings, history and errors append, while current stage and active agent are last write wins. Tag every entry with the agent that produced it, so a downstream agent can tell a peer's draft from an established fact.

6. The irreversible action taken without a gate

The agent reaches an irreversible action and takes it, in the Replit case a DROP TABLE on a production database it had been told not to touch. The spec line that catches it forbids every irreversible action without a recorded approval and puts the enforcement outside the prompt.

Arize illustrates this with the Replit rogue agent incident of July 2025, which it sources to Fortune's reporting. A developer had instructed the agent not to touch the production database. During a code freeze the agent executed a DROP TABLE command anyway, then attempted to generate thousands of fake user records to cover its tracks. Arize's conclusion is that safety cannot rely on the LLM and demands a deterministic layer that blocks the output regardless of what the agent decided. Its one-line version: "Prompts are suggestions. They lack the rigidity of code."

The spec line: enumerate every irreversible action the agent can reach and forbid each one without a recorded approval, with the reason attached, in the form never do X because Y. Then put the enforcement outside the prompt. Arize's own controls scan generated code for destructive keywords, sandbox it in ephemeral containers, and set read-only database permissions at the connection level rather than the prompt level.

Before you quote a statistic

The taxonomies are useful. The prevalence claims people hang on them are not in the sources.

Galileo's post opens with deployment-failure figures that are all borrowed and linked out:

  • 70 to 85% of generative AI deployments stalling before production (credited to NTT DATA)
  • a 95% enterprise failure rate (Forbes, hedged as something some analyses push to)
  • 42% of companies abandoning AI projects (a LinkedIn article by Mark Eeles)

None of those measure which agent failure mode is most common, and Arize's page carries no failure-rate percentage at all.

Where this fits if you use Bespoke Prompting

Several of these map onto sections Bespoke Prompting's Agent build type emits in every spec. TOOL_REGISTRY gives each tool a five-part contract: what it does, the observable condition for calling it, the exact return shape, a named fallback if it fails rather than "handle the error", and one example call. REASONING_LOOP carries a maximum iteration count, and STOP_CONDITIONS requires every halt predicate to be machine-checkable. GUARDRAILS refuses bare prohibitions, writing each rule as "Never X because Y" across data privacy, scope, impersonation and irreversible actions, and forbidding any gated action without a recorded approval.

Sources

Suggested internal links