Guide

Why every agent tool call needs a fallback, not just a happy path

Published 21 September 2026

Most agent specs describe what a tool does and when to call it, then stop. The question they skip is the one that decides whether the agent is safe to deploy: what does it do when the call comes back wrong, empty, slow, or not at all? Writing "handle the error" in that slot does not answer the question, it reserves space for an answer you never wrote.

"Handle the error" is a placeholder wearing a decision's clothes

It survives review because it sits exactly where a decision belongs and reads like one. At runtime it gives the model nothing: no branch, no threshold, no halt. It says be sensible about this, and being sensible under uncertainty is exactly what a language model is unreliable at. Faced with an ambiguous failure and an instruction to cope, it produces the most plausible continuation available, which is almost never an admission of failure.

A fallback is a decision you already made, written down before the incident. Three things make one real:

  • a retry budget with a number in it
  • a defined state for when that budget is exhausted
  • a named outcome the agent moves to next

The failure is well described, and nobody has sized it

Two published failure taxonomies both name tool calls as a recurring production failure mode.

Taxonomy Author and date Failure modes listed Tool call modes named
Arize AI, Why AI Agents Break: A Field Analysis of Production Failures Aryan Kargwal, January 2026 Eight Hallucinated arguments in tool calls; unhandled external API schema changes
Galileo, How to Debug AI Agents Conor Bronsdon, October 2025 Ten Tool invocation misfires

Neither attaches a frequency to it. Arize describes its evidence base as a year spent analyzing agent behavior in production, across millions of decision paths, but publishes no sample size, no per-mode counts, and no ranking. The numbering in both lists is order, not prevalence. Be suspicious of anyone who hands you a prevalence figure for this one.

What the sources give you instead is shape. Arize's worked example: an agent guesses a column named user_id because that pattern is common in training data, while the real schema requires customer_uuid. The database returns zero rows rather than an error. The agent sees an empty result behind a healthy status code and tells the user it could not find any data. Arize calls that a silent hallucination in the intermediate tool logic, and notes that an agent hitting a 400 cannot tell the difference between failing the task and the task being impossible, so it will often report a success message just to close the loop. The post's own summary of the behavior is short: "Agents are really confident liars."

Four outcomes, and you have to pick one

When a tool call fails, there are four things an agent can legitimately do, and a spec that does not choose one has not specified anything. This framing is ours rather than either taxonomy's.

Outcome What it means Right when Must also name
Retry Call the same tool again, same or adjusted arguments The failure is plausibly transient: timeout, rate limit, server error Attempt count, backoff, and what counts as the same failure recurring
Substitute Get the same fact from a different tool or source Another surface can answer the question Which tool, and whether its answer carries lower confidence
Degrade Continue with less: partial results, a cached value, a broader search A partial answer is genuinely useful to the person receiving it What is missing, and the requirement that the output says so
Stop and report Halt the run and hand the failure back The missing thing is load bearing, or the next step is irreversible What the user sees, and what state the run is left in

Degrade is not the same as substitute: substituting keeps the answer whole by changing where it came from, degrading keeps the run alive by shrinking the answer. Both are acceptable. Only degrading requires the caveat to reach the output itself; a substitution records its lower confidence in the spec, not necessarily in the answer.

The fifth outcome is the one nobody writes down

Proceed as if it worked. It never appears in a spec, which is precisely why it is the default. If you do not pick one of the four, the model picks this one.

That is the outcome worth engineering against, because it removes the signal. A stopped run is a visible problem with an owner. A masked failure is a confident answer built on a call that never returned, and it propagates: downstream steps treat the fabricated result as settled input, and by the time anyone notices, the trace looks healthy the whole way through.

Write the exhaustion case, not just the retry

Specs that do mention retries usually stop at the number of attempts. That is half a fallback. The half that matters is the state after the last attempt, because that is the branch that runs during an actual incident.

Retry is also only safe when the call is idempotent. A retried read costs latency; a retried send, charge or publish costs you a duplicate, and a retried delete can hit the wrong record if ids get reused. Write how that is prevented before you write the budget. And a retry budget with no exhaustion clause is worse than no retry at all, because it turns a fast visible failure into a slow invisible one.

Where this fits if you use Bespoke Prompting

In Bespoke Prompting's Agent build type, the TOOL_REGISTRY section gives every tool a five part contract: what it does in plain English, when to call it as a specific observable condition rather than "when needed", what it returns as exact fields and types, what to do if it fails, and one example call with real arguments. The fourth part is this article compressed into a slot. It asks for a fallback in the shape of retry once then X, or fall back to Y, and its wording rules out "handle the error" as an acceptable answer.

Sources

Suggested internal links