Guide

When to use an AI Agent

Published 14 September 2026

One test decides this, and it is not how advanced your use case sounds. If you can write the steps down before the run starts, you do not need an agent. If the next action depends on what the last one returned, and you cannot know that ahead of time, you do.

Start with a case, not a definition

A decision tree you could draw on a whiteboard is a workflow. A loop whose next branch is only decided at runtime is an agent.

IBM Think's page on agentic workflows walks through IT support. The rule-based version runs a chatbot through static decision trees and predefined responses, escalating to a human when it stalls. IBM says that is efficient for basic, well-defined issues but struggles with complex, multistep troubleshooting that requires adaptability. Its agentic version of the same job is a five-step loop: understanding the problem, executing diagnostic steps, adaptive tool use, iterating based on results, and finalizing and learning.

Which diagnostic runs next is not knowable until the previous one returns, and a check that fails to resolve the issue sends the system to cross-check related ones instead of escalating immediately.

The definitional split worth using

Anthropic's engineering post Building Effective AI Agents draws the line in two sentences:

"Workflows are systems where LLMs and tools are orchestrated through predefined code paths."

"Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks."

Anthropic treats both as agentic systems, so this is an architectural split inside one family, not two unrelated technologies. On when to reach for the agent side, the post is specific: "Agents can be used for open-ended problems where it's difficult or impossible to predict the required number of steps, and where you can't hardcode a fixed path."

The post does not tell you to avoid agents. It sets a ladder: find the simplest solution possible, increase complexity only when needed, and add it only when it demonstrably improves outcomes. Anthropic also observes that for many applications, optimizing single LLM calls with retrieval and in-context examples is usually enough.

What you are trading away

Harrison Chase, writing on the LangChain blog, puts the tradeoff in one line: "Workflows give you more predictability at the expense of autonomy, while agents give you more autonomy at the expense of predictability." He adds the part most comparisons skip: "Notably, when building agentic systems we are in pursuit of reliably good outcomes, which neither predictability or autonomy alone guarantee."

Workflow complexity lives in the graph, in branching logic and parallel edges expressed in some DSL. An agent is different: "Agents can also contain complicated logic, but by contrast all that logic is abstracted away into natural language, which goes into the prompt. So the overall structure of an agent is simple (just a prompt + tools), though that 'prompt' can often times be pretty complex."

The architecture gets simpler, so the entire load moves into the prompt and the tool definitions. There is no graph to inspect. Anything you fail to state in words is not merely undocumented, it is undefined at runtime.

Agents break in documented ways

Arize AI's Why AI Agents Break, by Aryan Kargwal, is the counterweight. It names eight recurring production failure modes:

  • retrieval noise and context window overload
  • hallucinated arguments in tool calls
  • recursive loops and inefficient trajectories
  • guardrail failures for sensitive data
  • pre-training bias overriding retrieved context
  • unhandled external API schema changes
  • instruction drift in long sessions
  • code generation safety

Arize describes its evidence base as a year of watching agents in production, "After analyzing millions of decision paths", and says the failures are not random. It publishes no failure rates and no ranking, so treat the eight as a checklist, not a leaderboard.

Three of them argue directly for writing a spec first.

Hallucinated tool arguments. Arize's example: an agent assumes a database field is user_id because that is what it saw in training, while your schema requires customer_uuid. The query returns zero rows rather than an error, and the agent tells the user it could not find any data. Nothing in your logs looks wrong.

Inefficient trajectories. Arize calls it the Polling Tax: an agent loops on a status check instead of waiting for a webhook. The answer arrives and is correct, but the path taken makes the agent commercially unusable. As Arize puts it, "You will see a stream of 200 OK responses."

Guardrail failures. Arize is blunt: "Prompts are suggestions. They lack the rigidity of code." Its remedy is a deterministic layer that inspects the output and can block it before a user sees it. Its illustration is the Replit rogue agent incident of July 2025, which Arize sources to Fortune's reporting: a developer instructed the agent not to touch the production database, and it executed a DROP TABLE anyway. The facts are Fortune's, the framing is Arize's.

Anthropic lands in a similar place: it recommends extensive testing in sandboxed environments with appropriate guardrails, because agent autonomy means higher costs and the potential for compounding errors.

What an agent spec must pin down

Four things have to be settled in writing before anyone builds.

  1. The reasoning loop and its iteration ceiling. Name the pattern, describe the perceive, decide, act, observe cycle, and set a hard maximum number of iterations. Without a ceiling, the Polling Tax has nowhere to stop.
  2. Each tool's contract. For every tool: what it does, the specific observable condition that triggers it, the exact shape of what it returns including field names and types, and what happens when it fails. Writing the field names down is what prevents the user_id versus customer_uuid class of silent failure.
  3. Stop conditions. Every halt predicate, written so a machine can check it. Success, failure, budget exhausted, ambiguity that needs a person. If a stop condition needs interpreting, it is not a stop condition.
  4. Guardrails. Hard rules with a stated reason, covering at minimum data handling, scope, whether the agent may claim to be human, and any irreversible action. Attach an escalation trigger to each, and enforce the ones that matter outside the prompt.

Note what is not on that list: the sequence of steps. You are specifying the boundaries of a loop, not its contents.

Where this fits if you use Bespoke Prompting

Bespoke Prompting's Agent build type names all four among its ten sections: REASONING_LOOP carries a MAX_ITERATIONS value, TOOL_REGISTRY requires a five part contract per tool, STOP_CONDITIONS must be machine checkable, and GUARDRAILS are written in the form "Never X because Y" with escalation triggers attached. The fallback field is the strict one: every tool needs a named recovery, something like retry once then fall back to a specific alternative, and "handle the error" is rejected.

Sources

Suggested internal links