Guide

When a single AI prompt is enough, and when it isn't

Published 14 September 2026

A single prompt is enough when you can describe the whole job up front and the model needs nothing from the outside world to do it. You need something more the moment the work has to call a tool, react to a result you cannot predict, run more than once, or remember anything between runs.

The default should be the prompt. Anthropic's engineering post "Building Effective AI Agents" states it directly: "When building applications with LLMs, we recommend finding the simplest solution possible, and only increasing complexity when needed." The same post adds that "For many applications, however, optimizing single LLM calls with retrieval and in-context examples is usually enough."

That is not the same as saying agents are a bad idea. Anthropic's position is a ladder: start with one call, and add complexity only when it demonstrably improves outcomes.

What a prompt can carry

A prompt works when three things are true at once: everything the model needs is in your head or pasteable into the message, there is exactly one output and you are the one who reads it, and you can predict the shape of the work in advance.

  • Everything the model needs is either in your head or pasteable into the message. If the answer depends on today's inventory numbers, a customer record, or a file you have not opened, a prompt on its own cannot reach them.
  • There is exactly one output, and you are the one who reads it. Nobody downstream is waiting on a machine-shaped result.
  • You can predict the shape of the work in advance. Anthropic draws the contrast on exactly this axis: agents suit open-ended problems where it is difficult or impossible to predict how many steps will be needed, and where you cannot hardcode a fixed path. If you can hardcode the path, hardcode it.

Four things break a single prompt

A single prompt breaks when the work has to call a tool or fetch outside data, react to a result you cannot predict, run more than once on a trigger or a schedule, or remember anything between runs.

What the work needs Why one prompt cannot do it
To call a tool or fetch outside data The information does not exist at the moment you write the prompt
To react to a result you cannot predict The next instruction depends on an answer nobody has yet
To run more than once, on a trigger or a schedule Something has to decide when to fire and what to pass in
To remember anything between runs A prompt starts from nothing every time it is sent

Anthropic names the same boundary from the other side. The building block above a plain model call, in their framing, is the augmented LLM: a model enhanced with augmentations such as retrieval, tools, and memory. Cross any one of those three and you are no longer writing a prompt, you are specifying a system, even if you have not admitted it yet.

The same job, twice: summarising a document

Summarising one contract you paste in is a prompt. Summarising every new contract in a folder, every night, is not, because nothing about the summarising gets harder but everything around it does, and that one change of scope trips all four triggers at once.

Summarise this contract and flag anything unusual in the termination clauses. You paste the contract in, you read the answer, you are done. That is a prompt, and a good one if you wrote it properly.

Now change the scope. Summarise every new contract that lands in this folder, every night.

Nothing about the summarising got harder. Everything around it did. Something has to notice the folder changed. Something has to read files, which is a tool call. Something has to know which contracts it already handled, which is memory that survives between runs. And something has to decide what happens when one of the files is a scan that will not parse, which is reacting to a result you could not predict when you wrote the instructions. One change of scope tripped all four triggers at once.

So do not ask how hard the thinking is. Ask what has to be true around the thinking for it to happen without you sitting there.

What the extra complexity costs

Anthropic is explicit that agentic systems often trade latency and cost for better task performance, and that you should consider when this tradeoff makes sense. They are equally explicit about the failure mode: the autonomous nature of agents means higher costs, and the potential for compounding errors, which is why they recommend extensive testing in sandboxed environments along with appropriate guardrails.

Compounding errors is the one people underestimate. A prompt that gets it wrong gets it wrong once, visibly, in front of you. A system that gets it wrong on step two carries that error through every step after it, at three in the morning, with nobody reading the output.

If you are doing the task once or twice, write the prompt. A system that runs itself only pays back over repetition.

Crossing the line does not waste the prompt

Writing a precise prompt is not wasted work whichever side of the line you end up on. Structurally the jump is smaller than it looks. Writing on the LangChain blog, Harrison Chase describes the difference in terms of where the complexity lives: in a workflow it sits in the graph, expressed in some DSL, while in an agent it is abstracted into natural language in the prompt, so the overall structure of an agent is simple, just a prompt and its tools. His framing of the tradeoff is that workflows give you more predictability at the expense of autonomy, while agents give you more autonomy at the expense of predictability.

Anthropic's own compact definition of an agent, in their context engineering post, is LLMs autonomously using tools in a loop. Neither description replaces the prompt with something else: Chase puts it at the centre explicitly, and the loop Anthropic describes is still a model being told what it is doing.

What does change is that a running loop keeps producing material that might matter next turn. Anthropic's Applied AI team frames context engineering as the natural progression of prompt engineering for that reason, and describes the shift as needing strategies to manage the entire context state once agents operate over multiple turns and longer time horizons: system instructions, tools, Model Context Protocol, external data and message history. That management problem is what you are buying when you cross the line. Worth buying when the job repeats, dead weight when it does not.

Where this fits if you use Bespoke Prompting

Bespoke Prompting's Prompt build type emits six fixed sections in a fixed order: ROLE, CONTEXT, CONSTRAINTS, SUCCESS CRITERIA, APPROACH, TASK. Those six are also a fast test of the fork in this article. If you cannot fill in CONTEXT without saying it depends on whatever the tool returns, or cannot write SUCCESS CRITERIA without referring to something a previous run produced, your task is not a prompt and no amount of rewriting will make it one.

Sources

Suggested internal links