Guide

Context Engineering vs Prompt Engineering: what changed

Published 26 September 2026

Anthropic has published its own answer to this question, and it is not the version circulating in most posts. Context engineering is not a replacement for prompt engineering, and it is not a rebrand. The real change is in what you are responsible for curating: not just the instruction you wrote, but every token that lands in the window alongside it.

The two definitions, as Anthropic actually states them

In "Effective context engineering for AI agents", Anthropic's Applied AI team writes: "At Anthropic, we view context engineering as the natural progression of prompt engineering."

The post defines the two terms as follows:

Prompt engineering Context engineering
Anthropic's definition Methods for writing and organizing LLM instructions for optimal outcomes "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference, including all the other information that may land there outside of the prompts."
What you manage Primarily system prompts System instructions, tools, Model Context Protocol (MCP), external data, and message history

That last clause is the load-bearing part. Outside of the prompts. Your carefully written system prompt is one item on that list.

The tidy summary that prompt engineering is about how you ask while context engineering is about what the model knows when it answers is a serviceable shorthand. It is not Anthropic's wording and does not appear on that page.

Why the shift happened: agents generate their own context

The shift happened because agents generate their own context. An agent runs in a loop, calling tools and reading results, and the data that loop produces, which nobody wrote, lands in the window. Prompt engineering optimizes the text you authored. Context engineering also has to govern the text you did not.

Early on, prompting was the biggest component of the work because most non-chat use cases needed a prompt optimized for one-shot classification or text generation. You wrote the instruction, you got the output, you were done.

Anthropic offers a deliberately plain definition of agents in the same post: "we've gravitated towards a simple definition for agents: LLMs autonomously using tools in a loop." The consequence, in their words: "An agent running in a loop generates more and more data that could be relevant for the next turn of inference."

Nobody wrote that data. The agent produced it by calling tools and reading results, and it lands in the window whether or not it deserves to be there. That is what gets added: prompt engineering optimizes text you authored, context engineering also has to govern the text you did not.

The attention budget

Anthropic's argument for why this is a real constraint rather than a tidiness preference: "LLMs have an 'attention budget' that they draw on when parsing large volumes of context." Every new token depletes it. The post traces that limit partly to the transformer architecture, where every token attends to every other token, which results in n squared pairwise relationships for n tokens. The conclusion follows: "Context, therefore, must be treated as a finite resource with diminishing marginal returns."

One rule from the post matters most: "Good context engineering means finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome." Smallest, not largest. That is the opposite of the instinct most people bring to a very large context window.

Anthropic's Context windows documentation names the phenomenon: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot. This makes curating what's in context just as important as how much space is available." Everything counts toward that window: tool definitions, tool results, images, documents and generated output. Cached prefixes still occupy it.

The independent evidence

Chroma's technical report, "Context Rot: How Increasing Input Tokens Impacts LLM Performance", dated July 14, 2025, evaluated 18 models across several long-input tasks. Four findings should change your behavior:

  • Distractors are expensive. Chroma reports that "Even a single distractor reduces performance relative to the baseline (needle only)".
  • Semantic distance compounds with length. As the similarity between the question and the target passage falls, performance degrades more sharply as input grows.
  • Coherent filler is worse than incoherent filler. Chroma found that "models perform worse when the haystack preserves a logical flow of ideas". Shuffled haystacks improved performance across all 18 models.
  • Focused beats complete. On the LongMemEval task, Chroma compared focused prompts averaging roughly 300 tokens with full prompts averaging roughly 113,000 tokens, and reported significantly higher performance on the focused prompts across all models.

Passing the whole conversation history because it might be relevant is a measurable cost, not a neutral safety measure.

An older result points the same way. Liu et al., "Lost in the Middle: How Language Models Use Long Contexts", TACL 2024, found a U-shaped curve: performance is highest when relevant information sits at the very beginning or the very end of the input, and degrades significantly when the model has to reach into the middle. Two caveats, because that paper is routinely overstated: its reported drop of more than 20 percent is relative, not twenty percentage points, and it tested 2023-era models.

What actually changes in your day

What changes is that you take on five curation decisions beyond writing the instruction itself: tool results, retrieved documents, conversation history, references instead of payloads, and isolated exploration in a separate window. Each is a decision about the attention budget rather than about phrasing.

The instruction-writing skills do not go away. Anthropic's prompting best practices page still teaches being clear and direct, using examples, structuring prompts with XML tags, and setting a role.

You curate tool results, not just tool descriptions. A tool that returns a full record when you needed one field spends attention budget on every subsequent turn, not just this one.

You curate retrieved documents. Anthropic's prompting docs set the threshold for long-context handling at inputs of 20,000 or more tokens, advise putting longform data above the query, and state that queries at the end can improve response quality by up to 30 percent in tests, particularly with complex multidocument inputs. The page also recommends wrapping each document in its own tags with content and source subtags, and asking the model to quote the relevant parts first.

You curate history. Anthropic describes compaction as summarizing a conversation that is nearing the limit and reinitiating a new context window with the summary. Its Compaction documentation gives the rationale bluntly: as a conversation grows, response quality degrades. In that beta feature the default trigger fires at 150,000 input tokens.

You prefer references over payloads. Anthropic describes a just-in-time approach where the agent holds lightweight identifiers such as file paths, stored queries and web links, and loads the actual data at runtime through tools. Their analogy: people build file systems, inboxes and bookmarks rather than memorizing entire corpuses.

You isolate exploration. In the sub-agent pattern Anthropic describes, a specialist works in its own clean window, potentially spending tens of thousands of tokens, and returns only a condensed summary, often 1,000 to 2,000 tokens, to the coordinating agent.

None of those are decisions about phrasing. They are decisions about the budget.

Where this fits if you use Bespoke Prompting

The Agent build type pushes these choices to the front rather than leaving them to runtime. Its TOOL_REGISTRY gives every tool a five-part contract, and two of the five are context decisions: what it returns, stated as the exact shape, fields and types, and when to call it, stated as a specific observable condition rather than "when needed". MEMORY_STRATEGY forces the same call on history, with the engine's own defensive bias when the answer is unclear: choose the simpler kind, none over in-session over persistent.

Sources

Suggested internal links