What is context engineering?

Context engineering is the work of deciding what a language model sees each time it is called. That covers the instructions, the tool definitions, retrieved documents, memory and the message history, all of which compete for room in the context window. For an agent, the job repeats at every step, because each tool call adds new material.

Anthropic starts from a narrow definition of the input: “Context refers to the set of tokens included when sampling from a large-language model (LLM).” Context engineering is then “the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference”. That includes “all the other information that may land there outside of the prompts” anthropic.com.

LangChain puts it in terms of outcomes. Harrison Chase’s definition reads: “Context engineering is building dynamic systems to provide the right information and tools in the right format such that the LLM can plausibly accomplish the task” langchain.com. His reasoning is that when an agent fails, the model is usually not the cause. “Most of the time when an agent is not performing reliably the underlying cause is that the appropriate context, instructions and tools have not been communicated to the model.”

Who coined the term context engineering, and when?

Nobody has a documented claim to coining it, and the term spread in June 2025. The earliest dated use we found is the line “Everything is context engineering” in Dex Horthy’s 12-factor agents guide, in a GitHub commit from early April 2025 github.com. On 12 June 2025, Walden Yan at Cognition published a post with a section headed “Principles of Context Engineering” cognition.com. Yan described it as the step after prompt engineering: “It is about doing this automatically in a dynamic system.”

Later that month, two posts on X pushed the term further. Shopify CEO Tobi Lutke wrote that he preferred “context engineering” over prompt engineering. “It describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM” x.com. Andrej Karpathy added his “+1” for the term x.com. In the same post, which Simon Willison quoted at length, he called it “the delicate art and science of filling the context window with just the right information for the next step” simonwillison.net.

Willison’s note, which quotes both posts, is dated 27 June 2025. He wrote that the term “has recently started to gain traction as a better alternative to prompt engineering”. Chase’s LangChain post of 23 June credits “recent takes” from Lutke, Ankur Goyal and Yan. He also wrote that the practice predates the name: “agent builders have been doing it for the past year or two.”

Horthy’s guide, under its factor 3 on owning your context window, now notes that the term became popular about two months after the guide was published github.com/humanlayer. Research followed within weeks. A survey by Mei and colleagues, submitted to arXiv on 17 July 2025, reviewed “over 1400 research papers” under the new heading arxiv.org. Anthropic published its own guide on 29 September 2025.

What is the difference between context engineering and prompt engineering?

Prompt engineering is writing the instructions. Context engineering decides everything else that lands in the window as well, including tool results, retrieved files, memory and history, and it is redone on every turn of an agent.

Anthropic calls context engineering “the natural progression of prompt engineering” and draws the line by timing. “In contrast to the discrete task of writing a prompt, context engineering is iterative and the curation phase happens each time we decide what to pass to the model” anthropic.com. Chase goes further: “I would also argue that prompt engineering is a subset of context engineering.” A prompt written for one input still matters. The new work is formatting “a set of dynamic data” that changes with each case langchain.com.

In a refund agent, prompt engineering tunes the system prompt: tone, steps, output format. Context engineering decides which customer record, order history, refund policy clause and tool results reach the model for this particular refund, and what gets dropped when the conversation runs long.

What goes into an agent’s context window?

An agent’s context is assembled from six or so parts, each with its own source and its own way of going wrong. Karpathy’s list, as quoted by Willison, covers “task descriptions and explanations, few shot examples, RAG, related (possibly multimodal) data, tools, state and history, compacting” simonwillison.net. The LangChain docs group the same material into model context, tool context and life-cycle context docs.langchain.com.

Neither source says who should be allowed to change each part. For a business agent, that decides whether a wrong answer can be traced and fixed.

The parts of an agent's context: where each comes from, how it fails, and who should own changes (our synthesis from Anthropic, LangChain and Breunig, October 2026)
Part of contextWhat it holdsWhere it comes fromHow it typically goes wrongWho should own changes
System instructions Role, steps, tone, output formatWritten by the agent builderEdge cases piled into the prompt until it is long, brittle and contradicts itselfThe engineering team, through code review
Tool definitions Names, descriptions and parameters of the tools the agent may callThe agent code or MCP serversToo many overlapping tools, so the model picks the wrong oneThe engineering team
Retrieved knowledge Document passages or records fetched for this stepSearch, RAG or tool calls at run timeSimilar but wrong passages, or an outdated version of a documentWhoever owns the source system
Message and tool history The conversation so far and the results of earlier tool callsAccumulates as the agent runsGrows until it distracts the model or a past error keeps getting repeatedThe agent, within limits set by compaction rules
Memory Notes, user facts and past episodes stored outside the windowWritten by the agent or a memory serviceA stale or wrong memory is loaded where it does not belongThe agent for its own notes; a reviewer for shared facts
Business rules Policies, thresholds and exceptions the agent must applyPolicy documents, SOPs and decisions people madeTwo versions of a rule both present, with nothing saying which one governsA named business owner who approves each change
Trail Approved business rules for the case, each cited to its source, drawn from a graph of policies, records, conversations and SOPsServed to the agent over an MCP server, LangChain or LlamaIndex retrievers, or REST and GraphQLCovers company rules and context; its product pages do not mention managing tool definitions, message history or compaction, so those stay with the frameworktrailhq.comA named approver; agents can report a problem but cannot approve a rule

Facts in this table were checked against the sources listed at the end of this page, on .

How memory is stored and who writes it is covered in our guide to agent memory. Where each part sits in a full agent design is in our AI agent architecture guide.

How does context go wrong?

Context fails in two broad ways: the model lacks something it needs, or it gets too much, including material that is irrelevant, wrong or contradictory. A bigger window helps with the first and makes the second more likely.

Drew Breunig’s June 2025 post names four ways long contexts fail dbreunig.com.

  • Context poisoning: “When a hallucination makes it into the context” and the agent keeps referring back to it.
  • Context distraction: “When the context overwhelms the training”, so the agent repeats past actions instead of planning.
  • Context confusion: “When superfluous context influences the response”, such as a tool the task does not need, because “if you put something in the context the model has to pay attention to it”.
  • Context clash: “When parts of the context disagree”, for example two versions of the same instruction.

Two studies show that length alone hurts. In 2023, Liu and colleagues found that performance “significantly degrades when models must access relevant information in the middle of long contexts” arxiv.org. Chroma tested 18 models in July 2025 and reported that “their performance grows increasingly unreliable as input length grows” trychroma.com. Anthropic calls this context rot and concludes that context “must be treated as a finite resource with diminishing marginal returns.” Its rule for the job is “finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome” anthropic.com.

Context that an agent rewrites for itself has its own failure. The ACE paper from October 2025 names “context collapse, where iterative rewriting erodes details over time” arxiv.org. Its fix treats contexts “as evolving playbooks” that are updated in small steps instead of being rewritten whole. Window sizes and token costs are covered in our post on token limits.

How do you do context engineering for AI agents?

Start by tracing what the model sees at each step, then apply four moves: write context outside the window, select what each step needs, compress what accumulates, and isolate heavy work. LangChain groups the techniques into those “four buckets” langchain.com. Anthropic’s guide gives its own versions of each.

Context engineering an agent, in working order (sequence ours; techniques from LangChain, Anthropic and Manus)

Step list
  1. Trace and measure firstLangChain's first step is to "track token-usage across your agent" and to have a way to test whether a change helps or hurts. Without a trace you are guessing which part of the context caused a mistake.
  2. Write context outside the windowSave plans, progress and findings to a scratchpad, a file or a memory store. Anthropic calls this structured note-taking: the agent writes notes outside the window and reads them back later.
  3. Select what this step needsLoad only the relevant facts, examples and tools. Anthropic describes "just in time" loading, where the agent keeps file paths or stored queries and fetches the data with a tool when it needs it.
  4. Keep tools few and distinctAnthropic counts bloated tool sets among the most common failures it sees: tools that cover too much, or leave it unclear which one to use. If an engineer cannot say which tool fits a situation, the agent cannot be expected to do better.
  5. Compress what accumulatesSummarize or trim long histories. Anthropic defines compaction as summarizing a conversation near the window limit and restarting with the summary; clearing old tool results is one of its lightest forms.
  6. Isolate heavy workGive sub-agents their own windows. In Anthropic's description each one returns a condensed summary, often 1,000 to 2,000 tokens, instead of everything it read.
  7. Keep errors and stable prefixesManus advises leaving failed actions in the context so the model avoids repeating them, and keeping the start of the context stable so the KV-cache can be reused.

Most of these moves have framework support. LangGraph lets each node choose what part of the state the model sees, which our LangGraph explainer covers. Anthropic’s platform adds context editing, which “automatically clears stale tool calls and results from within the context window when approaching token limits” claude.com. Manus argues that “the KV-cache hit rate is the single most important metric for a production-stage AI agent”, because cached prefixes cut cost and latency manus.im.

Where do business rules fit in an agent’s context?

Business rules often end up pasted into the system prompt as a growing list of special cases. Anthropic advises against that pattern. It says “teams will often stuff a laundry list of edge cases into a prompt in an attempt to articulate every possible rule the LLM should follow for a particular task. We do not recommend this.”

For a finance or operations agent, a rule needs more than relevance to belong in the context. It needs a source someone can check, an owner who approved it, and a version, so that when two documents disagree the agent knows which one governs. That is Breunig’s context clash, and retrieval by similarity does not say which version governs. Our guide to rule engines for finance and operations agents compares the ways teams store and serve those rules.

How Trail supplies approved rules to an agent’s context

Trail, the memory and context layer for enterprise AI agents, handles the business-rules row of the context table. Its Context Graph holds “Five kinds of context, connected and cited”, and “Every node and edge carries the document, page and text it was drawn from” trailhq.com. When an agent asks a question, “What comes back is a short, ordered set of rules”, “each cited, each scored” trailhq.com.

For conflicts, the ranking rule is: “Approved beats proposed; a current source beats one that a later document superseded.” The how-it-works page lists “Conflicts flagged, never resolved silently”, and where confidence is low, Trail “says so and names what’s missing”. Your team sets the threshold below which an agent must stop and ask. Approval comes before any of it reaches an agent. The Rule Engine page says “Nothing becomes available to an agent because a model inferred it”, and each rule records “A named approver, with a timestamp” trailhq.com.

Trail’s comparison page works one invoice decision both ways and reports “1,900 tokens an invoice instead of 128,000”. In that example it “sends the handful of rules that apply to the invoice in front of the agent” trailhq.com. Agents connect through an MCP server, LangChain and LlamaIndex retrievers, or REST and GraphQL trailhq.com. For other kinds of context layer, see our roundup of context layer tools for enterprise AI agents.

Where it is not the fit: we found no mention on the Context Graph or Rule Engine pages of managing tool definitions, message history or compaction. Those stay with the agent framework. Coding agents get a separate open-source tool from the same team, Trail-code, which keeps “what your agents learn and decide as files in your repo” trailhq.com. A research agent with no written business policy has little use for an approval queue, and every proposed rule takes an owner’s time to review.

What could we not confirm about the term’s history?

  • Who coined “context engineering”. No source we read claims it. The earliest dated use we found is the 12-factor agents README in early April 2025, and a commit date shows when the change was made, not when readers first saw it. Earlier, unrelated uses of the phrase may exist.
  • The exact posting dates of Tobi Lutke’s and Andrej Karpathy’s posts as X displays them. The static pages we fetched show no date, so we give June 2025 and rely on Willison’s 27 June note, which quotes both.
  • Ankur Goyal’s take, which LangChain credits. We did not verify the linked post, so it is not quoted.
  • A shared, published benchmark for context engineering as a whole. The studies we found measure parts of it, such as long-context recall.

How we sourced this context engineering explainer

We read the earliest dated uses we could find: the 12-factor agents guide and its GitHub commit history, Cognition, Tobi Lutke and Andrej Karpathy on X, and Simon Willison’s note. We also read LangChain’s two 2025 posts and its current docs, and Anthropic’s September 2025 guide and context management article. We read the research on long-context failure (Lost in the Middle, Chroma’s Context Rot report, Breunig’s failure modes), the Mei survey, the ACE paper and Manus’s engineering post. Every Trail claim comes from the trailhq.com homepage and its Context Graph, how-it-works, Rule Engine, comparison and Trail-code pages. Everything was fetched and checked on 30 October 2026. The context-parts table and the order of the steps are our synthesis; we did not build or test an agent for this page. Trail publishes this page.

Frequently asked questions

What is context engineering?

Context engineering is designing what a language model sees on each call: the instructions, tool definitions, retrieved documents, memory and message history that fill its context window. LangChain defines it as building dynamic systems that give the model the right information and tools in the right format to plausibly do the task. For agents, it is repeated at every step, because the context changes as the agent works.

What is the difference between context engineering and prompt engineering?

Prompt engineering is writing and organizing the instructions given to a model. Context engineering covers the instructions plus everything else that lands in the window, such as tool results, retrieved files and memory, and it is redone on every turn of an agent. Anthropic calls it the natural progression of prompt engineering, and LangChain's Harrison Chase calls prompt engineering a subset of it.

How do you do context engineering for AI agents?

Trace what the agent sees at each step, then change one part at a time and measure the effect. The usual techniques are to write notes outside the window and select only the facts and tools a step needs. Long histories get compressed by summarizing or trimming, and heavy work is isolated in sub-agents with their own windows.

Who coined the term context engineering?

No one has a documented claim to it. The earliest dated use we found is the line "Everything is context engineering" in Dex Horthy's 12-factor agents guide on GitHub, in early April 2025. Walden Yan's Cognition post of 12 June 2025 used it as a heading, and Tobi Lutke and Andrej Karpathy backed it on X later that month. LangChain and Anthropic then wrote it into their guidance.

Does a bigger context window remove the need for context engineering?

No. Chroma tested 18 models in July 2025 and found performance grew less reliable as input length grew. A 2023 study found models do worse when the fact they need sits in the middle of a long input. A larger window holds more, but each extra token still competes for the model's attention.

Where should company rules go in an agent's context?

Keep them out of a long system prompt and load only the rules that govern the current case, each with its source and owner. Trail, the memory and context layer for enterprise AI agents, does this for business agents: it returns approved rules for the case, each cited to the document it came from.

Sources checked

Show 24 sources