For a while, the craft of working with language models was called prompt engineering: find the right wording and the model behaves. That framing breaks down the moment you build an agent. An agent runs for dozens or hundreds of steps, calls tools, reads documents and accumulates history. The question stops being "what should the prompt say?" and becomes "what should the model be looking at right now?"
That second question is context engineering: designing and managing everything that enters the model's context window on each step, so it has what it needs and nothing that gets in the way.
What counts as context?
The context window is the model's entire working memory for a single call. It is made of several ingredients, and each one is a design decision:
- System instructions: the role, rules, tone and constraints.
- Tool definitions: names, descriptions and schemas of what the agent can do.
- Retrieved knowledge: documents, database rows, search results pulled in on demand.
- Conversation and action history: earlier turns, tool calls and their outputs.
- Memory: facts and notes persisted from earlier sessions.
- The current task: the user's request and any structured state.
Why more context is not better
Context is a finite, costly resource. Every token competes for the model's attention, so stuffing the window usually hurts. Practitioners often call the effect context rot: as the window fills with stale tool output, irrelevant documents and abandoned lines of reasoning, accuracy and instruction-following tend to degrade, and costs and latency climb. The goal is the smallest set of high-signal tokens that makes the right next action likely.
Core techniques
1. Just-in-time retrieval
Instead of preloading everything, give the agent lightweight pointers (file paths, IDs, search tools) and let it fetch details only when needed. This keeps the window lean and lets the agent explore progressively, discovering what matters as it works.
2. Compaction and summarization
When a long task nears the window limit, summarize the history: keep decisions, open questions and key facts; drop raw tool output and dead ends. A good compaction preserves what the agent must not forget and discards what it can re-fetch.
3. Structured note-taking and memory
Let the agent write notes outside the window (a to-do list, a scratchpad file, a memory store) and read them back when relevant. This gives long-horizon work continuity across resets without carrying everything in-context.
4. Sub-agents with isolated context
Delegate a focused subtask, such as "research this vendor", to a sub-agent with its own clean window. It may explore widely, but returns only a short, distilled result to the main agent, so the parent's context stays uncluttered.
5. Tool design and prompt caching
Tools are context too. Fewer, clearer, non-overlapping tools with precise descriptions beat a sprawling menu. Keeping stable content (system prompt, tool list) at the front of the prompt also lets you take advantage of provider prompt caching to cut cost and latency.
Select
Choose what is relevant
Compress
Summarize and trim
Isolate
Sub-agents, scoped windows
Persist
Notes and memory outside the window
Context engineering in practice
When we build enterprise agents, context decisions are made explicit and tested:
- We define, per workflow, which data sources an agent may read and in what order.
- We measure answer quality as context size grows, and set compaction thresholds from evidence.
- We log the exact context sent on each step, so a bad outcome can be traced to a bad input.
- We treat retrieved text as untrusted input, a basic defence against prompt injection.
Context engineering is one half of building dependable agents. The other half is the surrounding system that runs the model, covered in harness engineering.