Context Engineering: What It Is, How to Work with LLM Agent Context, and Why You Get Paid for It

Context engineering is a set of strategies for collecting and maintaining optimal context during LLM inference, where optimal context is the minimal set of useful tokens. It differs from prompt engineering, which only handles instructions, whereas context engineering manages everything the model receives along with the instruction: history, documents, tools, memory, and state. RAG brings in external knowledge, MCP standardizes connections to data and tools, and context engineering decides how to assemble all of this into context for the agent's next step. An agent's context consists of several layers: user query, system prompt, tools, few-shot examples, data from RAG, message history and tool calls, memory, and current state (scratchpad). Although modern LLMs have large context windows, Chroma's report "Context Rot" showed that as the used context grows, quality can degrade even on simple tasks. Drew Breunig identified four typical context failure scenarios: context poisoning (an error enters context and repeats), context distraction (necessary information is lost among irrelevant data), context confusion (the model gets confused among similar tools/documents), and context clash (conflicting data without priority rules). LangChain proposes four actions for managing context: write (save important information to external state), select (choose relevant content), compress (compress), and isolate (move to a separate context).