I was taking a break from the AIE Workshops on Monday and stepped out by the food stands to check out the crepes. That's when I saw a line literally wrapping the entire length of the Moscone West windows looking out onto Fourth Street. I couldn't imagine what a several-hundred-person line was for, and when I went to ask, they told me it was for the Context Engineering Workshop. That sent me down a rabbit hole exploring and understanding and learning. So now, for you, I will share what I got.
For the past couple of years, the AI world was obsessed with prompt engineering, aka the art of speaking to a machine. But as developers move from simple chatbots to complex autonomous agents, a new discipline has taken center stage: context engineering.
Mike Swift (@theycallmeswift), CEO of Major League Hacking (@mlhacks) , gave me some critical background. He pointed me to Dex Horthy of HumanLayer (who is actually speaking later this week), who basically coined the term at the first AI Engineer World's Fair. Dex's core thesis, said Swift, is that "agents get bad after about 100,000 tokens," which represents roughly 10% of their total available context window.
So context engineering is essentially managing an AI's working memory. Context engineering, Swift noted, is "managing how many times the loop goes around to how much you have to remember every time you do it." It is a counterintuitive concept for humans; the more we talk about a subject, the deeper our shared understanding becomes. But models work the opposite way. They lose focus as their context window fills up.
For many developers, meticulously curating this working memory is a practical necessity. I sat down with Ben Halpern (@ben), founder in residence at MLH and co-founder of DEV, who told me that context engineering is the "latest frontier of the optimization point" where developers can leverage their expertise. Beyond just keeping models coherent, Ben pointed out that developers who are doing product work manage context for "latency and cost effectiveness."
During our chat, I tested a somewhat spicy thesis I'd been developing: that context engineering feels like a proxy for control, perhaps even our "last real attempt at managing control" before models outpace us. Ben agreed that there is a massive opportunity for human impact here, while not fully buying into my thinking. He noted that because "model progress is going to be slower than the tooling progress," context engineering allows developers to "reduce the feeling of models as a black box." Instead of waiting for the next foundational model to magically fix memory limitations, developers can actively add value to workflows today.
Next, I talked with Dominik Kundel, who works on developer experience for Codex. Dominik argued that while strict context management is valuable, for example, in narrow, customer-facing support bots, it actively hinders general-purpose agents.
When developers meticulously craft context windows, Dominik said, "What you're doing is shrinking and limiting the possible range of tasks that it can solve." Instead of filtering what the model sees, Dominik advocates for progressive discovery. "I don't filter anything," he said, noting that Codex connects directly to vast, unstructured sources like Slack, Gmail, and Notion so that it can learn to navigate the noise on its own.
This unstructured approach enables what he calls "vague prompting." Because the agent has access to everything, you can talk to it like a colleague and ask it to "fix the thing that Karen asked me about yesterday." The agent will independently figure out which Karen is relevant, locate the Slack thread, and pull the necessary context. When I brought up my thesis about context engineering acting as a proxy for human control, Dominik acknowledged that strict management definitely makes outputs more predictable. However, he warned that this "determinism results in reduction capabilities" and a severe loss of flexibility.
Ultimately, walking away from Moscone West last night, it was clear that context engineering is far from a one-size-fits-all discipline. Whether a developer chooses to heavily filter an agent's working memory to maintain a human thumbprint or embrace progressive discovery to unlock vague prompting, the era of treating an AI's context window as an endless dumping ground is definitively over.
Top comments (6)
context windows as a design constraint changes how agent workflows work - less "what should it do" and more "what does it actually need to see." spent two weeks rebuilding a workflow around this last month.
I keep getting stuck on how progressive discovery squares with that 100k number. If models start slipping around 100k tokens, then "I don't filter anything" can't really mean everything lands in the window, right? It reads more like the filtering moves from the developer to the model's own retrieval when it decides to pull something, so the working set stays small even while the available context is huge. Is that how you'd frame it, that Dominik isn't skipping curation so much as handing it to the agent? Asking because those two ideas sound opposed until you separate available context from what's actually in the prompt.
The strict-vs-progressive framing captures the retrieval axis cleanly. What's missing is the second axis, which shows up once you have more than one context source arguing with itself.
Progressive discovery works when the agent's job is "find the relevant Karen and pull her Slack thread." It stops working when the question is "which of these two Karen policies overrides the other in this task type." That is not a retrieval problem. It is a precedence and lifecycle problem, and the agent cannot infer it from vector similarity or vague prompting.
So context engineering is not one discipline. It is two: retrieval (what the model sees) and lifecycle (what supersedes what, what is locked, what is stale). The strict camp is doing retrieval by hand and lifecycle by accident. The progressive camp is doing retrieval by discovery and lifecycle not at all. The interesting move is separating the layers on purpose.
Context engineering is the right thing to focus on, and worth being precise about what it buys you. Feeding the model the right files, rules, and examples decides whether it can get the answer right; it doesn't decide whether it does. Context raises the ceiling; it doesn't stop the agent from making a locally reasonable call the context didn't rule out. The piece that closes that gap is enforcement, checks the output has to pass regardless of what was in the window. That's the layer we're building Mneme on, the constraint side of context.
The context engineering discussion always focuses on the "within-session" challenge â how much to pack into the window before the model starts hallucinating. But the across-session problem is arguably bigger. Even with perfect context curation, every new session starts from zero unless you have a persistence layer that knows what to carry forward and what to discard. Dex's "100K token ceiling" is real, but the bottleneck I keep hitting is that the ceiling resets every conversation. You can engineer the perfect context for today's chat, and tomorrow the agent has no memory of why that context was structured that way. That's the piece I think progressive discovery models (and even strict curators) are going to bump into next.
Context engineering is becoming closer to systems design than prompt writing. The important work is deciding what enters memory, what expires, and what must be verified every time.