The Context Problem AI Agents Can't Solve Alone

There's a version of the AI coding agent future that looks really clean on a slide: agent gets task, agent reads codebase, agent ships perfect PR. And then there's the version that actually happens, where the agent confidently nukes your legacy compatibility layer because nobody told it that code existed for a reason.

Peter Werry, co-founder at Unblocked, spent a chunk of a recent AI Engineer session dissecting exactly why that gap exists—and why the fixes most teams reach for first don't actually close it. His framing: you don't have an agent intelligence problem anymore. You have a context problem. And those require different solutions.

You used to be the context engine

Werry opens with a reframe that's worth sitting with. Four years ago, the developer was the context engine. You grabbed the issue ticket, assembled the relevant files, pointed the agent at the right language ("No, not the JavaScript dummy—the Python source code"), and managed the whole retrieval process manually. It worked, sort of, because a human who'd lived through the org's incidents and decisions was making all the judgment calls about what the agent needed to see.

The problem is that this model doesn't scale. As agents get more capable and teams try running them in parallel—or worse, in full "YOLO mode" with no human in the loop—the human becomes the bottleneck. Werry describes the cognitive experience of managing multiple parallel agents as a "really painful" context-switching nightmare, and it's hard to argue with that characterization if you've tried it.

His analogy for what good organizational context actually looks like is instructive. Think about what it means to be genuinely useful at a job after a few years: you've absorbed the codebase slowly, survived some incidents, learned which architectural decisions came from painful lessons versus convenience, and now you know what questions to ask when something breaks. That accumulated scar tissue is the context. The goal of a context engine is to give an agent some version of that without the years of suffering.

Three myths, decoded

Werry structures his core argument around three things people assume will solve the context problem—and don't.

Myth 1: Naive RAG over your docs is a context engine. Vector search over your documentation is a start, not a solution. For large organizations with multiple repos and teams, undifferentiated retrieval surfaces conflicts, pulls in irrelevant code from adjacent teams, and burns tokens without producing clarity. More critically, it runs into what Werry calls "satisfaction of search"—borrowed from radiology, where a tech finds one thing on an x-ray that explains the symptoms and stops looking, potentially missing cancer in the process. Agents do the same thing: they find something that looks like the answer in Confluence and stop, never checking the Slack thread from six months ago where the team actually debated and rejected that approach.

Myth 2: Connecting a bunch of MCP servers (Model Context Protocol—basically a standardized way to pipe data sources into an LLM) gives you understanding. Werry is pointed here: "You could just wire up a bunch of MCP servers and it's not going to be able to understand what the relationships are between all that data, how it got there, and why it is the way it is." Access to data and comprehension of data are different things. Knowing that a Slack message and a PR exist doesn't tell you that the PR reversed a decision made in the Slack message three weeks earlier.

Myth 3: A bigger context window will eventually solve this. Gemini was first to push toward million-token contexts, and it was impressive for needle-in-a-haystack retrieval—find a specific fact in a huge document, great. Cross-source reasoning, understanding contradictions, surfacing what's actually true? Much harder. And practically speaking, most organizations already have more than a million tokens' worth of relevant context. Scaling the window doesn't change the fundamental problem of knowing what to put in it.

What a context engine actually has to do

The more interesting part of Werry's talk is what he thinks a real context engine needs—not as a product pitch, but as a set of genuine hard problems.

The first is building relationships between data, not just indexing it. Linking a Slack message to a PR because someone posted a URL is easy. Understanding that repeated patterns in PR review comments represent an organizational best practice—and distilling those into retrievable "memories"—is a different class of problem. Werry describes Unblocked's approach: scrape the history of PR comments, find the repeated patterns, crystallize them into stored context, and surface them when an agent is working on similar code. That's not retrieval; that's synthesis.

The second is conflict resolution, and Werry's candor about getting this wrong initially is one of the more useful parts of the talk. Their first approach was simple recency bias: newer information wins. That broke down quickly, because people write docs and Slack messages that are aspirational, speculative, or just wrong about how the system actually works. So they layered in a "bias toward main branch code" heuristic—the code is the real source of truth. But that broke down too, because sometimes what matters most is where the codebase is going, not where it's been. The expert conversations happening right now in Slack might be more predictive of correct behavior than the current state of the repo. There's no clean universal rule here.

The third is personalization—which turns out to matter a lot more than it sounds. Werry describes a concrete technique: track which repos a developer contributes to most (via PR history), then during vector retrieval, do a deep search on those focused repos and a wider search everywhere else, biasing toward the focused set. The logic is straightforward but easy to overlook: context relevance isn't just about the task, it's about who's doing the task.

And then there's data governance, which Werry treats as non-negotiable rather than a nice-to-have. His example: private Slack channels might contain HR discussions or sensitive strategic conversations. A context engine that surfaces that information to someone who shouldn't see it isn't just a privacy violation—it's a trust violation that would end adoption immediately. The access controls baked into the original data sources have to flow through the entire retrieval stack.

The iceberg problem

The framing I keep coming back to from this talk is Werry's iceberg metaphor. Code that compiles is the visible tip—that's table stakes, the baseline. Everything actually important is underwater: the user's original intent, what the team tried and rejected, what was deleted and why, the reasoning behind decisions that look arbitrary from the outside.

"How are you going to surface that kind of content just by looking at docs and code?" he asks. The answer is: you mostly can't, with naive retrieval. The rejected approaches don't live in the codebase. The "we tried this in 2022 and it caused an outage" conversation isn't in a doc. That knowledge exists in comments, in Slack threads, in incident postmortems—distributed across sources that a vector search over your docs won't touch.

Werry's concrete demonstration of why this matters: they ran an agent on a task without the context engine, wired up to MCP servers. It did reasonably well—until it deleted legacy code that depended on an older Anthropic API method for controlling thinking token budgets. The code was there for reasons the agent couldn't see. With the context engine active, it surfaced those reasons, made the right changes, and preserved backwards compatibility. That's not a capability difference in the underlying model. It's a context difference.

Whether Unblocked's specific implementation is the right answer to these problems is a separate question. But Werry's diagnostic—that the bottleneck has shifted from model intelligence to context quality, and that most teams are solving it with tools that address the surface rather than the structure—seems hard to argue with.

The open question is how much of this can be systematized versus how much will always require the kind of judgment calls that currently take humans years to develop. Background agents running autonomously on your codebase sound great until you realize you'd be trusting a context engine to have absorbed the institutional knowledge your most experienced engineer spent a decade accumulating. How close can you actually get?

---

By Yuki Okonkwo, AI & Machine Learning Correspondent