AI Models vs Agentic Harnesses: What Drives

Here's a question worth sitting with: if two AI products are running the exact same underlying model, why do they feel so completely different?

You've probably experienced this without having a name for it. The same model that feels like a slightly smarter autocomplete in one app becomes something almost eerily capable in another—capable of spending hours autonomously debugging a codebase, or chaining together dozens of steps to complete a task you'd normally block out an afternoon for. Same brain. Very different behavior.

IBM's Martin Keen put a useful frame around this in a recent explainer video, drawing a clean line between two things that usually get lumped together: the AI model and the agentic harness. It's a distinction that sounds technical but has real implications for how anyone—user, developer, buyer, skeptic—should think about what AI systems can and can't do.

The brain in a jar

Start with the model itself. When people say "ChatGPT" or "Claude," they're usually gesturing at a product—an interface, a set of behaviors, a general vibe. But underneath is an artificial neural network: a thing that takes text in and produces text out, trained on vast amounts of data. That's the model.

What a model can't do, on its own, is act. It can't open a file. It can't run code. It can't browse the web. As Keen puts it: "On their own, these AI models are essentially a brain in a jar. They're capable, but they're trapped."

That's not a flaw—it's just what a language model is. The confusion arises because we rarely interact with raw models anymore. We interact with products, and products have infrastructure. That infrastructure is what Keen calls the agentic harness.

The timing of this framing matters. Benchmark scores between the leading frontier models have converged significantly; the top labs are now separated by a few percentage points on most standard evaluations. Which means when one AI product meaningfully outperforms another at a real task, the model is increasingly unlikely to be the explanation. "When one AI product clearly outperforms another," Keen notes, "the brain itself—the AI model—that usually isn't the explanation. It's what's wrapped around the brain."

This is a useful corrective to how AI progress usually gets narrated—as a sequence of better and better models, each one smarter than the last. The harness side of the ledger has been quietly doing a lot of the work.

What a harness actually does

The agentic harness is the scaffolding that turns a model into an agent—something that can plan, act, observe, and iterate. Keen breaks it into three components: tools, memory, and the agentic loop.

Tools are how the model reaches outside its own context. Through the harness, a model can read and write files, execute code in a sandbox, retrieve information from the web, and—increasingly—directly control a computer interface. That last capability, sometimes called "computer use," lets the model move a cursor and interact with software the way a human would. For connecting to external services like corporate databases or third-party applications, there's an emerging standard called MCP (Model Context Protocol), which lets tools plug into any compatible harness without needing to be rebuilt from scratch each time. That kind of interoperability matters: it means the tooling ecosystem can develop independently of any single model or product.

Memory is more complicated than it sounds. Models have a fixed context window—their working memory—that gets wiped at the end of a session. The harness can work around this in several ways: loading persistent instruction files (like an agents.md configuration that tells the model a codebase's conventions at the start of each session), compressing context by summarizing earlier parts of a conversation when the window fills up, and letting the model search a project selectively rather than ingesting everything at once. The effect is something closer to continuity—the agent "remembers" in the way a contractor does when they keep notes on a project folder.

The agentic loop is where the model and harness actually collaborate. The model plans a next step; the harness executes it; the results come back; the model observes and plans again. This cycle can run for seconds or hours. Modern harnesses add verification layers to this loop—running tests mid-task, taking screenshots of intermediate outputs, sometimes spawning a separate model to review the primary model's work. A system that checks its own work, as Keen observes, "might run a lot longer without going off the rails."

The persistence shift underlying all of this—from stateless chatbots to agents that maintain context, run autonomously, and see screens—is already visible in what Anthropic, Alibaba, and others are shipping. The harness is the mechanism that makes persistence possible.

Why this matters outside the technical weeds

There's a practical question embedded in this framework that rarely gets asked clearly: when someone says "AI is good at X," what are they actually evaluating?

Keen's answer: you're evaluating a combination, not a component. "The more useful version of that question is to better define what we mean by AI," he argues. Which model? And which harness? "The model can be brilliant in one harness and get stuck in another."

This has non-obvious implications. Organizations shopping for AI solutions are often comparing model specs—parameter counts, benchmark scores, context window sizes—when the more determinative variable might be harness quality: how well the tooling is integrated, how memory is managed, how the loop is instrumented. A mediocre model in a well-engineered harness may outperform a frontier model running in a poorly designed one. This is the argument being made, with increasing force, by the harness engineering side of the industry—that the systems surrounding models matter as much as the models themselves.

It also reframes what "AI progress" means. The capabilities that felt new and impressive this year—coding agents that can work autonomously for extended periods, agents that can navigate a browser or operate software—are largely harness advances, not model advances. That doesn't make them less real, but it does change who deserves credit, and who has the relevant expertise to build the next generation of capable systems.

A boundary that moves

There's one wrinkle Keen raises that's worth holding onto. The line between what lives in the model and what lives in the harness isn't fixed—it migrates over time, and in both directions.

Capabilities that used to be pure harness work—long-horizon planning, self-verification—are increasingly being trained into the models themselves. Meanwhile, behaviors that once felt intrinsic to a model, like staying consistent across a long task, are increasingly shaped by harness configuration: project files, system prompts, conventions baked into the scaffolding.

So the "brain vs. jar" metaphor is useful precisely because it's a little unstable. The jar shapes the brain. The brain outgrows what the jar was designed for. Each improves the other in ways that make clean attribution difficult.

Which is probably the most honest thing you can say about where AI capability actually comes from right now: it's not the model, it's not the harness, it's the specific combination—and that combination is still being figured out.

Marcus Chen-Ramirez covers AI, software development, and the intersection of technology and society for Buzzrag.