67% Comprehension, 2% Code: Rethinking AI Dev

Here's a number that should mess with your assumptions: 2%.

That's how much of Priscila Andre de Oliveira's AI usage at Sentry was code generation. She's a senior software engineer at a company with 100,000 organizations depending on its codebase, ~100 pull requests merged per day, and 15 years of accumulated code history. She's been there over six years, uses Claude constantly, and calls it her "favorite teammate." And when she actually analyzed 116 of her own Claude sessions, code generation barely registered.

Comprehension? 67%.

That gap is the whole story.

The assumption nobody interrogates

When AI coding tools get discussed — in demos, in hype cycles, in breathless LinkedIn posts — the implicit pitch is almost always about generation. Write code faster. Ship features without touching a keyboard. The tool as a shortcut from idea to implementation.

De Oliveira's data suggests that framing misses where the actual leverage lives, at least in a production codebase. "The biggest unlock from AI in a large code base isn't generation. It's comprehension," she said in her talk at the AI Engineer conference. "I tracked my own usage and I was surprised."

This isn't necessarily a universal law — it might be a feature of the specific environment she works in. Sentry's codebase is a genuinely difficult target: fair-source, multi-office, constantly evolving, with components getting deprecated and added on what sounds like a near-daily basis. She describes coming back from vacation to a PR full of conflicts that requires real detective work just to resolve cleanly. That's a context where understanding is perpetually in short supply, and where misunderstanding something before prompting an agent to implement it is a direct path to shipping garbage.

But that context isn't unusual. Most working engineers aren't building greenfield apps in isolation. They're wading into codebases with history, with decisions nobody remembers the rationale for, with architectural patterns that made sense three engineers ago. The question isn't whether comprehension matters — it's whether we've been dramatically undervaluing it relative to generation.

What a misaligned mental model actually costs

De Oliveira puts it cleanly: "You need to also understand because maybe you need to steer the AI to go on the correct path."

This is the thing that gets glossed over when people talk about just "letting the agent run." The agent's output quality is downstream of the quality of the context and direction it receives. If you don't understand what it found during its research phase, you can't tell whether it's heading somewhere useful — and by the time the implementation lands, the error is baked in.

She frames the workflow as a specific loop: understand what the agent found → then let it plan → then let it implement. The comprehension step isn't a nice-to-have warmup; it's the quality gate. "A misaligned mental model is where slop comes from" is essentially the thesis. And "slop" here isn't just aesthetically bad code — it's code landing in a codebase that 100k organizations depend on and that, as she puts it, "pays your salary."

This connects to a broader concern that's been surfacing across the industry. Armin Ronacher, Flask's creator and a former Sentry engineer, wrote something that de Oliveira quoted directly: "When more and more people tell me they no longer know what code is in their own code base, I feel like something is very wrong here." That's not a fringe worry. When generation becomes so easy that understanding becomes optional, you get codebases that work until they don't — and nobody knows why.

The "catch me up" skill

Because her comprehension prompts kept repeating the same patterns, de Oliveira built what she calls a skill — a structured, detailed Markdown prompt saved locally, called "catch me up." It organizes comprehension questions into six exploration modes: architecture, conventions, feature trace, syntax, testing, and history.

A skill, in this context, is basically a very precise template for a particular type of cognitive task. It's not magic — it's structured prompting made reusable. What makes it interesting is the intention behind it: she's not trying to get Claude to do more; she's trying to get Claude to help her understand more, faster, so she can make better decisions about what to do next.

She demoed it with a repository she'd never contributed to before — the AI SDK testing repo, where her team had told her not to write any code at all, only prompt. She fired the skill with: "I am a new contributor. Catch me up on how this repository works." The output mapped the architecture, answered specific questions about how Sentry envelopes were handled, and gave her the orientation she needed to start contributing meaningfully without touching a line of code herself.

"If I didn't have AI, I would have to do this myself," she noted — then immediately clarified the stakes: "I don't like to just ship something I don't understand. If it's a vibe coded project, that's fine, but this is real serious business."

That "if it's a vibe coded project, that's fine" is doing a lot of work. There's an implicit taxonomy here: projects where slop is acceptable, and projects where it isn't. The problem is that people are applying vibe-coding instincts to codebases firmly in the second category.

Where this sits in the larger debate

The standard framework for agentic coding — research, planning, implementation — has been getting traction. Jack Nation's blog post drawing on Rich Hickey's "simple made easy" philosophy proposes exactly those three phases, and Claude Code's planning mode operationalizes something similar. De Oliveira agrees with the structure but thinks it's missing a step: the human comprehension pass between research and planning. The agent does its exploration; you need to actually absorb what it found before you let it start architecting a solution.

This is a meaningful distinction. It means the human isn't just a prompt-writer and output-reviewer — they're an active participant in sense-making at a specific point in the loop. The agent can explore; only the human can decide if what it found is the right thing to act on.

It's also worth noting what developer productivity research keeps turning up: the speed gains from AI coding tools aren't as straightforward as they look. The developers getting real leverage tend to be the ones investing in structure — well-defined constraints, testing frameworks, clear mental models — not the ones treating AI as a code faucet to turn on. De Oliveira's data point fits that pattern almost uncomfortably well.

The thing worth sitting with

De Oliveira's 67/2 split is her data, from her context, in her codebase. It's not a controlled study, and she's the first to say others should track their own usage and see what they find. Different work environments, different codebases, different skill levels with prompting — the ratio will vary.

But the directional implication is hard to dismiss. If a senior engineer working at the frontier of AI-assisted development, in a complex production codebase, is spending two-thirds of her AI interactions just trying to understand things — and generating code with it only 2% of the time — then the pitch that AI coding tools are primarily code generators might be describing the wrong product entirely.

The generation capability is real and it's genuinely useful. But it seems increasingly like the thing you get to after you've done something harder: building enough of a mental model that you can point the agent somewhere worth going.

What you prompt matters. But understanding enough to prompt well might matter more.

Yuki Okonkwo is Buzzrag's AI & Machine Learning correspondent.