AI Coding Agents Still Need a Human in the Loop

There is a recurring moment in every technology hype cycle where a practitioner who actually built the thing comes back from the frontier with a more complicated report than the boosters want to hear. Dexter Horthy is having that moment right now with AI-assisted software development.

Horthy, who built HumanLayer and runs what he calls a software factory — an automated pipeline where AI agents handle coding tasks end-to-end — sat down with podcaster David Ondrej for a nearly hour-long conversation that is worth parsing carefully. Not because Horthy is a skeptic. He is not. He is genuinely bullish on what AI agents can do. But he has also, according to reporting from BigGo Finance, run a fully automated "dark factory" that corrupted his codebase over three months. That experience has given him a specific and useful kind of credibility: he is describing the limits of a system he actually pushed to its limits.

The core of his argument is not complicated. AI agents are excellent at solving discrete, well-defined problems. They are poor, structurally and for reasons rooted in how they are trained, at writing code that remains easy to modify six months from now. The gap between those two things — solving a problem versus building something maintainable — is where most of the current AI productivity narrative quietly breaks down.

I have watched this same gap appear in every major platform transition. The early web gave us sites that worked but couldn't be updated without calling the person who built them. Offshore outsourcing gave us software that passed every test and was incomprehensible to every subsequent developer. The pattern is not that the new tool fails to deliver — it usually delivers something. The pattern is that what it delivers has hidden carrying costs that only appear later, at the worst possible time, usually at 3 a.m.

Why the benchmarks don't tell you what you need to know

Horthy's sharpest critique is aimed at the performance benchmarks the AI industry uses to declare progress — tests like SWE-Bench, which measure whether a model can fix a reported bug in an open-source codebase. Think of these as standardized exams: the model gets a specific problem, a set of tests to pass, and a score based on whether the tests pass afterward.

The problem, as Horthy explains it, is structural. Reinforcement learning — the training method that makes models improve — requires what he calls an oracle: something that can immediately and definitively say whether a solution was correct. Running a test suite takes seconds. You can run millions of training iterations that way. Measuring whether code will be easy to change in three months has no fast oracle. The cost of bad architecture is paid in weeks, not milliseconds, which means it cannot be fed back into the training loop in any practical way.

The result: models get relentlessly better at passing tests while the training process remains blind to whether the code they write is a readable, maintainable structure or a tangle of workarounds that technically produces the right output. "If a model gets 99% on SWE-Bench, I don't give a damn," Horthy says, "because I know that it's not being penalized for slop."

This connects directly to a pattern anyone who has reviewed AI-generated code has noticed — unnecessary error-handling wrappers, redundant type casts, logic that exists solely to satisfy a test condition rather than to accurately model the underlying problem. The model is not being lazy or malicious. It is doing exactly what it was trained to do. The training just doesn't include "future developer trying to add a feature" as a stakeholder. The benchmark gap between what scores well and what works in production is not a rounding error — it is the whole problem.

The factory metaphor, and what it actually demands

Horthy's software factory concept — a pipeline where a task enters, an AI agent builds it, automated checks run, and the result ships — is a genuine productivity multiplier for teams that implement it well. The metaphor he reaches for to explain its limits is Eli Goldratt's management classic The Goal, a 1980s business novel about a factory manager who discovers that maximizing the output of each individual station does not maximize the output of the factory. It just creates inventory pileups at the bottleneck. The lesson Horthy draws: if your bottleneck is human code review — the final check before software ships — adding more AI agents at the coding stage makes that bottleneck worse, not better. More output upstream, same constrained throughput at the chokepoint.

This is not a novel insight in operations management. It is, however, an insight that a lot of teams chasing AI productivity gains appear to be missing. The structural need for human oversight doesn't disappear just because generation got faster.

The practical framework: design before you delegate

The more actionable part of Horthy's argument is what he calls program design — the work that happens before an AI agent writes a line of code. His four-stage approach moves from product clarity (what problem are we solving, and how will we measure whether we've solved it?) through system architecture, down to what he calls program design proper: decisions about file structure, function signatures, and the order in which pieces of the system get built.

That last piece is where most teams currently shortcut. AI models, left to their own devices, tend to build software horizontally — they complete the entire database layer, then the entire service layer, then the entire front end. The problem is that nothing is testable until everything is done. Horthy advocates instead for what he calls vertical slices: build a thin, working path through the entire system first — a stub endpoint, a rough front end that connects to it, a minimal data layer underneath — then add complexity incrementally. This is how experienced engineers have always worked. It is how you learn a new programming language: Hello World first, then push outward. The insight is that AI agents will not do this naturally, and nudging them toward it before they start costs far less than correcting them afterward.

Think of it this way: a general contractor who draws up plans before breaking ground saves money. One who tells the crew to start on the kitchen while he figures out where the load-bearing walls are does not. The AI agent is a very fast, very eager crew that will build whatever you point it at. The plans are your responsibility.

The part that cannot be automated away

Here is the finding Horthy keeps circling back to, and it is the one I find most credible: if you stop reading the code your agents produce, you will eventually encounter a bug that no model can fix, and you will have to diagnose it in a codebase you no longer understand.

This happened to him. In mid-2025, his team ran a light-touch factory where they reviewed plans and tickets but largely stopped reading the actual code. Then a bug appeared. Weeks passed. Multiple models, intensive prompting, no resolution. The team had to go back into code they had not read in months, code that had accumulated the architectural sloppiness that unreviewed AI output tends to accumulate, and find the problem themselves. "You absolutely will eventually hit an issue the agent can't solve," he says. That is not a prediction. That is a post-mortem.

The skills erosion risk here is real and underappreciated. The counterargument — that you don't need to understand code, just logic — has some merit. Horthy's co-host pushes back on this directly: maybe what matters is understanding the product's behavior and business logic, not the specific implementation. That tension is genuinely unresolved, and reasonable people will land in different places depending on the complexity of their systems and how much they can afford to be wrong.

What is not genuinely unresolved is Horthy's broader thesis. Compared to the previous ten waves of AI-is-going-to-transform-software-development claims, this one is more credible precisely because it comes with failure stories attached. He is not selling the dream from a position of never having tried. He is reporting back from having tried hard, for months, and having found a real boundary. The human review bottleneck he describes is structurally similar to every previous automation wave: the machine accelerates production, and human judgment remains the final constraint on quality.

What changes when AI agents get dramatically better at writing maintainable code is not a question anyone can answer today, including Horthy. But his point about the training process is the right one to hold onto: until there is a fast, reliable way to score long-term maintainability and feed that signal back into how models learn, the models will keep optimizing for passing tests rather than for the thing that actually matters. They will keep waking someone up at 3 a.m.

---

Bob Reynolds is Senior Technology Correspondent at BuzzRAG.