AI Code Verification Debt Is a Growing Security

The productivity spike from AI coding tools is real. It just doesn't last.

That's the uncomfortable finding at the center of a Carnegie Mellon study that Sonar's Anirban Chatterjee presented at the AI Engineer conference. Researchers sorted GitHub projects by whether an AI tool was used to write the code — Cursor, in this case — and tracked what happened over time. Productivity did jump. Then, around the three-month mark, it came back down. What didn't come back down: static analysis warnings and code complexity, both of which continued climbing well past that inflection point. Sonar's own SonarQube tooling was used to collect that data.

Chatterjee calls the gap that creates "verification debt." The term is useful because it makes the accounting visible. For a throwaway internal tool with two users, you can carry that debt indefinitely. For a production codebase with adversarial users actively probing for weaknesses, you cannot. The quality the model gives you and the quality the application actually needs diverge — and the wider that gap, the more expensive it gets to close.

Why models aren't the whole problem

The tempting framing here is "AI code is bad code." That's not quite what Chatterjee is arguing, and the distinction matters.

Models make mistakes, yes — that's a baseline limitation of the technology. But the more interesting problem he identifies is context. "They only know what you tell it," he said in his presentation. "They don't know the broader things that are happening elsewhere in the code base. They don't know what's happening with your business. They don't know what happened in the meeting you had with somebody else 2 weeks ago that's going to influence the code you're writing today."

That missing context isn't a bug that better models will eventually fix. It's structural. Even a perfect model — one that never hallucinates, never produces syntactically broken output — operates on what it's given. Organizational knowledge, implicit constraints, tribal decisions made in Slack threads three quarters ago: none of that makes it into the prompt.

There's a separate problem hiding underneath that one, which is model diversity. Two models from the same provider don't have the same failure modes. Sonar's LLM leaderboard — which, according to the company's own self-reported data, evaluates models across roughly 4,000 coding tasks using SonarQube metrics — makes this concrete. Chatterjee walked through a comparison of Claude Sonnet and Claude Opus (he referenced them as 4.6 variants). Sonnet scores well on correctness and task completion. Opus is the better choice when maintainability, security, or lower complexity is what you're actually optimizing for. If you're toggling between them purely to manage token costs, you may be making that trade-off blind.

The practical implication: you can't treat AI coding tools as interchangeable, and you can't assume that the model you're using is good at the dimension of quality that matters most for the thing you're building.

The human review problem

Here's where the argument gets uncomfortable for anyone who's been leaning on "just have engineers review the AI output."

A Wharton study cited by Chatterjee — and separately documented via researcher Rohan Paul's analysis of the findings — found that when human participants were given an AI tool to help complete tasks, they followed the AI's advice 92.7% of the time when it was correct. That number isn't surprising. What's striking is the other number: per reporting on the study by Snopes, participants still followed the AI's guidance nearly 80% of the time when the AI had been secretly instructed to lie confidently.

Chatterjee's inference — that this is "almost surely happening in code review as well" — is reasonable. Code review is cognitively expensive work. When multiple agents are writing code simultaneously and a human reviewer has to synthesize all of it before a ship deadline, the conditions for rubber-stamping are nearly ideal. The AI is confident, the code looks plausible, the deadline is real.

"There's a lot of rubber stamping that I'm sure is happening in all of your organizations," Chatterjee said. "It's happening everywhere."

This is not a condemnation of developers. It's an observation about cognitive load under production pressure. The developer skills trade-off that's emerging in AI-assisted development doesn't just affect what skills developers retain — it affects what they're able to critically evaluate in review, especially as the volume of AI-generated code increases. And research flagged separately, including an Anthropic study on junior developers, suggests the debugging and verification muscles most needed for quality review may be precisely the ones atrophying fastest.

The proposed fix: zero trust, multi-layered, in the loop

Chatterjee's framework isn't novel in its components — zero trust has been security orthodoxy for years, and static analysis predates AI coding by decades. What's interesting is how he's applying the logic to the specific problem of AI-generated code.

Zero trust here means: don't use the same method to verify code that was used to write it. A model grading its own output will inherit its own blind spots. The verification regime needs to be independent of the generation process — auditable, repeatable, consistent regardless of which tool or model produced the code.

Multi-layered means: no single technique covers the full surface. Syntax issues, data flow problems, architectural concerns, control flow — each requires different methods. Computational review and LLM-driven review need to run together, because they catch different things.

Sonar's implementation of this is called ACDC — the Agent-Centric Development Cycle, a name that is either a joke or a genuinely bold trademark decision. The loop has three phases: Guide (hand the agent architectural constraints, coding standards, and relevant context before it starts), Verify (run checks in real time inside the inner loop so issues get caught before they propagate), and Solve (give the agent the tools to remediate what comes back, rather than queuing it for a human who's already overloaded).

The company's new Sonar Vortex product operationalizes the inner-loop piece — integrating with Cursor, Claude Code, Codex, and other major AI coding tools to run verification as the agent writes, not after. The outer loop — the CI/CD process — gets SonarQube's computational review, which gates pull requests on passing grades across quality, security, and maintainability.

Sonar claims, per the company's own press materials, that more than seven million developers use their tools globally, analyzing close to 750 billion lines of code daily across their solutions.

What to take seriously here, and what to interrogate

Chatterjee works in product marketing at Sonar. This was a conference presentation, not a peer-reviewed paper. The framework he's describing is, by design, implemented by Sonar's products. That doesn't make the underlying diagnosis wrong — the CMU data and the Wharton findings are real research — but the prescription is inseparable from the vendor relationship.

The honest version of his argument stands up reasonably well: AI coding tools introduce quality debt at scale, human review is more compromised than people acknowledge, and some form of automated verification running inside the loop — not just at the gate — is probably necessary infrastructure for teams shipping AI-generated code into production environments with real adversarial exposure.

The harder question is whether any verification layer can actually keep pace. CVEs are being announced and exploited on the same day now, Chatterjee noted. If the code generation pipeline is accelerating, and the attack surface is expanding accordingly, the verification layer has to be faster and more comprehensive at the same time. That's a systems problem, not a product problem.

The productivity spike from AI coding tools lasted about three months. How long until verification debt becomes the story that replaces it?

Rachel "Rach" Kovacs is Buzzrag's cybersecurity and privacy correspondent.