When AI Models Find Bugs Faster Than Humans Can Fix Them

George Hotz threatened to release a zero-day vulnerability every day until AI companies released their newest models. Low Level Learning called it "the dumbest take I have ever read." What they're actually arguing about tells us more about the future of software security than either side wants to admit.

The immediate spark was Anthropic's announcement about Claude Mythos, an AI model that reportedly identifies bugs in code with 83% accuracy on CyberGym's benchmark—a collection of known vulnerabilities in software like FFmpeg and curl. That number has security researchers split between those who see an existential threat and those who see marketing hype wearing a lab coat.

The Economics of Finding Holes

Hotz's core argument is economic: "The reason there aren't zero days everywhere is cuz nobody seriously looks because hacking other people's [stuff] is illegal and criminals are usually not very skilled or they would choose a different line of work. Want more zero days to be found? Make hacking legal."

The counter-argument came immediately. Bug bounty programs already pay substantial rewards—Apple offers up to $2 million for a zero-click remote code execution vulnerability on iPhone. Microsoft pays $250,000 to $500,000 for similar Windows vulnerabilities. The money exists.

But that response misses Hotz's point, which is less about absolute dollar amounts than opportunity cost. A skilled programmer [who could find zero-day vulnerabilities can also work at Google, get stock options, and not risk federal charges. The legal and social costs of vulnerability research—even legitimate white-hat work—constrain the talent pool in ways that no bug bounty can fully compensate for.

One participant in the discussion framed it plainly: "There are so many programmers who if they had been raised in some kind of a way in a society and a religion where stealing people's money was considered virtuous, we would have found so many more zero days right now than we have."

The Talent Bottleneck

The deeper issue isn't whether Claude Mythos works as advertised—though that matters—but what it reveals about software security's structural problem. Vulnerability research has always been constrained by specialized knowledge. Finding bugs in hypervisors requires understanding hypervisors. Finding bugs in drivers requires understanding drivers. When you divide the world's security expertise across all the niche technologies that need scrutiny, you end up with perhaps ten or twenty people on Earth who can effectively attack any given system.

One security researcher on the panel noted this explicitly: "Software security a lot of the times can be marked up to the fact that a lot of software just has not had elite attention... AI isn't solving a unique problem, the AI is solving the scalability problem where it's like you can train the AI to do a thing that Joe knows how to do and now you have a hundred mediocre but 100 Joe's right."

This is the actual threat vector—not that AI makes hacking easier for criminals (who already have economic incentives and tools), but that it democratizes access to specialist knowledge. Someone with basic security understanding could theoretically use Claude Mythos to audit hypervisor code without spending years learning hypervisor internals.

The Benchmark Problem

Before accepting that 83% success rate at face value, consider what it measures. CyberGym tests models against known vulnerabilities. We don't know the token costs for achieving that success rate. We don't know if the models were trained on similar data. We don't know if they're pattern-matching specific bug classes or genuinely understanding security flaws.

As one developer put it: "Dirty data is like a huge gigantic problem in all benchmarks. All benchmarks are being fed back into the models. It's really actually hard to tell like what does a 20% improvement on software engineering bench actually mean?"

The history of AI capability claims suggests caution. When Anthropic previously announced an AI that "wrote a C compiler," the details revealed it couldn't fit a bootloader in the required memory space and was tested against 30 years of existing GNU C compiler tests—essentially playing Jeopardy with the answers already provided.

The Access Dilemma

Anthhropic's decision to restrict Claude Mythos access creates its own policy problem. If the model is as capable as claimed, limiting access might prevent immediate harm. But it also concentrates vulnerability-finding capability in the hands of a few organizations while everyone else's software remains vulnerable to the same bugs—they just don't know about them yet.

The discussion surfaced three scenarios: universal access creates a "dangerous cyber no man's land" where anyone can find zero-days; restricted access means only a handful of companies can find them; or (the joke version) Anthropic moves to the Cayman Islands and "takes over every government by hacking all the software."

What's missing from this framing is the fourth option already happening: nation-state actors and sophisticated criminal organizations already have resources to find these vulnerabilities. They're not waiting for Claude Mythos. Restricting access to defensive security researchers doesn't make the bugs disappear—it just ensures fewer people are looking for them on behalf of users.

What The Incentives Actually Show

Both sides of this argument point to the same underlying reality: software security has been a talent allocation problem disguised as a technical one. We haven't lacked the theoretical knowledge to find bugs. We've lacked enough skilled people spending enough time looking.

Bug bounties address this partially by compensating security research, but they can't compete with the earning potential of using those same skills in software development—or the asymmetric rewards available to sophisticated attackers selling exploits to governments or criminal enterprises.

If AI models can genuinely find vulnerabilities at scale, they're solving for the scarcity of elite security attention. Whether that's deployed defensively depends entirely on who has access and what their incentives are. The policy question isn't whether to release Claude Mythos or withhold it—it's how to structure incentives so that vulnerability discovery translates into vulnerability remediation rather than exploitation.

Anthhropic claims it needs to restrict access to prevent harm. But the economic argument suggests harm is already happening in the gap between vulnerabilities that exist and vulnerabilities that get fixed. The real test of Claude Mythos won't be benchmark performance—it'll be whether the bugs it finds actually get patched before someone else finds them first.

Samira Okonkwo-Barnes is Buzzrag's Tech Policy & Regulation Correspondent