AI Coding Tools May Erode Developer Skills

There's a question that keeps circling developer communities that no one has been able to fully shake: when you outsource a cognitive task to a machine, what happens to the part of your brain that used to do it?

The "AI brain rot" discourse has been loud and largely vibes-based. Hot takes on X, anxious Reddit threads, the occasional Medium post from a senior engineer who noticed something felt different. What's been harder to find is actual data—structured, controlled, specific enough to be useful. A recent Anthropic study on AI assistance and coding skills is, at minimum, a step toward that. Dr. Károly Zsolnai-Fehér covered it on Two Minute Papers, and while his framing is characteristically enthusiastic, the underlying research deserves a slower read.

What the Study Actually Measured

Fifty-two junior software engineers. Two groups. One used AI assistance for a coding task; one didn't. Then both groups took a quiz.

The speed results landed softly: the AI group finished roughly 8% faster, or about two minutes ahead. Zsolnai-Fehér acknowledged this with appropriate epistemic honesty: "the result is not statistically significant." So on the productivity question—the one every AI vendor leads with—the honest answer is: maybe.

The skill-retention question hit differently. The AI group scored 50% on the post-task quiz; the non-AI group scored 67%. That differential is statistically significant, and Zsolnai-Fehér didn't underplay it: "this one would be much harder to dismiss as noise." The methodology details behind that gap are worth examining in full—but the gap itself is real enough to warrant attention.

The location of the biggest underperformance is what makes this finding useful rather than just alarming. It wasn't general knowledge. It was debugging. The AI group was measurably worse at diagnosing and fixing broken code. Which makes a particular kind of intuitive sense: if you've been leaning on a tool to write the code in the first place, the moment something goes wrong you're debugging outputs you didn't fully construct. That's a harder problem than debugging your own logic.

The Hammer Doesn't Make You Worse at Building

Zsolnai-Fehér opens with a frame worth sitting with: "For thousands of years, humans have built tools to make hard things easier. A hammer does not make you worse at building. But is AI different? Because AI helps your brain, not your hands."

It's a clean distinction, and it gestures at something that cognitive science researchers have been poking at for years under the banner of "cognitive offloading." We've always used external tools to extend our minds—calendars, GPS, calculators—and the research on whether that degrades underlying capability is genuinely mixed. GPS probably did make some people worse at spatial navigation. Calculators didn't obviously make people worse at mathematical reasoning in ways that mattered. The question is always: what are you actually offloading, and how central is that capacity to the skill you're trying to develop?

For a junior developer, debugging isn't a peripheral skill. It's close to the core of how you develop intuition about systems. It's how you learn what can go wrong, how to read error messages, how to hold a mental model of a codebase. If AI is absorbing that process—generating code you then run, encountering errors, and asking the AI to fix those too—you might be getting the outputs of software development without building the judgment that underlies it. That's the structural risk this study is pointing at, even if 52 participants can't confirm it at scale.

The Study's Own Honest Limitations

To his credit, Zsolnai-Fehér doesn't oversell this. "This is not an ultimate study. No, this is not the final word on this question. This is 52 mostly junior developers, one Python library, one short task, and one quiz."

That's a meaningful set of constraints. One library, one task, one quiz—this isn't a longitudinal study tracking developers over months or years. We don't know if the skill gap persists, widens, or closes with continued AI use. We don't know how senior developers fare. We don't know how results would shift with a different kind of AI tool—the study used a chat-style assistant, not the kind of deeply integrated agentic coding systems that are increasingly common. Zsolnai-Fehér speculates the agentic case might produce even larger differences, which is plausible but not demonstrated.

What the study does do is give you a concrete signal in a specific context: for junior developers, relatively new to a codebase, using AI assistance for an unfamiliar library, skill retention takes a measurable hit. Whether that generalizes is the open question.

The sample size also matters in a specific way here. At 52 participants, the study is powered enough to detect large effects—and the quiz score differential qualifies—but there's genuine uncertainty about effect size precision. What we can say is that something real happened; the magnitude deserves replication before anyone draws organizational conclusions.

The Pedagogy Question Nobody Is Asking at Scale

Here's what I find most interesting about the framing Zsolnai-Fehér lands on, which is effectively a pedagogical argument: use AI as a tutor, not an answer machine.

"If you use AI as a tutor, it will sharpen your mind." His three recommendations track with established learning science—work on things you already understand to reinforce fluency, ask questions rather than request answers for things you're learning, and attempt problems yourself before delegating them. These are basically the principles behind spaced repetition, productive struggle, and formative assessment, translated into the context of AI-assisted development.

The problem is that none of these norms are being transmitted at the organizational level with anything like consistency. Companies are rolling out AI coding tools at scale, with productivity metrics on the dashboard, and very little conversation about what those tools might be doing to the learning trajectories of the junior developers who will eventually become the senior developers responsible for systems that matter. The incentives point one way. The pedagogical research, such as it is, points toward something more complicated.

There's also a question about what "junior developer" even means in a few years if AI handles the tasks that have historically built junior developer competency. That pipeline concern isn't new—it comes up whenever a layer of automation absorbs entry-level work—but the debugging finding sharpens it. If the cognitive load of debugging is where foundational engineering judgment develops, and AI is steadily absorbing that cognitive load, the field may be looking at a capability gap that only becomes visible much later, when the people who were juniors in 2024 are seniors in 2030 and something breaks in a way that requires actual human reasoning to untangle.

What the Data Gives You

Here's where I'd land: this study is a pointer, as Zsolnai-Fehér calls it, not a verdict. Treat it like a warning sign on a road you're already driving—worth slowing down for, not worth turning around over.

The productivity case for AI coding tools is, at best, "maybe, in certain contexts." The skill-retention concern is real enough to take seriously at the individual level, especially for developers still building core competencies. The structural question about how the field trains its next generation of engineers is larger and more urgent than any single study can resolve.

What the study does is replace vague anxiety with something more specific: the risk isn't AI in general, it's AI-as-answer-engine applied to tasks you haven't yet internalized. That's a precision that actually helps you make decisions. Whether the field makes any structural decisions based on it is, as usual, a different question entirely.

---

Dev Kapoor covers open source software, developer communities, and the politics of code for Buzzrag.