Continual Learning Could Reshape AI Regulation

The dominant mental model for AI regulation goes something like this: a lab trains a model, evaluators inspect it for dangers, regulators sign off, and the model ships to users. That is a clean, legible pipeline. It is also, argues podcaster and essayist Dwarkesh Patel, a model that continual learning would render obsolete before the ink on any framework dried.

In a new video essay published to his channel, Patel lays out eight predictions for what he calls the "era of continual learning" — a future in which AI systems don't just retrieve from a frozen set of weights but actively update those weights through the work they do in deployment. The predictions range across regulation, alignment research, market structure, and corporate strategy. Taken together, they sketch a version of the AI landscape that looks quite different from what most policy discussions assume.

Whether Patel is right about the technology is a separate question from whether his downstream implications follow. Both are worth examining.

The Saxophone Problem

Patel opens with an analogy designed to motivate why continual learning matters at all. Imagine a saxophone student who fails, takes notes, and passes those notes to the next student — who has also never played. That student fails, adds more notes, passes them on. Repeat infinitely.

"I don't think there's any sequence of text they could write to each other that would allow the subsequent student to just nail the saxophone from the first try," Patel says. "At some point, you actually have to accumulate the relevant experience into your brain."

The point is that some kinds of skill are embodied in a way that text cannot fully capture. Current AI systems, in Patel's framing, are like that infinite line of note-passing students: they reset every session. Continual learning — systems that actually update their parameters through deployment experience — is his proposed remedy.

This is not a novel technical claim. Continual learning has been a research area for years, with the core challenge being "catastrophic forgetting," where updating a neural network on new data causes it to degrade on old tasks. Whether that problem is tractable at production scale remains genuinely open. Patel waves at this, acknowledging the engineering challenges, but treats them as solvable in due time. That optimism is worth flagging because most of his eight predictions are conditional on it.

What This Does to Regulation

Patel's first prediction is the one with the most immediate policy relevance. Current regulatory proposals — including the EU AI Act's conformity assessment framework and various U.S. proposals — largely assume a discrete training-then-deployment pipeline. Run your evaluations pre-deployment, assess the risks, and you're good. Patel argues this logic collapses if the model is learning continuously from millions of daily interactions.

"What if the model is improving every single day based on the millions of sessions of work it does in that day?" he asks. "If that happens, we could potentially be locking in an archaic and potentially counterproductive approach to dealing with the threats from AI."

His proposed alternative is periodic risk inspections — monthly or quarterly — rather than a single pre-deployment gate. The underlying logic is sound as far as it goes: a regulatory framework designed around a static artifact will fail when the artifact is no longer static. This is the same structural problem that arises when AI trains AI in autonomous loops — the accountability scaffolding assumes human intervention points that may not exist in practice.

But Patel's argument here cuts in two directions simultaneously, and he doesn't fully reckon with the tension. On one hand, he is saying the pre-deployment checkpoint won't work technically. On the other, he seems to suggest we should be cautious about locking in any safety regime right now because we don't know what technology we'll be dealing with. The first is a technical argument for better-designed regulation; the second is an argument for regulatory restraint. Those positions share a conclusion but don't share the same logic, and conflating them risks letting the second ride on the credibility of the first.

Alignment Gets Harder

Patel's second prediction is that technical alignment research will need to fundamentally reorient. Most current work focuses on ensuring a frozen set of weights behaves safely during deployment. Continual learning scrambles this entirely — the weights are no longer frozen.

"I'm not aware of much research on the question of how we make it so that even with constant weight updates, the AI system never falls prey to jailbreaks or changes into a deceptive or evil persona," he notes. More concerning, if a model is consolidating learnings across users, a bad actor could potentially inject malicious inclinations into the base model through deliberate manipulation of their sessions.

The parenting analogy Patel reaches for here is genuinely interesting: humans also improve in a self-directed way, and we manage that through instilling deep values early. The question is whether that analogy scales to a system learning from millions of simultaneous interactions rather than one childhood at a time. The research community has no settled answer.

Market Structure: Where This Gets Commercially Interesting

Predictions three through eight are where Patel's analysis becomes more commercially pointed, and where the policy implications get thornier.

On model diversity: right now there are fewer than five prominent base models, and they're trained on largely similar data. Continual learning, Patel argues, would produce genuine divergence — models shaped by different deployment contexts would develop meaningfully different capabilities. He frames this as a positive outcome, a hedge against "mode collapse" in the model landscape.

On competitive dynamics: once deployment is training, the advantage of being the leading model compounds. More users running harder tasks generates more useful feedback, which makes the model smarter, which attracts more users. It's a flywheel, and Patel argues it would intensify the AI race in ways that push labs to ship their most capable models faster than they currently do.

The switching costs prediction is arguably Patel's most commercially significant insight, and it maps neatly onto dynamics already visible in cloud infrastructure. He quotes Anthropic CEO Dario Amodei's cloud provider analogy directly: cloud margins are high because switching is expensive. Continual learning would create an analogous lock-in for AI, where changing your model provider means, in Patel's words, "firing an employee that has accumulated months of context on your organization and replacing them with a very fresh, very unexperienced new intern."

This has obvious antitrust implications that Patel doesn't linger on. Lock-in is not inherently illegal, but when the lock-in mechanism is also the primary value proposition of the product — your model gets smarter the longer you use it — the incentive structure for incumbents to foreclose competition becomes structurally embedded in the technology itself. That's a different kind of market concentration than what regulators have been examining in consent decrees and merger reviews.

The Subsidy-or-Exclusion Play

Prediction seven is the one I'd watch most carefully from a policy perspective. Patel argues that because real-world usage becomes the primary training signal, labs will face pressure to get enterprises to let them train on session data. The lever they'll pull is price: subsidize compliant enterprises, deny the best models to non-compliant ones.

"With both carrots and sticks, the labs can do a lot to get users to allow AIs to learn from experience," he says.

This is already structurally present in how large AI platforms handle data. The difference in the continual learning regime is that the bargain becomes explicit and high-stakes: share your operational data or be relegated to a less capable model. Enterprises will recognize the lock-in risk, but the competitive pressure to access state-of-the-art capability may overwhelm that recognition. It's roughly the dynamic that produced GDPR in the first place — companies offering free services in exchange for data, until regulators decided the exchange required more informed consent than it was getting.

Economies of Scale, and Who Gets Left Behind

The final prediction concerns inference economics. Continual learning with full weight updates — rather than lightweight adapters — requires large batch sizes to serve efficiently. Patel cites back-of-envelope math suggesting optimal batch sizes for sparse models like DeepSeek V3 exceed 2,400 concurrent sequences. An individual user running a single sequence suffers dramatically worse compute efficiency.

The implication is structural: personalized AI at scale will be economically rational for large enterprises but potentially prohibitive for individuals or small organizations. This is a meaningful distributional question that Patel notes but doesn't particularly dwell on. If continual learning does become the dominant paradigm and large-batch inference is the efficient regime, that's a capability gap between enterprise and individual users that compounds over time.

That gap sits at the intersection of competition policy and digital rights in ways that no current regulatory framework is designed to address. Whether it will materialize depends entirely on whether continual learning at scale proves tractable — a question Patel treats as a matter of timing rather than a genuine open question.

The technology may vindicate his optimism. But policy debates have a habit of arriving after the market structure is already set.

Samira Barnes covers technology policy and regulation for Buzzrag.