How Zed Trained Its Zeta2 Edit Prediction Model

There's a particular kind of engineering talk that operates as a confession booth—someone walks up to a podium, lists everything that didn't work before getting to what did, and the audience nods along because they've lived most of those failures themselves. Ben Kunkle's presentation on building Zeta2, Zed's edit prediction model, belongs to that genre. It's a tight, technically dense account of how a small team built a production ML pipeline from scratch, and what's interesting isn't just the system they landed on—it's the gap between what they assumed would be easy and what turned out to be genuinely hard.

What edit prediction actually is

Before getting into the pipeline mechanics, it's worth being precise about the problem Zed is trying to solve, because "edit prediction" is one of those terms that sounds self-explanatory until you think about it for thirty seconds.

This isn't autocomplete in the Gmail sense. The model gets a region of code around your cursor—type definitions, variable declarations, recent edits, diagnostic errors—and tries to predict the next edit you're going to make. Not the next character. The next meaningful change. It runs on every keystroke, which means latency is existential: a model that's right 80% of the time but takes two seconds to respond is worse than useless in a live coding context.

That constraint is why Kunkle's team built a small, specialized model rather than reaching for GPT-4 or Claude. "It can do this task and this task only," he explains. That's the bet: a narrow model, trained relentlessly on one job, beats a generalist doing that same job at inference time. It's a fairly orthodox argument in applied ML, but it requires you to actually build the narrow model well—which is where the interesting problems begin.

The distillation pipeline, and why it's messier than it sounds

The training approach Zed uses is knowledge distillation: take a large frontier model (the "teacher"), have it generate predictions on real production data, then train the smaller model (the "student," Zeta2) to replicate those outputs. The concept is clean. The execution is not.

"If you ask them 100,000 times, they're going to give you 100,001 answers," Kunkle says—a deadpan acknowledgment that frontier models are probabilistic engines, not lookup tables. Getting consistent, high-quality teacher predictions required significant prompt engineering, and even then some fraction of outputs came back broken: predictions that ignored the editable region boundary, or that simply reversed what the user had just typed.

To catch the bad outputs, the team implemented what Kunkle calls a repair step. A heuristic layer flags suspicious predictions, then routes them to a second frontier model call with a prompt that essentially says: here's what went wrong, can you fix it? Only after that repair pass do predictions get formatted for student training. The whole pipeline is built in JSONL—each stage adds or reshuffles fields in a single-line JSON object—which keeps the stages modular and cacheable. That last part matters: the distilled teacher outputs can be reused across training experiments; only the prompt formatting changes when Kunkle's team wants to test whether including diagnostic error counts improves the model.

It's not glamorous infrastructure, but it's legible. Someone reading the pipeline six months from now can follow what happened to any given training example.

The settled data problem

The more conceptually interesting challenge—and the one that occupies most of the talk—is what Zed calls "settled data." The idea is appealing: since Zed is the editor, it can wait until a user stops editing a region and snapshot the final state. That final state is the ground truth. The user wrote exactly what they wanted. Why not train on that?

Because users are messy. You might pause, reconsider, let an AI agent rewrite the entire block. A prediction that looked reasonable at the moment of generation might look completely wrong against the settled state—not because the prediction was bad, but because the user changed direction. Training on that signal would be training on noise.

The filter Kunkle's team landed on uses Levenshtein distance: generate multiple predictions from the teacher model, and check whether any of them are close to the settled state. If they are, you have reasonable confidence that the settled state was predictable—that it wasn't the result of some chaotic mid-session pivot. If none of the predictions are close, the example is probably noise and gets dropped.

The catch was cost. Running ten teacher predictions per example, across 100,000 training examples, means a million frontier model API calls. "That is prohibitively expensive," Kunkle acknowledges, with the tone of someone who ran the math and felt it in their chest.

The resolution is elegant in a way that only becomes available after you've already done a lot of hard work: Zeta2 itself has gotten good enough that you can run it 50 times instead of the teacher. The student now approximates teacher quality closely enough that its distribution of predictions is a useful proxy. Running your own trained model 50 times costs "basically nothing" compared to frontier API calls. So the system bootstraps—the model trained in round one becomes the validation tool for round two's training data.

The metrics that actually matter

Kunkle tracks several evaluation signals, and the most revealing one is something called the reversal ratio: how often the model predicts the exact opposite of what the user just typed. A high reversal ratio is a reliable sign that something has gone wrong—the model isn't making contextually appropriate predictions, it's confusing forward progress with undo operations.

It's a heuristic, not a theorem. But that's often the point of a good diagnostic metric: you don't need it to be theoretically perfect, you need it to catch bad behavior before it ships. The reversal ratio is the canary.

For offline evaluation, the team uses delta ChrF—an n-gram similarity metric—running predictions against a held-out test set to avoid contamination. But Kunkle is candid about the limits of offline evals: they don't necessarily correlate to what users actually want in their editor. The final signal is production acceptance rate: how often developers actually keep what the model suggests. You can optimize your offline metrics into the stratosphere and still ship something users find annoying.

The production deployment is gradual. Experiments get sampled at 15% of traffic before they're promoted to become the live model. The version Zed released as Zeta2—built on a seed coder base—went through that same ramp.

What this says about the current moment in applied ML

What Kunkle describes isn't cutting-edge research. He's not proposing a new architecture or a novel training objective. What he's describing is the plumbing: how you get from "frontier models are smart" to "a small, fast, specialized model runs well in production on real user data." That plumbing is genuinely underreported relative to the architecture papers that dominate AI coverage.

A few things about this pipeline are worth sitting with. First, it depends entirely on user consent—Zed collects opt-in production traces, and the whole system falls apart without real-world editing data. The quality of the training set is inseparable from user participation. Second, the "settled state" problem is philosophically interesting: you're trying to define ground truth for a task where ground truth is whatever the user eventually decided to do, which may bear no resemblance to what they intended when the prediction was made. Ten seconds of inactivity is Zed's current threshold for "settled"—a number Kunkle essentially calls a rough heuristic, not a principled choice.

And third, the bootstrap dynamic deserves attention. Zeta2 is now good enough that it validates training data for future versions of itself. The system is eating its own tail, in a way that could be virtuous or could compound errors over training generations. Kunkle doesn't raise this concern—there's no indication the team thinks it's a live issue—but it's the kind of thing that tends to become a problem quietly, well before anyone notices the metric drift.

Whether that's a reason for concern or just a routine engineering challenge to monitor probably depends on how much you trust the offline evals that are supposed to catch it.

---

Marcus Chen-Ramirez covers AI, software development, and the intersection of technology and society for Buzzrag.