Jeff Dean at YC: Napkin Math, Bad Transistors

In 2001, Google search ran on hard drives. Then Jeff Dean and Sanjay Ghemawat did a little arithmetic, noticed the entire search index would fit in RAM across the machines they already had, and shipped a new version in a few days. Google got fast. You probably felt it without knowing why.

That's the move Dean keeps making — not heroic engineering marathons, but a moment of clarity where the math says wait, this is wrong, followed by a build. At YC's Startup School 2026, he walked through that pattern in enough detail that it stops feeling like mythology and starts feeling like a method.

The talk is worth your hour. Here's what I found myself actually thinking about afterward.

The napkin that built the TPU

In 2013, Dean watched deep learning cut speech recognition error rates roughly in half — the equivalent, he said, of "20 years of advances in speech recognition in just a few months." The natural next thought: people are going to use this. So he ran the numbers. If every Google user talked to their phone for three minutes a day using the new speech system, Google would need to double its entire server fleet just to handle that load.

That's the moment where a reasonable engineer says "well, speech recognition is probably a niche use case anyway" and moves on. Dean went the other way: what if we built a different kind of chip?

The TPU that came out of that calculation was, in Dean's framing, deliberately general — specialized for low-precision dense linear algebra (the math that lives at the heart of nearly all modern ML), but not for any specific algorithm. That generality is what let it survive the transformer revolution years later, an architecture that didn't exist when the TPU was being designed. According to Google's own documentation, that first TPU was 30 to 80 times more energy efficient than the CPUs and GPUs of the day, with 20 to 30x lower latency.

The lesson he's trying to hand to a room full of founders isn't "build chips." It's: what are the napkin calculations you're not running? What assumption about compute or latency or cost are you treating as a fixed constraint when it's actually a design choice?

Inference is the new memory

Dean thinks we're at a similar inflection point right now, and his "fits in RAM" equivalent for 2026 is inference hardware. Not training hardware — inference. The moment when a model has already learned what it knows and is just answering your question.

Everyone building AI products lives at inference time. Every response you've ever waited on from a model, every tool call that took a beat too long, every agent that felt like it was thinking in slow motion — that's an inference problem wearing a model problem costume. Dean's argument is that inference latency is still treated as ambient background, the way disk seek time used to be background. The opportunity is to stop accepting it.

"Imagine what you could do with something where the latency is 50x better," he told the YC audience. He's thinking about hardware that minimizes data movement, commits to very low precision operations rather than supporting a zoo of precisions, and strips out everything else. Less like a GPU, more like a very fast calculator that's really good at exactly one kind of calculation and doesn't apologize about it.

This matters for a non-obvious reason: the more specialized the hardware, the more it forces you to make explicit choices about what your system actually needs. Vagueness is expensive at inference time.

The faulty transistor thought experiment (okay, this one got me)

Here's where Dean went off-script in a way I found genuinely surprising. 🤯

He described a thought experiment he'd been running with colleagues: what if you designed chips out of transistors that error-corrected 20 times a day instead of once every million years? The entire semiconductor industry has spent 60 years moving in the opposite direction — toward perfect, identical, zero-error transistors. What Dean's imagining is something closer to how large distributed systems actually work: unreliable components at the bottom, reliability engineered in at higher levels. Three copies of data on three machines in three racks, Reed-Solomon encoding, the whole deal.

Nobody builds transistors this way. The entire fabrication ecosystem is organized around the premise that they shouldn't. But Dean's point isn't "let's do this tomorrow." It's that this assumption — that the hardware layer must be reliable and the software layer uses that reliability — is never examined because it's always been true. What if it didn't have to be?

The interviewer noted it rhymes with neuromorphic computing and how the brain actually works. Dean agreed. Signals in the brain aren't especially reliable. Important signals travel redundant pathways.

I've seen versions of this idea float around ML Twitter, but hearing it from the person who built TPUs and MapReduce — who has actually shipped systems that defied analogous assumptions — it lands differently. Most thought experiments like this are philosophy. When Dean runs them, they occasionally become infrastructure.

The 1% rule for what to build

The most practically useful thing Dean said, aimed squarely at founders: before you commit to a problem domain, test the frontier models on it. Not in a "does this generally work" way. Rigorously.

"Look for something where the model succeeds 0% or 1% of the time, not 20%."

If a general model is getting 20% on your target task, that's a capability that's starting to emerge — it'll get to 80% with scale and better training data, probably sooner than you think. But 0-1%? That's a shape of problem the model genuinely hasn't developed for. That's where a specialized system, or a product with access to data the general model will never see (your personal files, a company's proprietary logs, a niche scientific dataset), can have a durable advantage.

The AlphaFold example is the clearest version of this: a highly specific model for protein folding that does one thing extraordinarily well, in a domain where general-purpose models weren't close. Dean pointed to materials science and chip design as spaces with similar structure — hard, bounded, evaluable, data-constrained.

The caution he embedded in this, though, is real and worth sitting with: general models are improving across the board. Continuously. The question isn't just "can I beat the general model today?" but "will I still be beating it in 18 months?" That's a harder question, and he didn't pretend to have a formula for it.

Specs are actually more important now, not less

This one runs against the vibe. Lots of the AI-native founder discourse is about iteration speed — just ship, test, let the model surprise you. Dean's position is almost the opposite when it comes to agents.

The clearer your spec, the better the agent does. He illustrated it with code translation: if you have a Python codebase and want a Go version, that's a task modern models handle remarkably well — because the Python code is the spec. Everything the Go implementation needs to do is written down. The model can translate, run the tests, compare behavior, and iterate until there are no divergences.

The failure mode he's seen with agents is vague direction: here's a fuzzy goal, go figure out what I meant. The agent will figure something out. It just might not be what you imagined. "The importance of specifying what you want has actually gone up," he said, "because before you'd be handing it off to a very intelligent human who maybe has context or can ask you follow-up questions."

Agents can sometimes ask. But don't count on it.

When agents fall off the rails at step 40

Every person who's built an agent workflow knows the 10-step cliff. Things work until they don't, and they don't in ways that compound.

Dean's explanation: models degrade as they move off the distribution of what they were trained on. Give them skills and hints that keep them on familiar paths — the "brightly lit path," as he put it. Use multi-agent systems where separate evaluator agents are scoring which approaches look promising. Let inference-time compute do search over the solution space rather than betting everything on a single chain.

His team does this internally with coding workflows, writing explicit skills that teach models how to use Google's internal tooling — fetching log files, running benchmarks, the proprietary bits a base model wouldn't know. The skill doesn't make the model smarter. It keeps the model from wandering.

What stays scarce when agents write all the code

If agents handle implementation, the limiting resource becomes taste — which Dean defined as knowing what's worth working on at all.

"A researcher can have all the tools and all the techniques, but often most of the battle is what problem are you going to spend your time on."

He offered a concrete practice for building it: write down predictions about what'll matter in the next 12 months. Don't just pick one and go. Come back in a year and audit which ones played out, which ones other people built, which ones nobody touched. That's taste calibration data. You're not just working on problems — you're getting better at choosing them.

The distillation paper story punctuates this without him saying so explicitly. Dean described a paper he co-wrote on knowledge distillation — the technique of training a small, efficient "student" model from a larger "teacher" model — that got rejected from a major venue. The reviewer said it was "unlikely to have significant impact." That technique is now everywhere, including in how Google builds its Gemini Flash models from its larger Pro ones. The paper went to arXiv and people read it anyway.

The reviewer wasn't malicious. They just didn't have the industrial context — the urgency of needing to serve models to billions of people at lower cost — that made the problem's importance obvious to Dean. Taste is partly experience, partly vision, partly having felt the pain of the constraint you're solving before anyone else feels it.

That's the 2001 RAM moment, replicated across every system Dean has built. The math shows something uncomfortable. Most people look away. He builds the chip.

By Yuki Okonkwo, AI & Machine Learning Correspondent