Jeff Dean: The 1% Rule for Building in AI

Jeff Dean sat down with Y Combinator at Startup School 2026. MapReduce, BigTable, TensorFlow, the TPU, Gemini — he's behind all of them. He's now Google's Chief Scientist.

The most valuable thing in the 57 minutes wasn't his résumé. It was a set of judgment tools you can run tonight.

Here are the seven things I took away.

1. The 1% Rule: a 20% success rate is the dangerous one

The host asked the question every founder is asking: as general models get stronger, where can a two- or three-person team still win?

Dean answered like an engineer — he gave a criterion you can actually measure.

First, test: take the strongest general model available and throw your domain's problems at it. See how far it gets.

Then comes the counterintuitive part:

"If they're completely failing, that's probably a good sign. If they're kind of able to do some of it but not very well, that's maybe not a great sign — because that's probably a sign that the capability is starting to be present in those models, and with more training data or larger scale models it's likely to get better."

"So look for something where the model succeeds 0% or 1% of the time, not 20%."

Write that on the wall. A 20% success rate creates the illusion that a little more work will get you there. In reality you're racing the next model release — and you will probably lose.

So where do the 0% opportunities live? He gave two shapes.

Data the model can't reach. His example is sharp: Google is organizing the world's information, and that's covered. But organizing your personal information is wide open — a general model simply can't see that data. Once your product has that visibility, the advantage is structural.

Narrow models. He pointed at AlphaFold: not general, but it handles protein folding extremely well. He named materials science and chip design as similar shapes.

He also added an honest warning: "The general models are definitely getting better at a broader and broader range of things. So you have to figure out — is that thing you're working on going to be a durable thing, or do you think the models at the forefront are going to get better at that in the next six months or 12 months?"

That judgment is the bet.

2. Energy is the unit that decides what's possible

This was the most technically dense stretch.

Dean now measures everything in energy. A single compute operation costs roughly one picojoule. Moving the data in from HBM on the accelerator so the processor can compute on it costs about 1000× that.

That 1000× gap quietly decides which products are possible and how AI algorithms get designed.

The most direct consequence is batching. Without the 1000× difference, you wouldn't need to batch at all. Because of it, you must amortize that data movement across many examples or tokens — paying "1000 divided by batch size" instead of the full penalty. And for genuinely low-latency work, batching is not great.

Which leads to the question worth asking: how many problems founders call "model problems" are actually energy or data-IO problems? Batching and epochs in training look like a model concern. They're a systems and data-movement concern.

He also updated his famous list. Plenty of engineers have had Latency Numbers Every Engineer Should Know taped to a wall. The 2026 AI edition, per Dean, is: bandwidth from accelerator main memory to on-chip memory to the multiplier unit; how much energy a single multiply costs; interconnect bandwidth between chips and how many chips that bandwidth can serve; and the falloff in network bandwidth when you need to talk to 10,000 chips instead of 500.

His own current focus is inference: minimize data movement, use extremely low precision, and don't support many different precisions — if you know which precision you need, build that into the hardware and not much else.

3. The TPU started as napkin math

That last point isn't theory. He did it once, and the result was the TPU.

In 2013, deep-learning speech models started working and halved the error rate. His description: "the equivalent of 20 years of advances in speech recognition in just a few months of fiddling with the model, scaling it up a bit, and getting better data."

Then he got worried — because if speech worked better, people would use it more.

So he ran the numbers: if every Google user talked to their phone for three minutes a day, Google would have to double its entire server fleet. For speech alone.

The conclusion was that CPUs weren't going to work.

Hence the TPU: a chip specialized for low-precision dense linear algebra. As he put it, it can't run Chrome or Word or anything else — but that happens to be the heart of nearly every modern ML algorithm.

The chip that shipped two years later was 30–80× more energy efficient than the CPUs and GPUs of the day, with 20–30× lower latency.

But the part that made me stop was the trade-off. The TPU was built before Transformers existed. Why does it still hold up?

"That's sort of why we built a general purpose linear algebra system. Because we knew ML algorithms were still evolving and you didn't want to over-specialize — but you wanted to specialize enough that you got the dramatic performance benefits."

Specialize exactly enough, and no further. That's the most valuable engineering philosophy in the whole talk.

And his advice on what napkin math you should run tonight: look at the bottlenecks in front of you, and ask whether a completely different approach could get you one or two orders of magnitude. "Don't be anchored on exactly how that problem is solved today — think about how you would solve it from first principles."

4. The skill he wrote himself

On context engineering, he gave a personal example worth copying.

First, a framing I like: the model is only one piece of the system. Information you put in the context is clear to the model — unlike training data, which is "trillions of tokens stirred together into a soup" of hundreds of billions or trillions of parameters.

How do you get better? Use the models on real problems and watch where they fail. "Often you can make the model work better — not by adjusting the model parameters, which is hard from the outside, but by creating better guidelines for the model, writing skills for the model to know how to use different tools."

"As you do that, you end up on this kind of self-improving setup."

Then the concrete case. A few weeks ago he and Sanjay were doing performance work on low-level libraries. Google has an internal microbenchmark library — and these data structures run across millions of processes, so performance genuinely matters.

Without agents, the loop is: measure current benchmarks, modify code, re-run, see what improved, run a broader set, measure cache footprint.

They wrote a skill that taught the model to do all of it. The model could then run the measure → change → re-measure → iterate loop itself.

His framing is the part to remember: "It really just is us giving the approach we would use as people to the model in a form that it could use."

When the host joked that this skill must be worth infinite money, Dean pointed out that he and Sanjay published a 30-page document called Performance Hints a few months ago. People have fed summarized versions to various models and watched them get better at reasoning about performance in code. It's free and public.

The capability was never in the skill file. It's in the method that got written down.

5. Why agents stop working, and how to keep them on the lit path

The host asked why agents go off the rails around step 30 or 40 — context, evaluators, or compounding error in an open-loop system?

Dean's diagnosis: the model was trained on a particular distribution, and as soon as you get a little off it, performance degrades sharply. The farther from its comfort zone, the more likely it fails.

Three remedies:

Give the model skills and hints that keep it on "the more brightly lit path of things it does know how to do."

Run multi-agent systems where several agents try different approaches and another model evaluates which look promising — keeping the good branches, discarding the ones that went off the rails. He calls this searching the space of possible solutions.

Spend inference-time compute on that search. He calls it a very useful general technique for getting much higher performance and reliability in long-running agent flows.

Google does exactly this internally: a harness plus a full set of skills so agents know how to use internal tooling for coding, code review, performance measurement, and fetching log files. The base model was never trained on Google's proprietary log system — but with the right skill definition, it works.

He also flagged something he thinks most people haven't internalized: agents can run for days or weeks, not one or two hours, on some problem domains with capable enough models. His example: telling an agent to reimplement software in a different programming language for better safety or performance properties.

Which leads to a counterintuitive conclusion. Now that agents can write the code, specifying what you want matters more, not less.

"Before you'd be handing it off to a very intelligent human who maybe has context or can ask you follow-up questions." Agents can sometimes do that — but clear specification is the key.

The proof is language translation. Why are today's models so good at Python → Go? Because the spec is extraordinarily detailed — the whole software is the spec. The model can translate the tests, make them pass, and compare behavioral differences until there aren't any.

"Because that spec is so clear."

Flip it around: when your agent underperforms, it's often not the model. It's that your spec isn't much of a spec.

6. Taste is the scarce skill — and it has three practice routines

If every founder learns to run hundreds of agents and all the code gets written for them, what becomes scarce?

"It's really having incredibly good taste in what you ask your agents to work on."

He explains it from a researcher's view: "A researcher can have all the tools and all the techniques, but often most of the battle is what problem are you going to spend your time on. If you pick the problem well and you succeed in solving it, that's way better than if you delightfully execute a research investigation into a rather boring problem."

"And models are not necessarily going to be that good at it. So you're going to have people steering."

Three ways to build it:

Experience. Working on many different problems teaches you what might be interesting later, and what's just barely possible by cobbling together previous approaches.

The 12-month list — the most executable one. Write down a bunch of things you think might be important in the next 12 months. Pick one to work on. Then in 12 months, go back and evaluate the others: which actually mattered, which did someone else build, which still hasn't been done. "That can give you a lot more samples for your own taste creation capability."

Crazy thought experiments. His live example is excellent. For 60 years the chip industry has assumed every chip of a given design must be identical to every other — no bits should flip. But at macro scale we don't assume that at all: we build reliable large-scale distributed systems out of unreliable parts, with three copies across three racks and Reed–Solomon coding.

So: what if you built a system out of transistors that had 20 errors per day instead of one every million years? "That would be a very different design point." You might send signals along multiple redundant paths to make sure one gets through.

The host noted this sounds like the brain. Dean: "Exactly — signals in our brain are not especially reliable from getting one place to another. So when there are really important things, there are multiple pathways."

He also gave the honest boundary: "Oftentimes these thought experiments don't work out, because there are very good reasons that for the last 50 years we've done this thing this way. But it's good to revisit those every so often."

MapReduce came from exactly this. Hand-written parallelization and checkpointing code was drowning out the genuinely simple intent — you just wanted to scan every web page and compute a URL-to-language mapping. Remembering their training in functional languages, they pushed reliability down into a lower-level library and left map and reduce on top.

7. Three things we're changing

Make the 1% Rule a project-approval criterion. Is the general model at 0% or 20% on this? 20% means racing the next release. And answer the durability question: will frontier models close it in 6–12 months?

Write down the human method as a skill instead of outsourcing the outcome. Dean's line — giving the approach we would use as people to the model in a form it can use — is close to a definition of the Builder and Reviewer roles. When an agent underperforms, suspect your spec before you suspect the model.

Start the 12-month list today. It's the cheapest way I've seen to make taste trainable, and it costs one document.

Two numbers to close on.

Evaluator speed. In quantum chemistry, a density functional theory simulator might take a night of computation for one molecule. Dean's colleagues trained a neural approximation on the simulator's inputs and outputs — 300,000× faster, and nearly as accurate. The result: "Now you have 10 million things to screen, you could do that while you go to lunch rather than it being a six-month endeavor." His optimization target: discoveries per unit of compute input.

Data efficiency. Today's large models see roughly 1000× as much data as a human does by age 18 — yet the 18-year-old is better at a lot of things and on par with frontier models at others. That gap is unsolved. By his own rule, it looks like a 0% problem.

One last thing. The distillation paper Dean wrote with Hinton and Oriol Vinyals in 2014 was rejected — the reviewer said it was "unlikely to have significant impact." Gemini's Flash models are distilled from the larger Pro model today.

His take is remarkably calm: "It gets rejected every so often, that's fine. We put it on arXiv, people read it, people use it, it's all good."

---

Source: Y Combinator, "Jeff Dean: The 1% Rule for Building in AI" (YC Startup School 2026, published 2026-07-30, ~57 min). Quotes are from the recorded conversation; views are the speaker's own.