Anthropic published results this week that are easy to file under "cool but not my field": Claude models autonomously designed de novo protein binders that succeeded against 14 of 15 targets. I almost skipped it for the same reason I skipped the Astra math proofs at first — I can't evaluate protein binding any more than I can evaluate a Lean proof about non-sofic groups. Then I looked at how the result was validated, and it's the same lesson twice, which usually means it's a real one.
The number that matters isn't the hit rate
The headline stats are genuinely strong. Across 1,320 designs against 15 targets, 354 binders were confirmed to actually work. Hit rates landed around 22–27% over 48-hour campaigns and up to 35% when a model focused on a single target — against a 10–15% industry norm. On one target, RBX1, Claude hit 40% where the human competition field managed 3.7%.
But the stat I keep circling isn't any of those. It's the word confirmed. These proteins weren't scored by another model or graded on a benchmark Anthropic controls. Adaptyv Bio and Twist Bioscience physically synthesized the designs and tested whether they bound, in a lab, with reagents. The AI proposed; the wet lab disposed. That's the entire difference between a press release and a result.
The pattern, for people who don't do biology
Here's why this belongs on a developer blog even though none of us are pipetting anything.
The workflow underneath is one I've now written about three times in different clothes: generate a flood of candidates, verify them with something that can't be fooled, keep the survivors. In the Astra math work, the model generated proofs and the Lean kernel checked them. Here, the model generated 1,320 protein designs and a wet lab checked them. In both cases the model's job wasn't to be right — it was to be productive, and the verifier's job was to be trustworthy. Ninety-six percent of the designs didn't bind. That's fine, because the check was cheap relative to the payoff and honest about failure.
This is the shape of AI work that actually holds up, and it's the shape most software teams still aren't using. We ask a model to write code and then we read it, which is the slow, fallible, human-in-the-loop verification we should be trying to escape. The protein result works because the verification is mechanical and the generation is disposable. The teams getting durable value from models are the ones building the wet lab — a checker so reliable that the model can be wrong 96% of the time and the system still wins.
What I'd actually take from this
Two things, one concrete and one directional.
Concrete: when you point a model at a problem, spend your design effort on the verifier, not the prompt. What's the cheapest mechanical check that separates a working output from a plausible-looking one? For proteins it's a binding assay; for code it might be a property test, a type checker, a simulation, a diff against known-good behavior. If your only verifier is "a human reads it and it seems fine," you've built the slow version of this, and you'll get the slow version's results.
Directional: the same detail that makes the protein story credible — third-party physical validation — is the detail I'd start demanding from every big AI claim. Anthropic also had Claude process raw NMR and lab chemistry data with purity readings within 0.1% of the lab's own instruments. That's checkable. "Our model is amazing at science" is not. As these claims get bigger, the ones worth believing will be the ones that hand you the assay, and the ones worth ignoring will be the ones that hand you a vibe.
The proteins are the flashy part. The wet lab is the part I'd copy. If you've built a genuinely mechanical verifier around a model in your own work — one good enough that you trust the output without reading it — I'd like to hear what the checker is.
Top comments (0)