Orbyt went from first commit to launch in 32 days with 243,000 lines and 4,124 tests, then kept getting rebuilt on models that had shipped only weeks earlier.
- Published benchmarks measure yesterday's questions on everyone else's problems. My own eval suite, scored mechanically and run on every release, rules on a new model within hours: helps, hurts, or nothing.
- Of 98 recorded incidents, 74 are now mechanized into permanent checks that no future release gets to un-learn. A finding that stays in someone's head restarts from zero on the next release.
- My gates discarded 30 of the last 51 agent runs at the boundary. Every one was written down with its conditions, because an unrecorded dead end is a tax you pay annually.
Read the full article: Frontier Development Is a Discipline.
This summary was written with AI assistance (Claude) from the full article at https://www.orbytlabs.ai/blog/frontier-development.
Top comments (0)