A system returns an impressive answer. What happens next?
If the next step is “send it to someone experienced and ask whether it seems right,” the system is still bottlenecked by human confidence. If the next step is “run the artifact through a deterministic correctness check,” you have a different architecture.
That architectural difference is the important lesson in a reported set of mathematics results: ten open problems solved in one week for about $2,000 in total compute, or roughly $200 per problem. The outputs included machine-checkable Lean 4 proofs with a sorry count of zero.
For developers, the interesting question is not whether every software task can be turned into theorem proving. It cannot be assumed that it can. The useful question is: which parts of our systems can move from plausibility-based evaluation to enforceable verification?
Start with the acceptance function
Think of an AI-assisted system as two components:
generator(input) -> candidate
checker(candidate, specification) -> accept | reject
Most attention goes to the generator. Teams compare models, prompts, latency, and benchmark scores. But the reliability of the overall workflow often depends on the second function.
In Lean, sorry is a placeholder for an unfinished argument. Zero occurrences mean the submitted proof contains no such escape hatch and has been checked end to end by the proof assistant. The checker is not rating whether the prose is elegant or whether the reasoning resembles what a mathematician might write. It is enforcing the formal proof obligation.
That gives us a practical mental model: generated output is a candidate, not a result, until it passes the strongest relevant check available.
Check 1: Is correctness actually executable?
Before automating generation, write down the acceptance condition.
Ask:
- Can success be expressed as a test, type constraint, invariant, proof obligation, or other machine-enforceable rule?
- Does the check cover the entire output, or only a convenient subset?
- Can the generator bypass the check with a placeholder, ignored branch, or unsupported assumption?
- Is acceptance binary, or does the task still require a judgment call?
The Lean example is powerful because it has a crisp answer: the formal proof checks or it does not. The source emphasizes that the reported proofs had zero sorrys, so unfinished steps were not being hidden behind Lean’s escape hatch.
A common failure mode is verification theater. A pipeline displays a green status because some checks ran, while the difficult portion remained excluded. In software terms, this resembles a test suite that passes because the untested behavior contains the bug. The lesson from zero sorrys is not merely “use a checker.” It is “audit every escape hatch in the checking path.”
Check 2: Does the specification match the intended problem?
A checker can establish that a candidate satisfies a formal condition. It cannot, by that act alone, establish that the condition captures what people wanted.
Use two separate review questions:
1. Did the candidate satisfy the specification?
2. Did we specify the right thing?
Conflating them creates the wrong-target failure mode: a fully verified artifact answers a nearby, weaker, or otherwise unintended question. The first question may be automated. The second can still demand domain judgment.
This caveat does not weaken formal verification. It locates its boundary. Once the boundary is explicit, teams can spend human attention on the statement, assumptions, and relevance instead of repeatedly inspecting every generated step for plausibility.
Check 3: Is your benchmark measuring the capability you care about?
The source contrasts the mathematics results with benchmark charts that appear stalled. It notes that SWE-bench Verified had sat at 78.8% since September and that LMArena’s data was frozen at July 2024. Those numbers describe their tests. They do not automatically describe every capability a system may exhibit.
Treat every benchmark result as a tuple:
(score, task, dataset, interface, collection_time, acceptance_rule)
Dropping the rest of the tuple and retaining only score encourages overgeneralization. A stable number can mean that performance on that test is stable. It does not prove that unrelated, newly observable behavior is also stable.
Concrete review questions include:
- Does the benchmark accept the same artifacts our production system needs?
- Is its data current enough for the conclusion we are drawing?
- Does its interface permit the workflow we intend to deploy?
- Is it measuring a proxy, or applying the domain’s actual correctness condition?
- Could meaningful progress occur outside the tested distribution?
The failure mode here is instrument lock-in: continuing to treat one leaderboard as the definition of progress after useful behavior has moved beyond its measurement frame.
Check 4: Price verified outputs, not generated tokens
The reported average of about $200 per open problem is worth examining, but it should not be converted into a universal rate card for discovery. Ten problems do not establish that all problems have equal difficulty, that every attempt succeeds, or that every relevant cost is represented by compute.
The system-design insight is to choose the correct cost unit:
cost_per_candidate = total_cost / generated_candidates
cost_per_verified_result = total_cost / accepted_results
The second number is usually the one that matters. Cheap generation can be operationally expensive if expert reviewers must inspect every candidate. A higher generation cost may be acceptable if a reliable checker rejects bad candidates automatically and accepted outputs are genuinely useful.
This leads to several checks:
- Count failed attempts as well as accepted results.
- Include the cost of constructing and maintaining the verifier.
- Track human work required to define the specification.
- Separate verification cost from interpretation and deployment cost.
- Never infer a general price from a small, heterogeneous set without additional evidence.
The associated failure mode is candidate-cost accounting: celebrating inexpensive output while ignoring the cost of deciding whether it is correct.
Check 5: Find the escape hatches in your own stack
Lean’s sorry is explicit. Production systems often have less visible equivalents.
An escape hatch is any mechanism that allows an artifact to pass without satisfying the intended condition. Examples at the level of system design include skipped checks, acceptance based only on formatting, partial validation presented as complete validation, or a human approval step that has become a rubber stamp.
A useful audit looks like this:
for each acceptance path:
identify enforced conditions
identify unchecked assumptions
identify bypasses
identify who can override rejection
record what “green” actually guarantees
The goal is not to claim that every override is bad. Some domains cannot state complete correctness conditions, and human judgment may be essential. The goal is to prevent a partial check from being advertised as an end-to-end guarantee.
Where this model applies—and where it stops
The source extends the implication beyond mathematics to any domain that can state its own correctness condition. That qualification matters.
Some work has strong, executable conditions. Other work depends on taste, ambiguous goals, changing environments, or competing human values. In those areas, forcing a binary checker can conceal uncertainty instead of removing it.
A practical classification is:
A. Fully checkable: acceptance can be enforced end to end.
B. Partially checkable: invariants can be enforced, but relevance still needs judgment.
C. Judgment-led: no adequate executable correctness condition exists.
Do not apply category A language to a category B or C system. “Passed validation” should always answer the follow-up question: validation of what?
The system-design shift
The reported mathematics results suggest that the scarce resource may be moving. When generating candidates becomes cheaper and a proof assistant can reject invalid ones, more value moves toward selecting consequential questions, stating them precisely, and building trustworthy acceptance mechanisms.
That changes what developers should optimize. Model quality remains important, but a production architecture should not depend solely on an output looking convincing. It should make the transition from candidate to accepted result explicit.
The final review can be compact:
[ ] The intended claim is specified.
[ ] The checker covers the full artifact.
[ ] Escape hatches are absent or disclosed.
[ ] The benchmark matches the deployed task.
[ ] Cost is measured per verified result.
[ ] Human judgment remains where correctness cannot be formalized.
A leaderboard tells you how a system performed on a particular test. A verifier tells you whether a particular artifact satisfied a particular formal condition. Those are different instruments.
When the output is checkable, design around the check.
Format note: DEV gains an implementation-shaped mental model: generator versus checker, audit questions, cost units, and named failure modes that developers can apply to systems. Pasted into a long-form blog, the checklist structure would compress the broader reasoning; pasted into video, the code blocks and verification audit would be impossible to absorb at spoken pace.
Top comments (1)
switched to a similar approach last month and the difference is night and day.