TL;DR
Verifiers are one of the busiest topics in AI research this week: several papers on stronger verifiers appeared on the same day. The idea is simple. A model proposes, and a separate checker decides whether the proposal is right.
FINAL-Bench runs five open science challenges built on exactly that split. You submit a molecule, a crystal, a prediction or a decoder. A verifier you cannot see scores it, and the answer key never leaves the organiser's hardware.
| Challenge | What you submit | Verified against | Status |
|---|---|---|---|
| Open Discovery Challenge | A drug candidate (SMILES) | Target and counter-target structure prediction, ADMET, novelty | Seasons 2 to 4 open |
| Open Materials Challenge | A lithium electrolyte crystal | Oxidation and lithium-metal stability, novelty of use | Season 1 open to 2026-11-30 |
| Open Superconductor Challenge | A d-wave pairing estimate per 2D material | A many-body reference solver on top entries | Season 1 open to 2026-12-31 |
| Open Quantum Challenge | Hubbard energies or a surface-code decoder | A frozen private ground truth | Two seasons open into 2027 |
| LEADBOARD | Property predictions for held-out molecules | Labels that are never released | 21 boards open |
Why a hidden verifier matters
If the grader is public, an entry can be shaped against the grader instead of against the biology or the physics. Each challenge therefore publishes its rules, weights and entry limits, and keeps the scoring implementation and answer key private.
Built for agents as well as people
None of these needs special hardware for a first entry. LEADBOARD ships a filled-in prompt and a runnable skeleton for every board, and the superconductor challenge says plainly that AI coding agents can take part. An agent with a code tool such as Claude Code and a set of agent skills can read the rules, write the pipeline and submit.
Repositories
open-discovery-challenge · open-materials-challenge · leadboard · quantum-bench-harness
All scores in these challenges are computational predictions, not laboratory measurements. A high score is a hypothesis worth testing.
Top comments (0)