We keep most of our work on Hugging Face, but code belongs where developers already look for it. So the open parts of what we build now live on GitHub at github.com/final-bench. Here are six projects, what each one does, and where to start.
1. ONGRID: GraphRAG that also decides
Most GraphRAG tools retrieve and generate. ONGRID adds the step that usually goes missing: deciding whether the answer can be trusted.
Drop in documents and ONGRID builds a 3D ontology graph of typed entities and relations. Ask a question and an LLM answers from facts in that graph. Then a confidence judge checks the answer without generating any tokens.
Repository: final-bench/ongrid
2. FINAL Bench: does the model know when it is wrong?
Accuracy benchmarks ask whether the final answer is right. FINAL Bench asks a different question: can a model detect its own error, admit it, and fix it?
It covers functional metacognition across 100 tasks in 15 domains, from mathematics and medicine to ethics and economics. The paper is on arXiv and the dataset is on Hugging Face.
Repository: final-bench/leaderboard
3. Open Quantum Challenge harness: the yardstick is public
A leaderboard is only as trustworthy as its scoring code, so we published the scoring code.
The harness holds reproducible scoring and frozen test problems for the Open Quantum Challenge: quantum simulation tasks and surface-code decoder scoring, reported as logical error rate with 95% Wilson intervals. Anyone can run it and get the same numbers the leaderboard shows. Contestants keep their own methods to themselves.
Repository: final-bench/quantum-bench-harness
4. POCKET: strong models on your own laptop
POCKET is a family of models quantized to run locally on a laptop CPU, with no GPU and no network connection.
The repository is the model index plus run recipes. The GGUF weights are on Hugging Face, where the POCKET builds have been downloaded more than two million times in total.
Repository: final-bench/pocket
5. ZTC: can you trust this answer?
ZTC (Zero-Token Confidence) reads a question and an answer written by any model and returns whether the answer can be trusted. It does this in one forward pass, without generating text.
It comes in 4B, 9B and 27B sizes, from CPU-friendly to highest quality. The repository holds the inference code, the model index and a description of the method.
Repository: final-bench/ztc
6. ALL Bench Leaderboard: one place to compare
Instead of hunting through scattered tables, ALL Bench Leaderboard gathers public benchmark results in one place so models can be compared side by side. It is released under Apache-2.0.
Repository: final-bench/ALL-Bench-Leaderboard
Where to start
- Building retrieval with verification: start with ONGRID and ZTC.
- Evaluating models: FINAL Bench and ALL Bench Leaderboard.
- Running models offline: POCKET.
- Quantum error correction research: the Open Quantum Challenge harness.
Issues and pull requests are welcome on every repository. If something does not run, tell us there and we will fix it.
Top comments (0)