TL;DR
VIDRAFT's text-to-SQL system, Darwin-180B-NL2SQL, scored 83.80% execution accuracy (419 of 500) on BIRD Mini-Dev (SQLite), the official compact leaderboard of BIRD, the most cited text-to-SQL benchmark. The current listed #1 is 81.00. We used no gold SQL and no fine-tuning, published the predictions and scorer, and submitted the result to the BIRD team for listing. We also opened a public demo of the engine, in Korean and English.
- Result and predictions: huggingface.co/FINAL-Bench/Darwin-180B-NL2SQL
- Live demo: final-bench-nl2sql-demo.hf.space
- Leaderboard: bird-bench.github.io
What is BIRD?
BIRD was built by researchers from the University of Hong Kong, Alibaba and others and presented at NeurIPS 2023. It uses large, messy, real-world style databases. A prediction counts only if running it returns exactly the same rows as the reference query. Human data engineers score about 93. The main test leaderboard, scored by the BIRD team on a hidden test set, is led by AT&T at 83.40.
Mini-Dev is BIRD's official 500-question development leaderboard. We used the current canonical release (2026 revision "Mini-Dev 1024").
The result
| System | BIRD Mini-Dev SQLite EX |
|---|---|
| VIDRAFT Darwin-180B-NL2SQL | 83.80 (submitted) |
| schemagate + Claude Opus 5.5 | 81.00 |
| Jitto Build | 75.60 |
| Ontology2SQL | 70.20 |
| schemagate + Gemini 2.5 Flash | 63.60 |
| Difficulty | Accuracy |
|---|---|
| Challenging | 83.5% (384 / 460) |
| Moderate | 90.6% (29 / 32) |
| Simple | 75.0% (6 / 8) |
The model is Darwin-180B-RSI with unchanged weights. VIDRAFT's verification engine runs each generated query against the database and the model revises it from what the execution shows. The engine is not released. Gold SQL was used only for the final scoring. We scored twice on an idle machine with identical results.
What the demo does
The demo turns everyday questions into verified SQL, tables, charts and a DOCX report. It has four data sources:
- AML voice-phishing tracing. Pick a bank's information request (Korean statutory Form 7). The engine traces where the stolen funds entered an e-money account and where they left, then drafts the statutory reply (Form 8). Every figure and remark comes from a query; no sentence is generated. All institutions, people and accounts are fictitious.
- Cash management on a synthetic public-sector database.
- Approved drugs from ChEMBL public data (4,225 drugs).
- Your own CSV or Excel files.
Safety is layered: lookup-only prompts, a syntax-tree check that allows one SELECT, a read-only connection, time and row limits, and a per-visitor audit log.
What's next
We are preparing a submission to the main BIRD test, where the BIRD team runs the system on a hidden test set.
FAQ
Is this the BIRD main leaderboard?
No. It is the official Mini-Dev leaderboard. The main test uses a hidden set and is a separate ranking.
Was the gold SQL used?
No. Only the question and evidence are inputs. Gold SQL was used only to score.
Was the model fine-tuned on BIRD?
No. The weights are Darwin-180B-RSI, unchanged.
Is it listed yet?
We submitted the predictions to the BIRD team. Listing is up to them.
Can I try it?
Yes, at final-bench-nl2sql-demo.hf.space.
Originally published on VIDRAFT AI Media.
Top comments (0)