ViSQL: VIDRAFT's Execution-Verified NL2SQL Engine Hits 83.80% on BIRD Mini-Dev, Surpassing the Public Leaderboard
TL;DR: VIDRAFT has publicly demoed ViSQL, a Natural Language to SQL engine that converts plain-language questions into SQL, executes the queries against a live read-only database to self-verify correctness, and returns structured reports. Powered by VIDRAFT's Darwin-180B-RSI language model, the combined system (
Darwin-180B-NL2SQL) scored 83.80% execution accuracy on BIRD Mini-Dev (SQLite, 500 questions) — 2.8 points above the current public leaderboard top score of 81.00. If you're building data-access tooling for finance or enterprise, this architecture and its public benchmark results are worth a close look.
What it is
ViSQL is VIDRAFT's NL2SQL engine designed to take natural-language questions (Korean and English supported), generate SQL, execute that SQL against a connected database to verify the result is real, and then return the answer as a table, chart, or DOCX-format analytical report.
The underlying language model is Darwin-180B-RSI, VIDRAFT's flagship model which has claimed #1 rankings on 15 out of 53 recognized public benchmarks — including perfect scores on AIME 2026 and HMMT 2026 (mathematics), 94.44 on GPQA Diamond (science reasoning), and 90.29 on ExtractBench (document extraction). The NL2SQL configuration is published as Darwin-180B-NL2SQL.
The public demo supports four data domains:
- AML / Voice-phishing fund tracing — Given a Korean financial compliance form (Form 7), ViSQL queries synthetic transaction data and auto-fills the corresponding notification document (Form 8) with data-grounded figures.
- Public-sector fund management — Queries over a simulated government finance system, covering budget execution rates, anomalous transactions, and corporate card usage.
- Drug discovery (ChEMBL) — Natural-language queries over 4,225 approved drugs from the EMBL-EBI ChEMBL public dataset, including targets and indications.
- User-uploaded data — Upload a CSV or Excel file and immediately query it in natural language.
How it works
The key architectural idea is execution-based verification rather than pure generation. Most NL2SQL systems generate a SQL query and return it as-is. ViSQL goes a step further:
- Parse the natural-language input and generate candidate SQL.
- Execute the SQL against the actual (read-only) database.
- Verify that execution succeeds and the result is coherent — iterating if needed — without requiring gold-standard SQL labels or fine-tuning on task-specific examples.
- Format the verified result into a table, chart, or structured document report.
VIDRAFT explicitly notes this result was achieved without model fine-tuning on task-specific SQL examples and without reference ("gold") SQL answers — the engine relies on live database execution feedback as its correctness signal.
Safety guardrails baked into the engine include:
- SQL syntax validation before execution
- Read-only database connections (no write/DDL operations)
- Execution time limits
- Audit logging for all queries
Benchmarks & results
The benchmark in question is BIRD (Big Bench for Large-scale Database Grounded Text-to-SQL Reasoning), introduced at NeurIPS 2023 by researchers from HKU and Alibaba. BIRD runs two tracks: a held-out main test and a developer-facing abbreviated track called Mini-Dev.
| System | BIRD Mini-Dev Score (SQLite, 500 Qs) |
|---|---|
| Previous public leaderboard #1 | 81.00 |
| Darwin-180B-NL2SQL (VIDRAFT) | 83.80 (419/500 correct) |
| Human expert baseline (data engineers) | ~93.00 |
Key details to note:
- Metric is execution accuracy — the generated SQL must produce the correct result set when run, not just be syntactically similar to a reference.
- VIDRAFT has published its predictions and scoring artifacts, and has applied for official leaderboard registration with the BIRD team. The main test submission is planned as the next step.
How to try it
The public demo is accessible at VIDRAFT's site (linked from the press announcement). There is no publicly announced Hugging Face model card, GitHub repository, or OpenAI-compatible API endpoint for ViSQL or Darwin-180B-NL2SQL at the time of this writing. Access is currently demo-only.
Watch the official VIDRAFT channels for SDK or API announcements. This article will be updated when public programmatic access is confirmed.
FAQ
Q: How is execution-based verification different from standard NL2SQL generation?
A: Standard NL2SQL pipelines generate SQL from text and return it directly — correctness depends entirely on how well the model learned from training data. ViSQL's engine actually runs the generated SQL against the connected database and uses the execution result as a real-time correctness signal, iterating when a query fails or returns an incoherent result. This means it can catch logical and schema-mismatch errors that a generation-only system would silently pass through.
Q: Does the 83.80% score mean it was fine-tuned on BIRD training data?
A: No. VIDRAFT explicitly states that this result was achieved without fine-tuning on task-specific SQL examples and without gold SQL supervision. The engine's verification loop — executing queries live and using success/failure as feedback — is what drives the accuracy, not task-specific training signal.
Q: Is my data safe if I upload a CSV to the demo?
A: According to VIDRAFT, the engine enforces read-only database connections, SQL syntax checks, execution time limits, and audit logging. The voice-phishing demo explicitly uses entirely synthetic (fictional) entities, accounts, and institutions. For production or sensitive data, you should evaluate VIDRAFT's enterprise offering directly.
Q: What languages does ViSQL support for natural-language input?
A: The public demo supports Korean and English.
Originally reported by 조선비즈 (2026-10-30) — source article.
Top comments (0)