Ask the same question twice and get two different numbers.
That's not a bug you can patch. It's the architecture telling you what it is.
Variance isn't a tuning problem
Probabilistic systems sample. Sampling means variance. Variance in a marketing subject line is fine; variance in a regulatory filing is a finding.
And setting temperature: 0 doesn't fix it — greedy decoding over an ambiguous schema reliably picks the same wrong join every time. You've traded variance for consistent error, which is worse, because now it looks reliable.
What deterministic actually means here
Not that the model stops being probabilistic. That the model never decides anything that matters.
| Decision | Probabilistic design | Deterministic design |
|---|---|---|
| Language → intent | Model | Model |
| Which entity / metric | Model guesses | Semantic graph resolves |
| Which join path | Model guesses | Planner proves |
| Authorisation | Post-filter | Policy engine, at compile time |
| SQL emission | Model writes | Compiler emits |
Four of the five decisions move out of the model. What's left is the one thing an LLM is genuinely good at: parsing natural language into structure.
The consequences that follow
- Same question, same graph version, same answer — every time
- Ambiguity produces a refusal, not a guess
- The answer is reproducible point-in-time, which is what auditors need
- Results are cacheable and reviewable, because they're stable
Why it shows in the numbers
On real enterprise schemas, raw schema access scored 14.5% in our benchmark. The same model with compiled, governed context scored 98.2%.
That's not a better model. That's removing every point at which the model had to guess.
The full breakdown — why "deterministic settings" aren't deterministic, the architectural split, and the benchmark methodology — is here:
👉 Deterministic vs. Probabilistic Text-to-SQL: Why Accuracy Matters
Originally published at colrows.com/blogs/deterministic-vs-probabilistic-text-to-sql
Top comments (0)