Ask Your AI Agent the Same Question Twice. If the Answers Differ, No Amount of Fine-Tuning Fixes That.
A language model is probabilistic. Ask it the same question twice, worded exactly the same way, and you can get two different answers back. That's not a bug. That's what the model is.
For a chatbot, nobody cares. For an agent approving a loan, paying a claim, or touching a patient record, it's disqualifying. Enterprises will not put agents into production where "usually right" is the only guarantee on offer, and they're correct not to.
We spent two years building the layer that fixes this, not by making the model better, but by refusing to ask the model to be the thing providing the guarantee at all. We open-sourced the core of it under Apache 2.0. It's called Determ8.
The fight nobody can win
The instinct every team has is to make the model more trustworthy. Bigger model. More context. Sharper prompts. Fine-tune on more examples until it stops making the mistake.
This doesn't work, and it's worth being precise about why, because the reason isn't "today's models aren't good enough yet." The reason is mathematical. A model that samples from a distribution cannot return a guaranteed answer, at any accuracy level, because sampling from a distribution is what it does. A more accurate distribution is still a distribution. You can make the wrong answer less likely. You cannot make it impossible, and "impossible" is the bar a regulated decision actually requires.
Even the arithmetic underneath a model isn't fully deterministic by default. Floating-point operations can produce different results depending on execution order, hardware, and batching, the same weights, the same input, a different answer, before anything about the model's own judgment even enters the picture. If you're trying to build a guarantee on top of that, you're building on sand and calling it foundation.
So the guarantee has to come from somewhere else. Not a better model. A different kind of layer sitting next to it.
What we built instead
Determ8 wraps a probabilistic model in a deterministic pipeline. The model stays exactly as probabilistic as it always was. Nothing about it changes. What changes is that its output is no longer trusted on arrival.
The agent proposes a candidate answer. That candidate enters a pipeline of gates, one at a time. Each gate answers exactly one question, deterministically: pass or fail. Cheap gates run first. Expensive gates run last. The moment a gate fails, the pipeline stops and rejects the candidate, and the whole run gets logged so it can be replayed and audited later. Only a candidate that clears every gate ships.
![The model proposes, the pipeline proves, and a single failed gate stops the run]
Every gate, free or paid, built-in or custom, implements the same contract:
gate(candidate, spec) -> Verdict(passed, reason, proof)
That single shared contract is the entire reason this composes. Because every gate looks identical to the pipeline, you can add one, remove one, reorder them, or write your own without touching anything else. The determinism guarantee is simple to state: same input, same gate configuration, same verdict, every single time, forever. Not "usually." Every time.
Where the line actually sits
Determ8 is open core, and the split between what's free and what's paid is itself a real design statement, not just a pricing decision.
![Four free structural gates versus four paid gates that require real formal-methods engineering]
The free tier handles structural correctness, pattern-matching against a known shape. The paid tier is where the hard, defensible gates live, formal proof, prebuilt regulatory rule packs, constraint solving, and cryptographically signed audit trails built for a regulator to inspect line by line.
That split exists because those two categories of correctness are genuinely different problems, and pretending they cost the same amount of engineering to build would be dishonest pricing.
A verdict, not a guess
Every run produces a PipelineResult. Nothing about that record is fuzzy. There's no confidence score anywhere in it, on purpose.
![The PipelineResult object: passed, failed_gate, reason, input_hash, and the full verdict trace]
Confidence scores are what you reach for when you can't actually prove the thing. The moment a system starts reporting "87% confident this loan decision is compliant," it has quietly admitted it cannot tell you whether the decision is compliant, only how comfortable it feels. Determ8 doesn't produce that number, because producing it would be lying about what kind of claim is actually being made.
The instinct this comes from, stated plainly
There's a rover on Mars right now that makes decisions no human is in the loop for, because at that distance a round-trip signal takes twelve minutes and the ground can't wait that long. The engineers who built it faced the same choice every team building agents faces: build something that acts with confidence, or build something that knows when to stop.
They chose to build something that stops. When uncertainty crosses a threshold, it halts and waits, rather than guessing at the safer path and committing anyway. That wasn't timidity. It's arguably the single most important engineering decision on the entire mission, because a system that knows exactly where its own certainty ends is more capable than one that never questions its own confidence, not less.
Enterprise AI mostly decided the opposite: that hesitation was the failure mode, that a model which stopped had somehow malfunctioned. So probabilistic systems got handed authority over processes that actually needed certainty, and for a while the industry called that progress. It wasn't. It was just a failure that hadn't been noticed yet, because production systems don't announce their own degradation. The gap between likely and correct just quietly hardens into operational reality until an audit finds it.
Where this actually matters
Banking and finance: loans, payments, fraud detection, trading. Healthcare: diagnosis support, medical billing, claims. Insurance: underwriting, claims approval. Legal and compliance: contract review, KYC, regulatory filing. Government: benefits and eligibility determination. Critical infrastructure: energy, automotive, manufacturing.
Low-stakes agents don't need any of this. A chatbot that occasionally phrases something oddly costs you nothing. A high-stakes agent that occasionally approves the wrong claim costs you an audit, a regulator, or a customer who can't get their money back. The gap between those two situations isn't a matter of degree. It's a different category of problem, and it needs a different category of solution, one that doesn't ask a probability distribution to hold a guarantee it was never built to hold.
The model proposes. The pipeline proves. That's the whole idea, and once you've seen the distinction, it's hard to unsee how many production AI systems are quietly skipping the second half.
The open core, the pipeline and the four free gates, is live on GitHub under Apache 2.0: github.com/Tech4Biz-Solutions-Pvt-Ltd/-Tech4Biz-Determ8. If you're putting agents anywhere near money, health, or compliance and need the formal verification, rule packs, or audit trail the Pro gates provide, get in touch.



Top comments (0)