AI reliability is how consistently an AI system, or software an AI has changed, does the correct thing under real production conditions, measured over time. Most writing on the term covers only the first half: whether a model gives good answers. For engineering teams in 2026 the second half matters as much, because agents now write and merge a growing share of the code that runs in production.
Short version: reliability for AI has two halves.
- Runtime half: whether a model, an LLM app or an agent behaves correctly and consistently while it runs. Judged with evals, tracing, guardrails and consistency across runs.
- Change half: whether the code, config and infrastructure an agent produces behave correctly once merged and deployed. Judged with review, tests, merge gates and change risk scoring.
- Where they meet: an agent that acts, because its output is a change.
Uptime measures neither. You need a quality signal next to availability, and a change-level view of what agents ship.
NIST's AI Risk Management Framework borrows the ISO/IEC definition of reliability: the ability of an item to perform as required, without failure, for a given time interval, under given conditions (NIST AI RMF). That definition fits both halves. What changes is which "item" you're judging.
The two meanings of AI reliability
| Reliability of AI systems | Reliability of AI-written changes | |
|---|---|---|
| What is being judged | A model, LLM app or agent at runtime | Code, config or infra an agent produced |
| Typical failure | Wrong, inconsistent or unsafe output | A merged change that breaks production |
| Who notices first | Users, evals, support tickets | On-call, error rates, rollbacks |
| Main tools | Evals, tracing, guardrails | Review, tests, merge gates, change risk scoring |
AI SYSTEM AT RUNTIME AI-WRITTEN CHANGE
model, LLM app, agent code, config, infrastructure
judged by evals, judged by review, tests,
tracing, guardrails merge gates
\ /
\ /
AN AGENT THAT ACTS: its output is a change
(both halves apply when an agent edits, migrates or deploys)
The two meet when an agent takes actions. An agent that edits files, runs migrations or calls production APIs is an AI system whose output is a change. For the broader picture on the second column, see What Is Production Reliability in Software Engineering?
Is uptime the same as reliability for an LLM?
No. A traditional service is mostly reliable when it's up and fast. An LLM endpoint can return 200 OK in 400 ms with a wrong answer, and the availability dashboard stays green the whole time.
Uptime is necessary, not sufficient. An LLM feature needs at least one quality signal next to the infrastructure ones. Without it, a prompt change that quietly lowers answer quality looks like a perfect week.
How to measure AI reliability
Google's SRE book defines an SLI as a carefully defined quantitative measure of some aspect of a service, and an SLO as a target for it (SRE book). The same approach works for AI, with a few extra indicators.
| SLI | What it catches | How to compute it |
|---|---|---|
| Availability | Provider or gateway outages | Successful responses / total requests |
| Latency (p95) | Slow tool loops, long generations | p95 of end-to-end request time, including retries |
| Eval pass rate | Quality regressions after a prompt or model change | Pass rate on a fixed, versioned eval set, run on every change |
| Consistency | Answers that flip between runs | Agreement rate across N runs of the same input |
| Guarded-action rate (agents) | Agents attempting risky actions | Tool calls blocked by policy / total tool calls |
| Change fail rate (AI-authored) | Agent PRs that break production | DORA's change fail rate, filtered to AI-authored changes |
DORA defines change fail rate as the ratio of deployments that need immediate intervention after they go out (DORA). Splitting it by author type is the simplest way to see whether agent-written changes are more or less reliable than human ones in your codebase. A PR label or the bot's author name is usually enough. How to Measure Software Change Reliability goes deeper on the change-level metrics.
Why agents turn reliability into a change problem
In July 2025, SaaStr founder Jason Lemkin described Replit's agent deleting his production database during a code freeze, then telling him a rollback wouldn't work when it actually did (The Register). Model quality wasn't the issue. The failure was an agent with write access to production and nothing between its decision and the database.
That's the part of AI reliability most teams under-invest in. Evals tell you whether the model is good on average. They don't tell you whether this specific change, to this specific service, is safe to ship. That question belongs to change risk management.
How to set reliability targets for AI features
- Pick one quality SLI per feature, not five. Eval pass rate is usually the right first one.
- Version the eval set and run it in CI on every prompt, model or retrieval change.
- Set the SLO from a baseline week of data, not a guess.
- Tag AI-authored PRs so change fail rate can be split by author.
- Decide which failures block a merge or a deploy, and which only alert.
Common mistakes
- Treating a provider's status page as your reliability number.
- Running evals once at launch and never again.
- Measuring the model while ignoring the code the agent ships.
- Averaging consistency across all inputs, which hides the few prompts that flip every time.
Tomosu works on the change half. It scores each pull request with a 0 to 100 Production Reliability Index before it merges, with an advisory Ready or Not Ready to Merge verdict at 70. A plain scan covers the code; the production-side sub-indices need observability and ticketing integrations connected. Run a free PRI scan on one repository.
Originally published on tomosu.ai.
Top comments (0)