DEV Community

Cover image for ResolveAI Puts a Specialized AI Agent on the Production On-Call Path
Jaromir Tesar
Jaromir Tesar

Posted on Originally published at spaisee.com Fully Autonomous

ResolveAI Puts a Specialized AI Agent on the Production On-Call Path

Quick take

  • Reinforcement learning raised SRE-1’s passing result from 6.9% to 38.8% across 49 difficult live production cases.
  • An ensemble of five SRE-1 investigations matched the strongest frontier model ResolveAI evaluated across those cases, while costing less.
  • ResolveAI says its system solved approximately twice as many incidents as general-purpose coding agents across 178 production-alert investigations.
  • ResolveAI says Primary On-call handles up to 80% of paged alerts without escalation, while Salesforce closed more than 70% during early dark-launch testing.

Why it matters

For reliability teams, incident response is shifting from human-led triage aided by general coding assistants toward specialized systems that investigate alerts and take policy-bound action, escalating when they cannot safely resolve them. The next test is whether ResolveAI’s results hold up across broader customer use, unusual failures and decisions about when acting is safe.

The next competition in enterprise AI may not be about building the largest general-purpose model. It may be about making smaller models reliable enough to investigate, and sometimes act on, the systems businesses cannot afford to break.

Software incidents expose a weakness in the usual coding-agent model. A general assistant can inspect code and suggest a likely fix, but a production investigation often depends on evidence that exists only briefly in live telemetry. The order of queries matters. Logs may expire, metrics may change, and an apparently harmless action can destroy the clues needed to identify the root cause.

ResolveAI is betting that these conditions justify a separate category of model. The company has launched SRE-1, a 27-billion-parameter system trained specifically for production investigations, alongside the generally available Primary On-call product, which is designed to receive alerts, investigate systems and take action when policies allow.

In its technical paper describing SRE-1, ResolveAI Labs says the model was developed with open weights, reinforcement learning and production-reliability environments. The final training run used as many as 512 GPUs and lasted 7.4 days. The objective was not simply to produce better code, but to teach the model how to gather evidence through tools and make progress toward a verified diagnosis.

That distinction is important for the business case. In a conventional software workflow, an incorrect suggestion may be rejected during review. In an incident, delay itself has a cost, while an automated change can cause downtime, data loss or a wider failure. A model operating on the critical path therefore needs to choose useful investigative steps before it attempts a remedy.

Reported gains from reinforcement learning

ResolveAI reports that reinforcement learning substantially improved the model's performance on a set of 49 difficult live production cases. A single SRE-1 investigation produced a passing result in 6.9% of cases before reinforcement learning, compared with 38.8% afterward, using the same incidents and tools.

The company says an ensemble of five SRE-1 investigations matched the strongest frontier model it evaluated across those cases, while costing less. Running several investigations can allow a system to compare competing hypotheses, although it also introduces additional latency and infrastructure expense. The reported result suggests that a specialized model may gain efficiency not only through its architecture, but through a workflow designed around production evidence.

A second evaluation covered complete systems rather than isolated model runs. ResolveAI says its system was tested on 178 production-alert investigations and graded against verified root causes. On that measure, it solved approximately twice as many incidents as general-purpose coding agents tested on the same alerts. Its advantage was reported as 15% to 25% on the hardest third of cases.

Those numbers are company-reported results, not an independent industry benchmark. Their significance will depend on how representative the alerts are, how difficult cases were selected, and whether the verification process distinguishes a plausible explanation from a demonstrably correct one.

From assistant to operator

ResolveAI’s launch announcement frames Primary On-call as more than a chatbot for engineers. The product can investigate telemetry, infrastructure, code and operational knowledge, then close or mitigate incidents when customer-defined policies permit. When it cannot safely resolve a problem, it is intended to escalate with an attached investigation for a human engineer.

ResolveAI says the product handles up to 80% of paged alerts without escalation in production. It also says Salesforce closed more than 70% of paging alerts without waking an engineer during early dark-launch testing. Those claims could represent a meaningful shift in the economics of reliability work, particularly for companies whose on-call teams spend much of their time handling repetitive alerts.

They also raise the hardest question for this category: what counts as a safe resolution? Suppressing a noisy alert is not equivalent to restoring a service, and mitigating symptoms is not always the same as fixing a root cause. Customers will need audit trails, approval controls, rollback mechanisms and clear boundaries around destructive actions.

The available reception cited by BusinessWire’s account of the launch was supportive. Snowflake engineering executive Vivek Raghunathan said SRE-1 showed what a domain-specific production model could deliver, while Zscaler reliability chief Misha Kuperman described significant opportunity to work with ResolveAI. No opposing reaction was included in the supplied coverage.

The broader implication is that enterprise AI may fragment by workflow. General models will remain useful for breadth, but high-cost operations may favor systems trained around narrow objectives, specialized tools and measurable outcomes. ResolveAI’s challenge now is proving that its reported success survives broader customer use, unusual failures and the moments when the model must decide not only what happened, but whether it is safe to act.

Sources


Originally published at spaisee.com.

This article was generated using AI and published automatically without human pre-publication review.

Top comments (0)