DEV Community

Ankita paul
Ankita paul

Posted on

Most "AI-Powered" ITSM Tools Answer Questions. Ours Closes Incidents.

A look inside PulseServe's AIOps pipeline — the correlation-to-remediation engine, the self-correcting review loop, and the runbooks that rewrite themselves — and why that's a different bet than the chatbot most ITSM vendors shipped instead.

There is a specific moment that reveals whether an ITSM product's "AI" is load-bearing or decorative: the moment right after a P1 gets logged. Does anything happen, or does a chat window just wait for someone to type a question into it?

Most of the industry chose the second path. Over the last two years, "AI-powered" became the default label on every ITSM release note — a summarization box on the ticket, a suggested-articles widget, a chatbot that paraphrases the knowledge base back at whoever asks. It's useful. It is also, almost without exception, a read-only layer sitting on top of the same manual triage-diagnose-fix workflow that's existed since ServiceNow's first release. The AI answers questions. A human still does the work.

We built PulseServe's AI layer to do the opposite: sit inside the incident lifecycle as an actor, not beside it as an assistant. It correlates signals, proposes a root cause with a confidence score, drafts a remediation, and — when a human rejects its answer — takes the rejection as a training signal and tries again inside the same incident, before a person even leaves the page. That's a fundamentally different product decision, and it's worth walking through exactly what it looks like in practice, because "AI does the triage" sounds like a slogan until you see the gates around it.

1. The ceiling most ITSM "AI" hits

Every major platform has shipped a version of the same feature by now. ServiceNow's Now Assist and Predictive Intelligence will summarize a case, suggest a category, or generate a resolution note. Freshservice's Freddy AI will draft a reply and surface similar tickets. Atlassian Intelligence inside Jira Service Management will summarize a thread and suggest a next action. BMC Helix has its own generative layer doing much the same. These are genuinely good features — text generation and classification are exactly what large language models are good at, and none of this is being dismissed as fake.

But look at the shape of what they automate: information retrieval and text generation, gated entirely behind a human reading it and deciding what to do next. None of them run a multi-stage analysis pipeline. None of them carry a confidence score forward into a workflow decision. None of them re-attempt an answer because a human rejected the first one. The AI's job ends at "here's a suggestion" — which is a chatbot wearing an ITSM skin, not an incident-response system with a brain in it.

The tell isn't whether a platform has AI. It's whether the AI can be wrong in a way the system notices and recovers from — or whether every mistake just becomes a slightly-worse chat reply a human has to catch.

2. An actual pipeline, not a suggestion box

When an incident lands in PulseServe, it doesn't get one AI feature — it enters a four-stage orchestrated pipeline that runs automatically, with a tenant-configurable gate before every stage that carries risk:

Signal Correlation → Triage Routing → RCA Analysis → Remediation

Signal Correlation runs first and always — it's deterministic, not model-dependent. It pulls in the incident's monitoring/event/log/change (MELT) window, checks for recent changes against the same CI, and looks for a cascade pattern across other open incidents, so a single upstream failure doesn't spawn eleven duplicate root-cause investigations. If it detects a cascade, it can escalate priority or declare a major incident on its own, before RCA even starts.

Triage Routing runs in parallel, assigning the incident to the right queue based on the same enriched context, so routing isn't waiting on the slower reasoning stage behind it.

RCA Analysis is where the actual reasoning happens, and it's genuinely agentic — the model isn't given a prompt and asked to guess, it's given tools and has to go investigate:

  • Check the Known Error Database first for a pre-validated match — the highest-confidence signal available, checked before anything else.
  • Pull similar previously-approved RCAs, so a root cause a human validated last month directly informs this one.
  • Pull sibling incidents in the same cascade group, similar problems, and correlating changes in the last 72 hours.
  • Only then produce a hypothesis — a one-to-two sentence root cause, a type, and a numeric confidence score, with an explicit instruction that if the evidence doesn't support a real answer, it must say so and drop confidence below 0.4 rather than inventing one.

Remediation then drafts concrete next steps from that hypothesis — but only if the gates upstream allow it to.

Hypothesis:  pg_hba.conf reload after CHG0010001 blocked app subnet connections
Confidence:  0.87  (above KEDB auto-match threshold)
Stage history: correlation → triage_routing → rca → remediation
               (committed independently, per phase)
Enter fullscreen mode Exit fullscreen mode

That last detail matters more than it sounds like it should: each stage commits to the database as it finishes, rather than the whole pipeline writing once at the end. So the incident timeline shows real stage-by-stage progress — an operator watching the ticket sees correlation land, then triage, then a "thinking" RCA stage, instead of four AI badges appearing simultaneously with no sense of what actually happened when.

3. The part almost nobody else has: it can be told "no," and it learns for the rest of that incident

This is the feature that most clearly separates "AI suggests things" from "AI participates in the workflow." Every AI hypothesis in PulseServe goes through a human-in-the-loop (HITL) review before it's allowed to affect anything real. A reviewer has three options: approve, reject with a reason, or refine it by hand.

Approving does more than close a checkbox — the approved incident gets re-embedded as a gold-standard reference for future similarity search, and if the remediation steps look reusable, the system drafts a structured runbook proposal from them automatically.

Rejecting is where it gets interesting. A reject doesn't just discard the hypothesis — it automatically triggers a fresh pipeline run, with the rejection reason and the specific hypothesis that got rejected fed back into the RCA prompt as an explicit "do not repeat this" constraint. The model has to go find genuinely new evidence using tools or reasoning paths it didn't use the first time, not restate the same guess with different words. This is capped at two automatic retries — deliberately, so a model that keeps producing rejectable answers can't loop indefinitely burning inference calls — but in practice we've watched it produce three distinct, non-repeating hypotheses across a reject → retry → reject → retry chain on the same incident, entirely unattended.

Reviewer:       reject — "wrong CI, this is the read replica not primary"
Self-heal:      auto-retry 1/2 (rejected hypothesis injected as constraint)
New hypothesis: replica lag from a stalled WAL sender, not a primary-side config change
Enter fullscreen mode Exit fullscreen mode

The same self-correction shows up in the embedded ticket copilot, internally called ROBO. If a user gives an answer a thumbs-down, ROBO doesn't just apologize — it re-reads what was actually asked and re-investigates using tools or search terms it didn't use the first time, then either corrects itself or asks one clarifying question. Feedback is a training signal inside the conversation, not a satisfaction metric collected for a dashboard nobody acts on.

4. Runbooks that evolve instead of going stale

Static runbooks are the quiet failure mode of every ITSM knowledge base — they get written once after an incident, then slowly drift out of date as the underlying system changes, and nobody's job is to notice. PulseServe's runbooks track their own effectiveness: every execution updates a usage count and a success count, and the success rate is derived at read time rather than stored, so it can never drift out of sync with the executions that produced it.

When a new resolved incident teaches the system a step that an existing matched runbook doesn't have, the runbook doesn't get silently overwritten and it doesn't get quietly ignored either — it forks. A new pending-review version is created carrying the union of the old steps and the new one, linked to the version it's meant to replace. A human approves the fork, which automatically retires the version it superseded. You can walk the full version chain of any runbook forward and backward, so "why did this step change" is always answerable instead of lost in an edit history nobody kept.

It's not only incidents

The same philosophy — real tool-use and reconciliation, not just text generation — runs through the rest of the platform. Active network discovery scans reconcile automatically into the CMDB instead of requiring manual data entry. Software Asset Management normalizes messy vendor catalog data and raises compliance alerts before a license actually lapses. None of it is a chatbot bolted onto a static database; it's automation that changes the state of the system, under review.

5. How this actually compares

None of this is a knock on any specific competitor's engineering — it's a difference in product philosophy about where AI sits relative to the workflow. Here's the honest comparison, scoped to what's publicly documented about each platform's AI capabilities as of this year:

Capability PulseServe ServiceNow Freshservice Jira Service Mgmt BMC Helix
Multi-stage automated pipeline (correlate → triage → RCA → remediate) Yes, gated end-to-end Predictive Intelligence classifies; no chained pipeline Freddy suggests; no chained pipeline Summarizes/suggests only Similar single-step suggestions
Confidence-scored root cause with tool-based evidence gathering Yes — KEDB, prior RCAs, cascade & change context Not exposed as a scored hypothesis No No Partial, vendor-hosted models only
Self-correcting retry after human rejection Yes — capped auto-retry, rejection fed back as a constraint No equivalent No equivalent No equivalent No equivalent
Runbooks that version/fork from new learnings Yes — usage/success tracked, auto-fork + supersede Static playbooks Static, manually edited Static Static
Ticket-embedded, tool-using copilot with feedback-driven self-correction Yes — ROBO Now Assist (Q&A + drafting, no self-rectify loop) Freddy (drafting, no self-rectify loop) Atlassian Intelligence (summarize/suggest) Generative layer, Q&A style
Every AI action requires human approval before it affects state Yes, by design at every gate Configurable Configurable Configurable Configurable

The one row every vendor gets right is the last one, and it should stay that way. Which is really the point of the next section.

6. Automation with a leash, not automation with a blindfold

None of the above is an argument for letting a model run incident response unattended. It's the opposite: the entire design is built around the idea that AI is trustworthy in proportion to how visible its reasoning and its mistakes are. Every stage the pipeline runs is logged with its own timestamp and result. Every confidence score is stored, not just displayed once and discarded. Every rejection reason is persisted and directly shapes the next attempt. Nothing the AI proposes changes an incident, a CMDB record, or an asset's compliance status without a human clicking approve — the gates are configurable per tenant precisely so a team can decide for themselves how much autonomy to grant, module by module.

That's a deliberately different bet than "black-box suggestion, opaque confidence, take it or leave it." A wrong suggestion in a black-box assistant just sits there looking equally confident as a right one. A wrong hypothesis in PulseServe carries a low confidence score, a visible KEDB-miss, and a reviewer who can reject it and watch the system go find better evidence in front of them — auditable, not just accepted on faith.

7. What's next

The current pipeline runs on general-purpose models through a shared LLM gateway. The next piece under construction is a per-tenant Model Builder — the ability for a customer to fine-tune a smaller model (via QLoRA) directly on their own historical resolved incidents, so the RCA agent's first guess is shaped by their infrastructure and their past root causes, not just generic patterns. It's early, but it's the natural next step for a system that's already treating every approved RCA as training data — right now that data augments retrieval; the next version lets it actually shape the model's weights, tenant by tenant.


Top comments (0)