Semantic Similarity Isn't Relevance: Stopping an Incident Agent From Hallucinating Across Services

B Gopinath — Tue, 29 Sep 2026 17:49:05 +0000

The first bad answer my incident agent gave me was also the most convincing one.

I had submitted a pool exhaustion incident for a service that had never had one. The agent replied with a crisp diagnosis: a connection leak in OrderPaymentProcessor. Correct format, plausible cause, confident tone. The only problem was that the service in question had no class by that name. The memory it drew from belonged to a different service entirely.

Vector similarity had done its job. The two incidents were similar. That's exactly why it was dangerous.

Why incident memory is different

In a general-purpose RAG system, a semantically close chunk is usually a good chunk. Incidents break that assumption, because the useful unit isn't just "what happened" but "what happened to this system." Two services can both exhaust connection pools for entirely different reasons. A fix for one might be meaningless, or harmful, for the other.

So the design question became: how do I let the agent use cross-service knowledge as a hint without letting it treat that knowledge as fact?

Step one: make service a parseable field

Every memory in the bank follows one format:

[<TIMESTAMP>] INCIDENT: <TITLE> | SERVICE: <SERVICE_NAME> | ROOT CAUSE: <ROOT_CAUSE> | FIX (<OUTCOME>): <FIX_DESCRIPTION>

The SERVICE: segment is what everything else hangs on. Recall is done through Hindsight, which blends temporal, entity, semantic, and recency strategies (TEMPR), so it already does better than raw nearest-neighbor. Service names are naturally entities, which helps. But I didn't want to rely on the retrieval layer alone to enforce a correctness boundary. Retrieval finds candidates; my code decides what they're allowed to mean.

That decision lives in relevance.py. It parses each recalled memory back into structured fields and compares the memory's service to the incident's service.

Step two: three tiers, not a threshold

I first tried a score cutoff: below 0.7, drop it. It didn't work, because score measures similarity and I needed to measure applicability. A 0.94 cross-service memory is still cross-service.

So each memory gets a grade:

direct_match: the memory is from the same service (is_same_service == True). This is the only tier that can ground a high-confidence recommendation.
analogy: a different service with a similar failure pattern. The agent can say "this resembles a pool exhaustion we saw elsewhere," but it is prohibited from claiming that cross-service internals exist in the target. Confidence is capped at medium.
low: spurious matches, filtered out or pushed down the ranking.

The grade travels with the memory all the way to the client:

  "id": "mem_01",
  "score": 0.94,
  "service": "order-service",
  "is_same_service": true,
  "outcome": "WORKED",
  "relevance_grade": "direct_match"
}

The client shows the grade next to each recalled memory. When an engineer looks at a recommendation, they can see whether it rests on a direct precedent or on an analogy, and calibrate accordingly.

Step three: give the model rules, not vibes

The grades reach the LLM as constraints. The call goes to Groq's openai/gpt-oss-120b at temperature 0.2 with a JSON response format. The model must produce:

interface StructuredRecommendation {
  likely_root_cause: string;
  confidence: "high" | "medium" | "low";
  historical_evidence: string[];
  recommended_fix: string;
  citing: string;
  runbook: string[];
}

The constraint on analogies is stated explicitly in the prompt: analogies may inform the pattern but may not import specific component names, configuration, or code paths from another service. And citing must name the memory the recommendation depends on. If the model says "high" but cites an analogy, that inconsistency is visible right there in the response.

This split matters. I don't ask the model to judge whether a memory applies to a given service. That's a boolean I can compute exactly. I ask it to write clearly within limits that code has already established. Models are good at the second job and unreliable at the first.

What happens in each case

Direct match. An order-service pool incident with a recalled order-service leak memory returns high confidence, the leak diagnosis, a circuit breaker fix, and a citing line referencing the 14 May precedent.

No history. A service with no prior incidents gets a recommendation that stays generic: symptom-driven diagnostic steps, medium or low confidence, no invented class names. This is a scenario in the test suite (test_scenario_a_and_c_no_matching_memory_and_unrelated_service), and it asserts that components from other services don't leak in.

Analogy. A different service with similar symptoms gets something like: "this pattern has appeared on another service, so check for leaked connections around external calls," with confidence capped at medium and no claim about that service's internals.

That third case is the one I'm proudest of. The agent is still useful when it doesn't have a direct precedent. It just says less, and says it honestly.

The cost of doing it this way

It's not free. A few things I'd flag for anyone copying this approach:

It depends on service naming discipline. If order-service and orders-svc are the same thing in practice, exact matching will miss it. Canonical service names are a real requirement, and you'll need a normalization step or a registry.
Analogies are conservative by design. Sometimes a cross-service memory really does apply, and the agent will under-claim. I accept that. In an incident, under-claiming is cheaper than a confident wrong fix.
Parsing is a contract. Because relevance.py parses the memory line format, any change to that format has to be made in the writer and the parser together. There's a dedicated unit test file for parsing, service matching, and ranking, because this is the part I least want to break silently.
Lessons learned

Similarity measures resemblance, not applicability. If your domain has a scope boundary (service, tenant, environment, region), enforce it in code rather than hoping the embedding captures it.
Grade before you generate. Compute what each piece of context is allowed to justify, and pass that to the model as a constraint.
Cap confidence structurally. I don't ask the model to "be careful." The tier determines the maximum confidence, and the schema carries it.
Surface the grade to humans. Showing "direct match" or "analogy" next to each memory lets a tired engineer decide how much to trust it in about a second.
Test the negative cases. The most valuable test I have checks that something doesn't appear in the output.

If you're building on a memory layer, the Hindsight docs are worth reading for what recall gives you out of the box, and Vectorize's overview of agent memory covers the broader design space.

Hindsight — Why Your AI Agent Needs Memory
Vectorize — What Is Agent Memory?
Hindsight — Agent Memory Evaluation Guide

DEV Community: B Gopinath

Semantic Similarity Isn't Relevance: Stopping an Incident Agent From Hallucinating Across Services

Why incident memory is different

Step one: make service a parseable field