I stopped treating API changes as stateless with Hindsight

I used to think of API compatibility as a property of a diff: compare the old contract with the new one, classify the change, and move on. That works until the interesting question becomes, “Who actually depends on this field?”

That question is why I built API Sentinel around Hindsight agent memory. The important part of the system is not that an LLM can explain an API change. It is that the analysis can be grounded in dependencies the system learned before the current request existed.

The problem is not detecting a removed field

Consider a Course API with a description field. Removing that field is easy to describe mechanically:

Course API v1
id
title
description <-- removed
category

A conventional compatibility checker can reasonably say that removing an existing response field may break consumers. But that answer is incomplete.

The useful question is whether anything we care about actually uses description.

In API Sentinel, an application can register a dependency such as:

{
"application": "E-Learning App",
"api": "Course API",
"field": "description"
}

That fact is persisted through the Python agent into Hindsight. Later, someone can submit:

Remove description from Course API

The compatibility analysis is no longer based only on the sentence describing the change. It can retrieve the earlier fact that the E-Learning application depends on that field.

That is the design decision I found most interesting: the API change is stateless, but the compatibility system does not have to be.

How the system hangs together

I deliberately split the application into a conventional Java service and a separate Python reasoning service.

The browser talks to a Spring Boot backend. Spring Boot owns API endpoints, API changes, persistence, and the HTTP boundary to the agent. MySQL stores the application's structured records.

The Python service is deliberately narrower. Flask exposes /remember and /analyze. The agent talks to Hindsight for memory and to Groq for the language-model portion of the explanation.

The resulting flow looks like this:

This separation matters because I don't want memory retrieval, prompt construction, and model-specific code spread through the Java application. The Spring service can treat the agent as an HTTP dependency while the Python side owns the reasoning workflow.

The Java integration is intentionally small:

public String analyzeChange(String change) {

Map<String, String> request = Map.of(
        "change", change
);

return restClient.post()
        .uri("/analyze")
        .body(request)
        .retrieve()
        .body(String.class);

}

There is no Hindsight-specific logic in this layer. That boundary became useful as the system grew because API lifecycle data and agent memory have different responsibilities.

The important part happens before the LLM sees anything

The easiest mistake here would have been to send the proposed API change directly to the model and ask, “Will this break anything?”

I didn't want the model inventing an answer from general API knowledge. Instead, the agent first identifies the field involved in the change and asks Hindsight for relevant historical dependencies.

The core retrieval looks like this:

result = hindsight_client.recall(
bank_id=BANK_ID,
query=(
f"Which API consumers depend on the {field} field? "
"Return only direct consumer dependencies."
)
)

That query is intentionally narrower than “tell me everything you know about this API.” Compatibility analysis needs evidence about consumers, not a dump of historical context.

I then filter the returned memories before putting them into the model prompt:

for memory in result.results:
text = memory.text.lower()
field_lower = field.lower()

if (
    field_lower in text
    and (
        "depends on" in text
        or "dependency on" in text
        or "relies on" in text
        or "rely on" in text
    )
):
    matching_memories.append(memory.text)

In a production implementation, I would make this filtering more structured and schema-driven rather than relying on phrase matching. But the architectural point remains: retrieval happens before generation, and the retrieved evidence becomes part of the compatibility decision.

That distinction is important when the audience is another engineer who has to trust the result.

Hindsight gives the analysis a history

The dependency is stored with an explicit context:

client.retain(
bank_id=BANK_ID,
content=(
f"API: {api_name}. "
f"Consumer: {consumer}. "
f"Dependency: {consumer} depends on the {field} field."
),
context="api-consumer-dependency"
)

This is where Hindsight becomes more than a vector lookup hidden behind an AI feature. I can treat an observed dependency as something worth remembering independently of the request that caused it.

There is another useful decision in the implementation: I also retain the compatibility analysis itself.

hindsight_client.retain(
bank_id=BANK_ID,
content=(
f"API compatibility analysis: "
f"Proposed change: {change}. "
f"Field: {field}. "
f"Result: {status}. "
f"AI analysis: {ai_reason}"
),
context="api-compatibility-analysis"
)

That creates two different kinds of memory:

api-consumer-dependency records what applications depend on.
api-compatibility-analysis records what the system concluded about previous changes.

I like keeping those concepts separate. A dependency is an observed fact about a system. An analysis is a derived interpretation. Mixing them into one undifferentiated memory stream makes it harder to reason about provenance later.

The Hindsight documentation describes this general idea of retaining and recalling information for agents. In this system, I use it for something much less abstract: carrying API dependency knowledge from one engineering decision into another.

The LLM is an explainer, not the source of truth

Once relevant memories have been retrieved, the agent constructs a prompt containing the proposed change, the field, a preliminary compatibility status, and the historical dependencies.

One instruction is especially important:

Do not invent consumers or dependencies that are not present in the Hindsight memories.

That changes the role of the model.

The model is useful for turning evidence into a developer-readable explanation: what could break, which consumer is affected, why the field matters, and what migration action makes sense.

It should not be responsible for discovering facts about our architecture out of thin air.

This is also why I don't describe the system as simply “an AI API checker.” The interesting engineering problem is the boundary between deterministic application data, persistent memory, and probabilistic language generation.

What an interaction looks like

The first interaction establishes knowledge.

An E-Learning application registers:

E-Learning App
-> Course API
-> description

The agent retains that dependency in Hindsight.

Later, a developer enters:

Remove description from Course API

The agent extracts description, recalls historical memories concerning that field, and finds the E-Learning dependency.

The resulting status becomes:

POTENTIALLY_BREAKING

The generated explanation can then say, in concrete terms, that the E-Learning application consumes description, why removing it can affect that consumer, and that the change should be migrated or versioned rather than treated as an isolated schema edit.

Now consider the opposite case. A developer removes a field for which no known consumer dependency has been recorded. The system can report:

NO_KNOWN_IMPACT

That does not mean the change is proven safe. It means the system has no known dependency in its memory. That distinction is one I would preserve in the production version because absence of evidence is not evidence that an API is unused.

This is also where agent memory for AI systems maps cleanly to an engineering problem. The memory is valuable because the next request can depend on something learned during an earlier interaction.

Why I kept a conventional backend in front of the agent

It would have been possible to put the whole workflow into Python. I kept Spring Boot in the architecture because the surrounding application is still a normal backend application.

ApiEndpoint represents an API contract. ApiChange represents a proposed change and keeps fields such as changeType, description, oldValue, and newValue. Repositories persist those records through Spring Data JPA.

That gives me a useful division:

Structured state Agent memory

---

API endpoints Consumer dependencies
API changes Compatibility analyses
Versions Historical context

I don't want to force everything into a database just because it is structured, and I don't want to force everything into agent memory just because it is related to an AI workflow.

The compatibility service also has deterministic handling for basic change categories. Adding a field is generally treated differently from removing or modifying one. That baseline is useful even before historical context enters the picture.

The agent then adds the part a static rule cannot know: who has previously depended on the thing being changed.

What I learned

1. Compatibility is contextual, not just syntactic

A schema diff can tell me that a field disappeared. It cannot, by itself, tell me which internal application depended on that field.

If I care about real compatibility rather than theoretical compatibility, I need dependency information.

1. Memory should contain evidence, not just conversations

The most useful Hindsight records in this system are not chat transcripts. They are compact engineering facts: application, API, field, and dependency relationship.

That makes retrieval much easier to reason about and gives the model something concrete to cite in its explanation.

1. Separate facts from generated conclusions

I retain dependencies separately from compatibility analyses. That gives me a cleaner provenance model: one memory describes something observed; another describes a conclusion derived from it.

That separation becomes increasingly important as the system accumulates history.

1. “No known impact” needs careful wording

A compatibility tool should resist turning an incomplete dependency graph into false confidence. If Hindsight cannot recall a consumer, the honest result is “no known consumer,” not “safe.”

That small distinction is important enough to make part of the API contract and UI behavior.

1. The model should explain the evidence, not manufacture it

The LLM is good at producing a concise migration explanation. Hindsight is useful for supplying historical context. My application code is responsible for connecting the two and keeping their roles explicit.

That separation gives me a system that can improve in several directions without changing its fundamental contract: better dependency ingestion, richer API-change parsing, more precise retrieval, stronger provenance, and better migration recommendations.

Where I would take it next

The next step is not a bigger prompt. It is better system knowledge.

I would ingest dependencies automatically from service specifications, code usage, client libraries, and deployment metadata rather than relying on applications to register every relationship manually. I would also replace free-form dependency strings with structured memory metadata and make retrieval sensitive to API version, endpoint, consumer, and field.

The core idea would stay the same.

When an engineer proposes an API change, I don't want the system to ask only, “What does this change mean?” I want it to ask, “What have we already learned about this contract, and who has depended on it?”

That is the part I could not get from a stateless compatibility check. Giving the agent memory made the API change itself only half of the input.