API compatibility is easy to describe as a diff. The harder problem is remembering which consumers made that diff risky.
I wanted API compatibility checks to answer a harder question than “what changed?” I wanted them to answer “who has depended on this before?” That distinction led me to build API Sentinel around persistent memory with Hindsight.
The interesting part is not detecting that removing a field can be dangerous. Any reasonable compatibility checker can tell me that. The interesting part is preserving the context that a normal API diff does not have: the history of consumers, dependencies, and previous compatibility decisions.
API diffs don't know who depends on them
An API contract can tell me that description disappeared from a Course resource. It cannot necessarily tell me which application still relies on that field.
That is the problem I focused on.
In API Sentinel, a developer can propose a change such as:
Remove description from Course API
The system extracts the field being removed, asks Hindsight which consumers have previously been recorded as depending on that field, and then gives that context to an LLM.
For the example I built around a course application, the important historical fact is:
E-Learning App depends on the description field.
That changes the analysis from a generic warning into something tied to an actual known consumer.
The core idea is simple:
The API change didn't change. The agent's knowledge changed.
That became the design principle for the whole system.
I separated the API from the agent
API Sentinel has a small number of deliberately separated components.
Spring Boot owns the application-facing API and the persistence model for API endpoints and changes. The Python service owns the agent workflow because the Hindsight and LLM integrations live there.
The Spring Boot service exposes the AI endpoints:
`@PostMapping("/analyze")
public String analyze(@RequestBody Map<String, String> request) {
String change = request.get("change");
return pythonAgentService.analyzeChange(change);
}`
`@PostMapping("/remember")
public String remember(@RequestBody Map<String, String> request) {
String application = request.get("application");
String api = request.get("api");
String field = request.get("field");
return pythonAgentService.rememberDependency(
application,
api,
field
);
}`
I kept this boundary intentionally boring. Spring Boot does not need to know how memory retrieval works. It sends a request to the agent and gets a result back.
That separation also made debugging much easier. I could test the Java API, the Python agent, Hindsight, and the LLM independently before testing the complete path.
First, I made dependencies explicit
The memory system is only useful if it contains useful facts.
API Sentinel has a remember operation that records a known application-to-field dependency in Hindsight:
client.retain(
bank_id=BANK_ID,
content=(
f"API: {api_name}. "
f"Consumer: {consumer}. "
f"Dependency: {consumer} depends on the {field} field."
),
context="api-consumer-dependency"
)
The stored memory is deliberately concrete.
It records:
• the API
• the consumer
• the field
• the relationship between them
I did not try to make the first version magically discover every dependency in every codebase. That would have made the interesting part of the system harder to reason about.
Instead, the current design treats dependency registration as an explicit event. In the larger E-Learning integration, the application registers the known dependency before API compatibility analysis happens.
That gives the memory layer a clear contract: when the agent says a consumer depends on a field, that statement came from a recorded dependency rather than an invented inference.
For a production implementation, the same interface could later be fed by OpenAPI analysis, runtime telemetry, API gateway logs, static analysis, or CI pipelines.
Hindsight became the historical layer
The analysis path is where the design becomes interesting.
When a developer proposes removing a field, the Python agent first extracts the field name:
`match = re.search(
r"remove\s+([A-Za-z0-9_]+)",
change,
re.IGNORECASE
)
if not match:
return (
"UNKNOWN",
"Could not identify the API field being removed."
)
field = match.group(1)`
Then it asks Hindsight for consumer dependencies:
`result = hindsight_client.recall(
bank_id=BANK_ID,
query=(
f"Which API consumers depend on the {field} field? "
f"Return only direct consumer dependencies."
)
)`
This is the point where API Sentinel stops being a conventional stateless LLM wrapper.
The proposed change is new information.
The consumer dependency is historical information.
Hindsight connects the two.
I then filter the recalled memories so that only dependency-shaped memories relevant to the field are passed forward:
`if (
field_lower in text
and (
"depends on" in text
or "dependency on" in text
or "relies on" in text
or "rely on" in text
)
):`
matching_memories.append(memory.text)
This filtering step matters because I do not want every historical memory in the bank to become context for every API change.
I made the LLM explain evidence, not invent it
Once the relevant memories are identified, I pass them to the LLM together with the proposed change.
The prompt explicitly tells the model not to invent consumers:
Relevant historical memories from Hindsight:
{memory_text}
Provide a concise developer-focused explanation.
Include:
- Compatibility assessment
- Affected consumers
- Why the change may cause problems
- A recommended migration action
Do not invent consumers or dependencies that are not present in the Hindsight memories.
That division of responsibility is important.
Hindsight provides historical context.
The compatibility logic establishes the status.
The LLM turns those facts into an explanation a developer can act on.
For example, if Hindsight contains a dependency for the description field, the system can report:
Compatibility Status:
POTENTIALLY_BREAKING
Affected Consumer:
E-Learning App
The LLM can then explain that removing description may affect the E-Learning application's use of course details and suggest a migration path.
If Hindsight has no matching dependency, the system instead records:
NO_KNOWN_IMPACT
That wording is deliberate. It does not mean “this change is guaranteed to be safe.” It means that the memory layer has no known consumer dependency matching the query.
The agent remembers the decision too
There is another useful part of the loop: the agent stores the compatibility analysis back into Hindsight.
`hindsight_client.retain(
bank_id=BANK_ID,
content=(
f"API compatibility analysis: "
f"Proposed change: {change}. "
f"Field: {field}. "
f"Result: {status}. "
f"AI analysis: {ai_reason}"
),
context="api-compatibility-analysis"
)`
That gives the system two kinds of memory.
The first is dependency memory:
E-Learning App depends on description
The second is decision history:
A previous analysis considered removing description
and classified the change as potentially breaking.
This is where persistent agent memory becomes more interesting than simply attaching a vector database to a chatbot. The goal is not to remember everything. The goal is to preserve information that can change the quality of a later engineering decision.
For more background on how persistent memory can be used in agent systems,
For more background on how persistent memory can be used in agent systems, I found the Hindsight GitHub repository, Hindsight documentation, and Vectorize's explanation of agent memory useful references.
A failure in the Hindsight client changed the design
One of the more practical lessons came from the Hindsight client lifecycle.
My first implementation created a long-lived Hindsight client and reused it across Flask requests. When the Spring Boot application called the Python service, I hit an asynchronous timeout-context error.
The problem was not the compatibility logic. It was the lifetime of the client relative to the request context.
I changed the implementation to create the Hindsight client when an operation begins and close it when that operation finishes:
`def create_hindsight_client():
return Hindsight(
base_url=os.getenv("HINDSIGHT_BASE_URL"),
api_key=os.getenv("HINDSIGHT_API_KEY")
)
and:
hindsight_client = create_hindsight_client()
try:
# recall, analyze, retain
...
finally:
hindsight_client.close()`
That small change made the Spring Boot → Flask → Hindsight path reliable in the end-to-end flow.
It was a useful reminder that agent architecture is still software architecture. Memory does not remove the need to think carefully about clients, lifecycles, network boundaries, and failure modes.
The result: the same API change gets different context
The complete flow is now straightforward.
First, the consumer dependency is remembered:
POST /api/ai/remember
{
"application": "E-Learning App",
"api": "Course API",
"field": "description"
}
The Python agent retains that relationship in Hindsight.
Later, a developer enters:
Remove description from Course API
The agent extracts description, recalls historical dependencies, finds the E-Learning application, and sends the relevant context to the LLM.
The result is:
POTENTIALLY_BREAKING
with the known affected consumer and an explanation of the migration considerations.
The important thing is that the dependency did not have to be repeated in the second request.
That is the whole point of the memory layer.
What I learned
Memory should change a decision
It is easy to add memory to an agent and call the system “memory-enabled.” I found that less interesting than designing a case where the answer actually changes because of remembered information.
For API compatibility, the useful question is not just “what changed?” It is “what historical dependency makes this change risky?”
Store facts, not vague summaries
A memory such as “the API has consumers” is almost useless.
A memory such as:
E-Learning App depends on the description field.
is actionable.
The more directly a memory can be connected to a future query, the more useful it becomes.
Let the system provide facts; let the model explain them
I wanted the model to explain the impact, not manufacture the dependency graph.
Hindsight supplies the historical evidence. The model interprets that evidence.
That makes the generated explanation easier to constrain and inspect.
Agent systems still need ordinary engineering discipline
The timeout problem I encountered had nothing to do with prompting or model quality. It came from how an external client was reused across web requests.
Agent applications still need ordinary engineering discipline around HTTP clients, resources, configuration, and boundaries.
“No known impact” is not the same as “safe”
This distinction is important for any compatibility system.
If memory contains no matching dependency, the system can report that it has no known impact from the information currently stored. It should not turn an incomplete knowledge base into false certainty.
Where I would take it next
The current architecture gives me a clean place to add more sources of dependency evidence.
The next step would be automatic dependency discovery.
An OpenAPI specification could provide contract-level information. Runtime API logs could show which fields consumers actually use. Static analysis could inspect client code. A CI integration could run the compatibility agent whenever an API contract changes.
The memory interface would stay similar:
New evidence
|
v
Hindsight
|
v
Historical dependency context
|
v
Proposed API change
|
v
Compatibility analysis
That is the part of API Sentinel I find most useful: the system does not need to rediscover the entire history every time an engineer changes an API.
It can remember.
And once an API agent can remember who depended on a field yesterday, an API change stops being just a diff. It becomes a question about the history of the systems connected to it.
Top comments (1)
the honest part of this design is "suppose I know that Billing depends on phone." the memory only holds what somebody recorded, and in most teams the way a dependency gets recorded is the incident where it broke. the seed data is the postmortem archive. feed the agent the last two years of "who paged whom" and it starts with the dependencies that actually matter, because those are the ones that already failed.