I Built an Engineering Agent That Remembers What Happened Before

#fastapi #python #ai #devops

I Built an Engineering Agent That Remembers What Happened Before

Every engineering team eventually runs into the same problem: we learn important lessons from deployments, incidents, and changes, but those lessons are difficult to reuse when the next similar change arrives.

A developer may know that a particular service previously had problems when a configuration was changed. Another engineer may remember the mitigation. Months later, someone else makes a similar change without knowing what happened before.

I wanted to build a system that could turn those past engineering experiences into useful context for future changes.

That became Engineering Change Memory Agent.

The idea is simple: before recommending how an engineering change should be rolled out, the agent recalls relevant experiences from the team's history, reasons about them, and then produces a recommendation.

The project uses Hindsight as the memory layer.

The Problem

Traditional deployment tooling can tell us what is happening now. Monitoring can show latency, errors, resource usage, and alerts. CI/CD systems can validate and deploy code.

But these systems don't necessarily answer a different question:

"What happened the last time we made a similar change?"

That question requires historical context.

For example, imagine an upcoming change to a payment service:

Service: payments-api
Change: increase the database connection pool
Environment: production
Timing: business hours
Risk: high payment traffic

If the team has previously experienced database saturation after a similar change, that experience should influence how the next change is approached.

The challenge is making that experience available to the agent at the right time.

The Approach

I designed the system around a simple flow:

Change → Recall → Reflect → Decide

The user first provides the context of the upcoming engineering change.

The agent then:

Recalls relevant historical experiences from Hindsight.
Reflects on those experiences together with the current change.
Decides what rollout strategy and safeguards make sense.
Returns the recommendation along with the memory evidence that influenced it.

After a change is completed, the user can also record the outcome.

That outcome becomes part of the team's engineering memory and can influence future recommendations.

Architecture

The prototype uses a small FastAPI backend with a frontend interface.

The basic architecture is:

Frontend → FastAPI → Change Agent → Hindsight + LLM

The frontend collects:

service
change type
change summary
environment
timing
risk context

The FastAPI /analyze endpoint passes this information to the change agent.

The agent retrieves relevant historical experiences from Hindsight and uses them as context for reasoning.

The resulting response includes information such as:

recommendation
rollout strategy
uncertainty
safeguards
number of memories used
memory evidence
explanation of why memory changed the recommendation

This makes the memory contribution visible rather than hiding it behind a single generated answer.

Connecting the Frontend to the Agent

The frontend sends the change context to the backend using a normal HTTP request.

For example:

const r = await fetch("/analyze", {
  method: "POST",
  headers: {"Content-Type": "application/json"},
  body: JSON.stringify({
    service: $("service").value.trim(),
    change_type: $("changeType").value.trim(),
    summary: $("summary").value.trim(),
    environment: $("environment").value,
    timing: $("timing").value,
    risk_context: $("riskContext").value.trim()
  })
});