A code review agent that remembers & evolves with you?! Dive in to know more.

Mithin Sai M — Mon, 28 Sep 2026 02:50:23 +0000

I've built a code review agent that remembers & evolves with you, here's what I learned.

The third time a teammate fixed the same unbounded retry loop in a pull request, I stopped blaming the developer and started blaming the reviewer. Not the human one. The tooling. Every review we ran, automated or otherwise, started from zero, so the same lesson had to be re-taught on every commit.

WinterFallX is what I built to fix that. It's a code review agent with a long-term memory, backed by Hindsight, and this post is about the design decision that mattered most: what belongs in memory, and what doesn't.

What the system does

WinterFallX is a Streamlit app with two views.

The developer view is where a review happens. You enter a team name and developer ID, pick or paste a failing piece of code, and run the pipeline. The pipeline does three things in order:

Recall. Query the team's memory bank for context on this file and this error.
Decide. Check whether this exact incident has been seen before by this team.
Retain or promote. If it's new, store it as a fresh memory. If it's a repeat, skip the write, bump the repeat counter, and hand back the resolution that was already verified.

The executive view reads from a local SQLite ledger and shows, per team, which files keep failing, which standards keep getting violated, and how many review cycles were repeats. It's the answer to "are we teaching the same thing over and over?" with an actual count.

Two stores, then: Hindsight and SQLite. Most of what I learned is about why I kept both and how I divided the labor between them.

The through-line: memory is not a table

My first instinct was to put everything in one SQLite table. Incidents have a team, a developer, a file, an error, a resolution. That's a row. A dashboard wants rows. Done.

The problem is that a table remembers exactly. If yesterday's incident was "JWT validation is bypassed when the Authorization header is malformed" and today's is phrased slightly differently, a table has nothing to say. Real review knowledge is fuzzy. People describe the same class of bug in different words, in different files, with different surrounding code. What I wanted was retrieval by meaning.

That's the job I gave to Hindsight. It's a long-term memory engine with retain and recall operations, and it searches by meaning rather than by exact key. If you're new to the concept, Vectorize has a good overview of what agent memory is and why stateless agents fall over. The Hindsight documentation covers the API surface I use here.

So the split became:

Hindsight is the memory of record. It stores what happened, in language, and answers "have we seen something like this?"
SQLite is the analytics ledger. It stores counts, timestamps, and per-team aggregates, and answers "how often, where, and who?"

Neither one is a cache of the other. They answer different questions.

Scoping memory to a team

The first Hindsight decision was the granularity of a memory bank. I considered one global bank, one per developer, and one per team. Global mixes conventions from teams that disagree with each other. Per-developer fragments the exact knowledge a team is supposed to share.

Per team is where the value is, because a coding standard is a team-level agreement:

def run_review(team_name, developer_id, selected, code_snippet):
    scenario = SCENARIOS[selected]
    bank_id = f"winterfallx-{team_name.strip()}-memory"
    ledger = HindsightLedger(bank_id)
    existing, memory_texts = ledger.recall(
        team_name, scenario["file"], code_snippet, scenario["error"]
    )

One line derives the bank ID from the team name, and that single line is what makes a lesson learned by one developer available to everyone else on the team. When a second developer on the same team hits the same problem, they're querying the same bank as the first.

What I ask Hindsight, and what I tell it

Recall takes a query, and the query is where I spent the most thought. I settled on the file name plus the error description:

query = f"{file_name}: {error_msg}"
result = self.client.recall(bank_id=self.bank_id, query=query)
memory_texts = [memory.text for memory in result.results]

I deliberately left the raw code snippet out of the query. Code is noisy: variable names, whitespace, and incidental details all pull retrieval toward surface similarity. The file and the failure description carry the signal I care about.

Retaining is the opposite problem: what do you write down? I don't store a row. I store a sentence a human could read back:

content = (
    f"Team {payload['team_name']} (developer {payload['developer_id']}) hit an issue in "
    f"{payload['file_name']}: {payload['error_msg']}. "
    f"Standard violated: {payload['standard_violation']}. "
    f"Resolution applied: {payload['resolution']}."
)
self.client.retain(bank_id=self.bank_id, content=content)

This turned out to matter more than I expected. A memory written as a complete statement, with the cause, the violated standard, and the fix in one place, is useful when it comes back. A memory that's a bag of fields needs the consumer to know how to reassemble it. Writing for the reader you'll have later, who has none of today's context, is the whole discipline.

Why the ledger still exists

If Hindsight does semantic recall, why keep SQLite and a hash?

Because "have we seen this exact incident before?" and "have we seen something like this?" are different questions with different tolerances for being wrong. When I'm deciding whether to increment a repeat counter that management will read on a dashboard, I don't want fuzzy. I want deterministic:

def signature_for(team_name, file_name, code_snippet, error_msg):
    raw = "|".join([team_name.strip().lower(), file_name.strip().lower(),
                    code_snippet.strip(), error_msg.strip()])
    return hashlib.sha256(raw.encode("utf-8")).hexdigest()

That signature gives me an exact-match path. Hindsight gives me the meaning-based path. When the signature matches, the incident is a confirmed repeat: the app skips retain(), promotes the stored resolution, and increments feedback_count. When it doesn't, the incident is stored as new.

An engineer reading this might ask why I don't just trust the semantic match for both. My answer is that a counter on an executive dashboard is a claim about the world, and I'd rather it be conservative and explainable than clever. "This exact code, in this exact file, with this exact error, has now been seen three times" is a sentence I can defend. Semantic recall feeds context; the hash feeds the number.

The trade-off is real and I'll say it plainly: two stores means two things that can disagree. Keeping them coherent was the most annoying part of the build.

What it looks like in use

Here's a concrete run. A developer on AlphaBuilders selects the auth_middleware.py scenario, where the code strips "Bearer " from the header and decodes whatever is left, so a malformed header slips through instead of failing closed.

First run. The pipeline recalls against winterfallx-AlphaBuilders-memory, and the trace shows what came back. The signature check finds nothing for this team, so the review reports no prior match, retains the incident to Hindsight, and inserts a ledger row with a feedback count of one. The remediation it returns is the fail-closed version:

def validate_request(request):
    auth = request.headers.get("Authorization", "")
    scheme, _, token = auth.partition(" ")
    if scheme.lower() != "bearer" or not token:
        raise Unauthorized("Bearer token required")
    return jwt.decode(token, SECRET_KEY, algorithms=["HS256"])

Second run, same code, any developer on the team. The signature matches. The trace reports the context match and the skipped retain(). The repeat count goes to two, and the UI shows the memory-hit banner with the count. Nothing new was written, because there was nothing new to learn. The stored resolution was promoted instead of regenerated.

The pipeline trace is a first-class part of the UI on purpose. It prints the exact recall call and how many memory matches Hindsight returned. If I'm asking engineers to trust a system that remembers things, I owe them a way to see what it remembered and why it acted.

Failure handling I'm glad I did early

Memory is a network dependency, and network dependencies fail. Both recall and retain are wrapped so that an API error, a quota problem, or a timeout degrades the experience instead of breaking it:

try:
    self.client.retain(bank_id=self.bank_id, content=content)
except Exception as exc:
    st.warning(f"Hindsight retain failed, incident still logged locally: {exc}")

The incident is still written to the ledger, the user sees a clear warning rather than a stack trace, and the review completes. I'd rather have a reviewer that occasionally says "my memory is unavailable right now" than one that goes dark whenever a dependency has a bad afternoon.

Credentials come from environment secrets (HINDSIGHT_API_URL and HINDSIGHT_API_KEY), the client is built lazily as a singleton, and a missing variable fails immediately with a message that names what's missing.

Lessons learned

1. Separate "what happened" from "how many times." Memory that retrieves by meaning and a ledger that counts by identity are different tools. Forcing one to do the other's job gives you either a fuzzy dashboard or a brittle memory.

2. Scope memory to the unit that owns the convention. A coding standard belongs to a team. One bank per team meant a lesson learned once was available to everyone, without leaking across teams that disagree.

3. Write memories for a reader who has no context. Store complete sentences: cause, violated standard, fix. The consumer of a memory is a future query that knows nothing about today.

4. Keep your retrieval query about the signal, not the noise. Leaving raw code out of the recall query and using the file plus the failure description gave me a query that describes the problem instead of the syntax.

5. Make the memory visible. Showing the recall call and match count in the UI turned "trust the agent" into "check the agent," which is a much easier sell to engineers.

The bigger takeaway is that the interesting part of an agent that remembers is rarely the remembering. It's deciding what deserves to be remembered, who gets to retrieve it, and how you'll explain it when it's wrong. If you're building something similar, the Hindsight repository is the place to start.

special thanks to pragnasri yellanki, Bharath sai Ganipineni, and Sreevallika Balagonda.

DEV Community: Mithin Sai M