Hindsight Remembers the Exception. My Code Decides If It Applies.

Nikhilesh Kamalapurkar — Tue, 29 Sep 2026 08:17:16 +0000

A reviewer writes: "Use the shared transport wrapper." The author answers: "We approved an exception for this file." Nobody in the thread can say who approved it, why, for which paths, or when it stops being true.

That missing context is what I built ReviewMind to keep. It is a code reviewer that remembers why an engineering exception was approved, where it applies, and when it stops applying. Its one question is: we approved this before, but does that approval apply here and now?

A note on status before the details. Everything runs on a synthetic TypeScript repository, and I only report what is recorded in the repo's evaluation file. Where something has not been run yet, I say so.

How the system hangs together

ReviewMind has three parts that each own one job:

Supabase is the authority for users, repository ownership, immutable source events, approval versions, revocation, and worker jobs.
Hindsight is the semantic memory layer: retain, recall, and reflect.
A generation model (any OpenAI-compatible endpoint with JSON Schema output) explains the current change and the evidence.

The piece that surprised people when I described it is what is missing from that list: nothing in it lets the model decide whether an exception applies. Deterministic application code does that.

The planned loop, which the repo implements end to end at the API and worker level, looks like this:

A pasted diff comes in and Hindsight recalls relevant history.
Recalled facts are mapped back to canonical events. Anything I can't verify is dropped.
A scoped review runs and a human gives reasoned feedback on a finding.
Feedback is retained. Hindsight reflect drafts a decision from it.
A maintainer approves the draft. The approved version is retained.
After ingestion and a recall check, a fresh session can use it.

The core idea: memory recalls, code decides

The tempting design is to recall a remembered exception and ask the model whether it still holds. I didn't trust that. "Is this date before that date" and "does this glob match this path" should never be a language model's opinion.

So a decision is a plain record with explicit conditions: rule key, path glob, dependency name, version range, effective time, expiry, rationale, and source event IDs. An exception is usable only when its canonical decision is approved, ready, in the same repository and run, effective at the recorded review time, unexpired, path-matched, version-matched, prerequisite-complete, and not revoked or superseded.

A simplified sketch of the check (the real module is larger, this shows the shape):

function classify(d: Decision, ctx: ReviewContext): Status {
  if (d.status !== "approved")               return "needs_context";
  if (ctx.reviewTime < d.effectiveFrom)      return "expired";
  if (d.expiresAt && ctx.reviewTime >= d.expiresAt) return "expired";
  if (!globMatch(ctx.filePath, d.pathGlob))  return "out_of_scope";
  if (!ctx.depVersion)                       return "needs_context";
  if (!satisfies(ctx.depVersion, d.versionRange)) return "out_of_scope";
  return "applicable";
}

The expiry boundary is exclusive: a decision applies while effective_from <= review_time < expires_at. That is unit-test territory, and it stays there. The model's structured output is validated against a schema, every historical claim must carry a source ID that resolves to a real record, and any waiver the model proposes is re-checked by the decision engine.

Why Hindsight agent memory

I could have stored exceptions in a table and searched it with keywords. I chose Hindsight agent memory because the workflow is exactly its three operations:

Retain stores feedback and approved decision versions as immutable event:<uuid> documents. Immutable IDs mean a transport retry can't quietly create a second logical event or overwrite history.
Recall finds semantically relevant history, including decisions worded differently from the current diff. Recalled facts must resolve back to authorized events in the current repository.
Reflect turns selected feedback into a structured draft decision. Reflection never approves policy. A maintainer does.

There is one more rule I like: a successful retain request is not proof the memory is usable. The app tracks queued, running, retained, and ready, and "ready" requires both finished ingestion and a targeted recall that returns the expected document provenance. That follows the core idea behind agent memory: memory should carry experience across sessions, while the application keeps authority over what that experience may do.

One exception, three boundaries

The example the whole project is built around is a temporary transport-wrapper exception on a legacy adapter, scoped to one path, one dependency version range, and an expiry of 30 September 2026.

On the legacy path, inside the window, the wrapper requirement is waived.
On a new adapter with the same dependency version, the path doesn't match, so the exception is out of scope and the requirement stays.
After expiry, the decision stays visible as history but can no longer waive anything.
If a change removes the explicit 2-second request timeout while the exception is active, the timeout finding stays. Waiving the wrapper does not waive reliability.

That last rule is the one I care about most. A remembered "yes" must never become blanket permission. The timeout check is a small AST check, and it is intentionally fixture-scoped: it only recognizes named request imports from the supplied client and helper when called with a literal options object. Aliases, spreads, computed values, and anything unfamiliar return needs_context, not a pass.

What I actually ran

Here is what the evaluation file records, as of 28 September 2026:

62 automated deterministic and local PostgreSQL tests passed, with 0 failures.
Production typecheck and build passed, and a dependency audit reported 0 known vulnerabilities.
A real-provider integration command authenticates through Supabase, retains through Hindsight, waits for the operation, recalls from a fresh client, resolves the document back to its event, and requests a validated reflection and a real structured review. It passed.
A live PR-A review through the local API and worker ran with real Hindsight recall and generation. Before any approval existed, recall correctly reported no approved memory, and the wrapper convention stayed a normal finding with no invented citations.

What has not been run: the full browser journey (feedback, reflect, approve, fresh-session review), and the frozen 36-generation comparison of Generic, Static, and Memory modes across twelve held-out cases. I'm not reporting any improvement percentage because there isn't one to report yet.

The bug that taught me the most

The first live integration run passed authentication, persistence, retention, fresh-client recall, and document resolution. Then reflection returned an HTTP 500.

A simple structured reflection worked, so the failure was in my schema. The configured endpoint rejected a nullable type-array schema. When I tried an anyOf variant, it returned the string "null" instead of a real null and omitted the source IDs entirely.

The fix was boring and worth stating plainly. The adapter now uses string wire fields for optional values, converts only empty or string-null values in designated nullable fields, and only then runs strict canonical validation. Reflection also explicitly asks for source UUIDs in its written answer so they survive structured extraction. Provider source facts are still checked independently, so an empty or invented source array does not pass.

I kept the failed run's evidence file next to the passing one in the repo. The lesson is that a memory system that hands you structured drafts needs a translation layer you own, and that layer must never be allowed to weaken validation.

Lessons

Store the conditions, not the conclusion. "Approved" without scope and expiry is a rumor.
Never let the model do date or path math. Use tested glob and semver libraries and keep the boundary exclusive.
Separate team exceptions from baseline requirements. The 2-second timeout sits above any wrapper waiver.
Treat retain as asynchronous. Design for pending states and prove readiness with a recall probe.
Keep evidence inspectable. For every recalled fact, show what was decided, by whom, why, where it applies, until when, and the source.

Recall can also miss. Absence from recall is not proof that no decision exists, and with no findings the reviewer says "No findings within supplied context," never "safe to merge."

The code, fixtures, migrations, and evaluation record are on GitHub: ReviewMind.

Tagging https://code.in/

DEV Community: Nikhilesh Kamalapurkar