I Let an Agent Fix Crashes Only If Hindsight Remembered Them

Sravanth Bikkasani — Sat, 07 Nov 2026 12:28:53 +0000

At 2:14 a.m. my settlement service crashed because nobody had bought anything for an hour. The nightly job averaged ticket size over zero transactions and died on a division by zero. I had fixed the exact same bug in a different service months earlier. Nobody, including the AI tooling I was using at the time, remembered that.

That is the problem I built around: agents are good at reasoning about code and terrible at remembering what already happened to you. So I gave one a memory, and then made that memory the only thing allowed to authorize a change to my code.

What the system does

It is one Python project with two halves that share a single memory bank.

The first half is a Streamlit dashboard with two audit modes. In Business Pitch mode I paste a startup pitch and get Team, Market and Product scores, red flags, and a verdict. In Technical Code mode I paste code and get a security score, a list of bugs, and a corrected version. In both modes the audit starts with a recall against past investments or past bugs, and only then goes to Groq (openai/gpt-oss-120b) in JSON mode.

The second half is autoheal.py, a supervisor that runs a service, watches it crash, and decides whether it is allowed to fix it. The dashboard can start and stop it as a background process, and shows an incident feed of what it did overnight.

Both halves use Hindsight agent memory as the only store. If you want the background on why that is different from stuffing history into a prompt, the Vectorize page on what agent memory is is a good primer.

The rule: memory is the permission slip

Most "self-healing" demos let the model patch whatever broke. I did not trust that, so I set two rules.

Read gate. AutoHeal only attempts a fix if recall returns a similar bug I resolved before. No precedent means no edit. It logs the crash and stops.
Write gate. A fix only goes back into memory after the patched service has stayed up for a configured window. If it crashes again, the file is restored from backup and nothing is learned.

The second rule matters more than it looks. Without it, the agent teaches itself its own mistakes.

try:
    code = proc.wait(timeout=args.stable_seconds)
except subprocess.TimeoutExpired:
    log("Fix held. Saving it to memory.")
    save_verified_fix(pending, args.stable_seconds)
    pending = None
    code = proc.wait()
else:
    rollback(pending, target)   # crashed again right after the fix
    return 2

A timeout here is the good outcome. The service is still alive after the window, so the fix gets retained with the root cause, the diff, and the past bug it was modelled on.

What came back was not what I retained

I seeded the bank with 14 records: seven past investments and seven resolved bugs. The first recall returned 99 results. Hindsight breaks each retained record into smaller facts, and many of them overlap. Piped straight into a prompt, that blew past Groq's request limits and mostly repeated itself.

The fix was dull: deduplicate, truncate, cap.

def prepare_memories(texts, limit=12, chars=700):
    seen, out = set(), []
    for t in texts:
        t = " ".join(str(t).split())
        if not t or t.lower() in seen:
            continue
        seen.add(t.lower())
        out.append(t[:chars])
        if len(out) >= limit:
            break
    return out

The other surprise was threading. The Hindsight client runs async code internally. Cached in Streamlit, it failed with "Event loop is closed" on the second rerun. Created fresh in the script thread, it failed with "This event loop is already running". What worked was giving every Hindsight call its own short-lived thread with its own loop:

def run_hindsight(fn):
    def worker():
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
        try:
            return fn(get_hindsight_client())
        finally:
            loop.close()
    with ThreadPoolExecutor(max_workers=1) as pool:
        return pool.submit(worker).result()

Stopping the agent from inventing history

A memory-grounded agent has a new failure mode: it fabricates the memories. Ask for "similar past investments" with an empty bank and a language model will happily supply a few.

So the audit code does not trust the model on this point. With no recalled records, it overwrites the answer:

if not memories:
    audit["similar_idea_found"] = False
    audit["historical_loss"] = "None found"
    audit["similar_past_cases"] = []

There was a subtler version of the same problem. Every audit saves its findings back to memory, so a few days in, a past audit of a pitch could be recalled and described as if I had invested money in it. Now every saved audit is labelled "an evaluated pitch, NOT an actual investment", and the system prompt says never to present those records as profit or loss.

What it does now

Business pitch. I paste a pitch for a peer-to-peer payments app: fast user growth, no revenue model, a solo founder. Before memory, the audit gave me what any generic prompt gives: "no clear monetization", "key-person risk". After, the same flags arrive attached to my own history. It finds a 2022 payments startup in my records that grew quickly, never monetized, and shut down with a five-lakh write-off, and it reports the outcome, the amount, and the lesson I wrote at the time. It also tells me when a pitch resembles nothing I have seen, and judges it on fundamentals.

Code. A login query built by string concatenation used to get "use parameterized queries". Now it matches my earlier SQL injection record and returns the fix I already trusted, along with the historical loss attached to that class of bug.

A crash at night. A divide-by-zero in the settlement job matches my earlier "guard empty inputs" fix. AutoHeal backs up the file, applies the guard, restarts, waits out the stability window, and stores the result. The next time a similar crash hits a different file, there are now two precedents.

A crash it should not touch. A traceback with no precedent in memory gets no edit. The incident feed shows "no similar past bug, needs a human", and the process stops instead of looping.

What I would tell you before you build this

Retention is not retrieval. What you store and what you get back are different shapes. Budget time for deduplicating and capping recall results.
Gate autonomy on memory, then gate memory on verification. Reading precedent decides whether the agent acts. Verified outcomes decide whether it learns.
Never let the model be the source of "past". Anything labelled history should come from recall, and the code should enforce that when recall is empty.
Label what you write back. An audit, an observation and a fact look identical in a vector store unless you mark them.
Pin nothing to one model. The model I started with was retired by Groq partway through. A fallback list plus JSON-mode detection means a retired model is a log line instead of an outage.

The limits are real. The fixes are written by a model, so for anything that moves money I run the project's tests before the restart. AutoHeal edits one file at a time. And it stops after an unfixable crash rather than hiding it with a restart loop. I prefer that, but it is a choice.

If you want to try the memory layer yourself, start with the Hindsight repository and the Hindsight documentation. The part I would copy first is not the dashboard. It is the two gates.

I'm Sravanth Bikkasani. The full code is on GitHub: https://github.com/sravanthbikkasani-ops/Streamlit.git. If you build something on agent memory, I'd like to hear what broke for you first.

Built with Hindsight, Groq and Streamlit. Tagging @Code.in.