What 404s Taught Us About Building on a Memory API

Tanjiro-90 — Tue, 29 Sep 2026 04:52:44 +0000

My first call to Hindsight failed with a 404, and so did my second. I'd guessed a REST path that looked reasonable and wasn't. That set the tone for most of this build: less "design a clever architecture," more "read the docs carefully and handle what a real API actually does."
I'm Antony Sebastian, and I built the backend for Promise-Keeper: a Streamlit app that tracks what you've promised people, briefs you before meetings, and closes the loop when a promise is kept. This is the story of wiring it to Hindsight for memory, and the specific things that broke along the way.
The shape of the system
Promise-Keeper is a Streamlit app that talks to Hindsight's REST API directly, no SDK in between. The flow for one meeting looks like this:

Paste in meeting text.
Gemini extracts the promises made in it.
Each promise is saved to Hindsight agent memory with a retain call.
Before the next meeting, a recall call pulls the relevant memories back, and Gemini turns them into a brief.

Each contact gets their own memory bank, named something like contact_priya_sharma, created automatically the first time something is saved for them. Nothing about the bank has to be provisioned in advance.
What actually gets written is plain text, one sentence per promise:

On 2026-09-29, Tony promised Priya Sharma to email the pricing
deck (due: 2026-10-02). Status: OPEN.

When a promise is later fulfilled, that isn't an edit to the old memory; it's a new memory:

FULFILLED: Tony sent the pricing deck to Priya Sharma. Evidence:
"Thanks for the deck, it's great" (2026-10-02).

Recall handles the merging. A query like "What promises are open or overdue?" run before saving, or "open promises, overdue items, recent interactions" for the brief, returns both the original and the fulfillment note, and Gemini reconciles them into the current status.

Problem one: Guessed REST paths

The first version of the retain function pointed at /v1/banks/.... It looked like a sensible REST path, and it was wrong. The real route is /v1/default/banks/{bank_id}/memories, and the request body has a specific shape too: {"items": [{"content": ...}]}.
There's not much of a lesson here beyond the obvious one: read the actual API documentation before writing the client code, rather than pattern-matching from other REST APIs you've used before. A wrong path fails immediately and loudly, which is at least easy to debug. The next few problems were quieter.

Problem two: a 404 that isn't an error

The very first recall for a brand-new contact returns a 404 because their bank doesn't exist yet; nobody has saved anything for them. My first instinct was to treat that as a failure. It isn't. It's the expected response for "this person has no memories yet."
The fix was to catch that specific case and return an empty list instead of raising:

def recall(bank, query):
    r = requests.post(f"{BASE}/banks/{bank}/memories/recall",
                      json={"query": query}, headers=HEADERS, timeout=60)
    if r.status_code == 404:
        return []  # bank doesn't exist yet
    return [m.get("text", "") for m in r.json().get("results", [])]

This matters more than it looks. Without it, the very first meeting with any new contact would crash the app. With it, a new contact just starts with an empty brief, which is exactly correct.

Problem three: retains are slow, and recall can miss a fresh save

Hindsight doesn't just store the text you send it; it processes it with an LLM on the way in. That means a retain call can take several seconds, and a recall run immediately after a save can occasionally miss something that hasn't finished indexing yet.
Two adjustments came out of this. First, the request timeout on both calls needs to be generous, not the default a typical HTTP client ships with. Second, I stopped assuming retain-then-recall is instant; the UI shows the save completing before offering to generate a brief, rather than chaining the two calls back to back with no gap.

Problem four: the LLM provider was the flakiest part of the system

Hindsight was, honestly, the reliable half of this stack. The LLM calls were where things kept breaking:

Groq's API returned 403s, blocked at the network level.
Gemini retired the model I'd started with for new users partway through.
A 503 "high demand" spike took the API down for a stretch.
The free tier caps out at 20 requests a day, which is nothing once you're testing. None of this is about Hindsight, but it directly affected how the backend had to be written, since retain and recall both depend on an LLM call succeeding first (to extract promises, or to write the brief). The fixes:

def call_llm(**kwargs):
    for attempt in range(6):
        try:
            return client.chat.completions.create(**kwargs)
        except Exception as e:
            code = getattr(e, "status_code", None)
            if code in (500, 502, 503, 504) and attempt < 5:
                time.sleep(3 * (attempt + 1))   # busy server: wait and retry
                continue
            raise

Retrying on 5xx errors with backoff absorbed the transient outages. Merging extraction and fulfillment-checking into a single model call, instead of two, cut the request count roughly in half, which mattered a lot against a 20-request daily cap. And the provider and model are read from a .env file rather than hardcoded, so switching providers didn't mean editing code.

Problem five: the setup itself

Some of the most time lost wasn't API-related at all. A Windows machine needs its PowerShell execution policy adjusted before a virtual environment will activate. A .env file saved from a text editor can silently become .env.txt, and the app fails with confusing "missing API key" errors that have nothing to do with the key being wrong. More than once, the version of app.py actually running wasn't the one I had just edited. None of this is glamorous, but it ate real hours, and it's worth mentioning because it's the kind of friction every teammate hits and nobody writes up.
Lessons learned