Hindsight Taught My Lending Agent a Rule I Never Wrote

Within a handful of defaults, my underwriter quietly stopped trusting one fleet of agents. I never told it that fleet was risky. I had buried that rule inside the simulation myself. A formula-only underwriter fed the same loans walked into it again and again, all the way to round 40.

The difference between the two was simple: one of them could remember what happened after it said yes.

The problem: agents earn money but have no credit file

Autonomous agents now run tasks, spend money and get paid. Anyone lending to them, whether a platform, an orchestrator or another agent, has to answer three questions:

1. Is this agent who it claims to be?
2. How much should I lend it?
3. Is the pattern across its whole fleet a warning sign?

A scoring formula can handle the first two. It can't handle the third, because a formula doesn't remember. It sees one agent's own record. It doesn't notice that three agents from the same operator all defaulted on the same kind of work last week.

What We built

VouchAI is a small lending middleware with four stages:

1. An identity gate that blocks lookalike agents before anything else runs
2. A formula score that sets a baseline credit limit
3. A memory-backed underwriter that adjusts that limit using what happened before
4. A feedback loop: when a loan is repaid or defaulted, the result goes back into memory

The memory layer is Hindsight, an open-source agent memory system. The underwriter model runs on Groq, the backend is FastAPI and the frontend is Next.js.

Step one: prove the memory works before building on it

Before writing any lending logic, I ran a tiny smoke test against Hindsight: store one default, recall it, then ask for advice.

client.retain(
    bank_id=BANK,
    content="Agent scraper-07 under principal NovaOps defaulted on a 400 dollar data-scraping loan.",
    context="loan outcome",
    retain_async=False,
)

res = client.recall(bank_id=BANK, query="What do we know about NovaOps agents?")

ans = client.reflect(
    bank_id=BANK,
    query="Should we lend 400 dollars to another NovaOps agent for data scraping?",
)

The calls took 3.5 s (retain), 1.4 s (recall) and 6.2 s (reflect). That output was the moment I knew the idea would work. reflect didn't just return the stored fact. It reasoned from it: a NovaOps agent had defaulted on an identical loan, so the risk was high.

I write outcomes with retain_async=False. The next recall has to see the previous outcome, or the learning curve lags behind reality.

The experiment: two underwriters, identical loans

I wrote a seeded simulator with 12 agents across four fleets and four task categories. Hidden inside it is one rule:

# backend/app/services/simulator.py
PATTERN_PRINCIPAL = "NovaOps"
PATTERN_CATEGORY = "data-scraping"
PATTERN_PROBABILITY = 0.75   # NovaOps on data-scraping
OTHER_PROBABILITY = 0.06     # everything else

I ran it with seed 7 for 40 rounds. Both underwriters see the exact same 40 requests and the exact same random outcomes, because every random draw is made up front from one seed. One underwriter uses only the formula. The other uses Hindsight memory.

	Formula only	With Hindsight memory
Total loss	$2,300	$700
Net P&L	-$1,872	-$344
Approved / denied	40 / 0	28 / 12

The formula-only underwriter approved everything, because every agent looked healthy on its own record. The memory underwriter lost money on the first few NovaOps loans. Then it started cutting limits for that fleet in that category, and its cumulative loss flattened at $700.

The replay site's Agents page shows where the damage sat. With memory off, three NovaOps agents, all on data-scraping, account for every dollar lost: summarizer-12 lost $850, reviewer-03 lost $850 and scraper-07 lost $600. Every other agent lost nothing.

What a memory-backed decision looks like

Same agent (scraper-07), same $400 request, two underwriters. Hindsight recalled 9 facts in about 1.5 seconds, and the underwriter cited them:

Mode	Multiplier	Final limit	Rationale
Memory off	1.00	$600	Formula only
Memory on	0.10	$60	scraper-07, managed by NovaOps, defaulted on a $400 data-scraping loan, matching the requested loan, indicating high risk

Notice the recalled facts. They include defaults from sibling agents (scraper-11, scraper-04, scraper-02), all under NovaOps. That is the fleet-level signal the formula never sees.

The live site does the same thing. A $200 request for a NovaOps data-scraping agent has a formula limit of $100, drops to a $10 final limit and is denied, with 36 recalled facts behind the decision.

An agent from a clean fleet gets the opposite treatment. scraper-02 (Quanta Works) has a clean repayment history in memory, so its limit goes up from $100 to $120 (a 1.20x multiplier). My $130 request still exceeded the new limit, so it was denied, but the direction is the point: memory can raise a limit slightly.

Keeping the model on a leash

I didn't want an LLM deciding how much money to lend. Memory only gets to scale the formula's limit, and the scaling is clamped in code, not in the prompt:

# backend/app/services/underwriter.py
MIN_MULTIPLIER = 0.0
MAX_MULTIPLIER = 1.2

def clamp_multiplier(value: float) -> float:
    if not math.isfinite(value):
        raise ValueError("multiplier is not finite")
    return max(MIN_MULTIPLIER, min(MAX_MULTIPLIER, value))

# ...
final_limit = round(formula.formula_limit * multiplier, 2)
decision = "approved" if amount <= final_limit and final_limit > 0 else "denied"

The model returns plain JSON with a multiplier and a two-sentence rationale. There is no tool calling. If Hindsight or Groq fails, the system falls back to the formula for that request instead of failing the loan. The identity gate runs before any of this, so a blocked agent never reaches memory.

The Decision log on the Memory page makes the clamp visible. Clean agents sit at 1.0x to 1.2x, while NovaOps data-scraping loans drop to 0.2x and 0.1x.

What reflect wrote down

Every ten rounds the simulator asks Hindsight to reflect on what it has learned. By round 20 it had written a summary on its own, with no hand-written rule behind it. It named the fleet (NovaOps), the category (data-scraping) and the specific defaults by reviewer-03 and summarizer-12, and noted that agents from other fleets had repaid consistently.

A formula can't write that, because a formula never looks at two agents at once.

Lessons

1. Make memory part of the decision, not a log. Recall before every decision and retain after every outcome.
2. Write memory synchronously when the next step depends on it. I traded speed for correctness.
3. Bound what the model can change. A clamped multiplier is easy to reason about and to audit.
4. Keep deterministic rules in front of memory. Identity checks should never depend on what a model remembers.
5. Plan for slow memory-backed calls. Mine averaged about 7.1 seconds per memory-on round, so the public site replays a saved run and the live pages call a separate backend.

Limits

The data is simulated. No real money moves, and the default rule is one I planted. Memory isn't perfect either. After learning the pattern, my underwriter still took one $300 loss on a NovaOps agent, and it denied 12 loans, some of which would have been repaid. Both underwriters lose money under this fee model, so the gain here is loss avoided, not profit.

Try it

The code is on GitHub and the replay runs at vouch-ai-hazel.vercel.app. If you want to give your own agents a memory, Vectorize's overview of agent memory is a good place to start.

cc: Code.in