What Changed When I Gave My Audit Agent Hindsight

Before I gave my audit agent memory, every question started from zero. It could tell me a control's status, but not that we'd seen the same gap before, or why we'd decided to leave it open. After I added Hindsight, it could answer both. That difference changed how I think about what an agent in a compliance tool is actually for.

What Veriq does and how it hangs together

Veriq is an AI compliance and audit platform for internal audit teams, built around the idea of audit memory: the system keeps what it learned in past audits and uses it in the next one. The screens are what you'd expect: active audits, risk items, a compliance rate, policy adherence split into in-policy, partial, and non-adhering, and a progress view for each operations audit (Financial Services, Human Resources, Customer Support). Behind that sits an agent that answers questions in plain language about all of it.

The architecture is deliberately boring:

• A web frontend that renders the dashboard and a chat panel.

• An API layer that owns the audit data: audits, controls, risk assessments, data requests, and the issue tracker.

• An agent service that handles questions, with one job: decide what context it needs before it answers.

• Hindsight, an open-source agent memory system, which holds everything the agent should remember across conversations and across audit cycles.

The first three are ordinary web engineering. The fourth is where the interesting decisions were, so that's the through-line of this article.

The problem: audits are about history, and my agent had none

My first version of the agent was stateless. Each question started from zero, with whatever I could fit in the prompt. That works for "what's the status of HR-24-005?" because the answer lives in the database. It fails for the questions auditors ask most:

• "Have we seen this gap before?"

• "Why did we accept this risk last time?"

• "Did anyone already explain this finding?"

Those answers don't live in a table. They live in the reasoning around the data: a note on a finding, a decision in a review meeting, a clarification in a chat thread. I tried the obvious fixes first. Replaying old conversations into the prompt got expensive quickly and buried the one relevant detail in the middle of a long context. A plain vector search over past transcripts retrieved things that sounded similar but weren't relevant, and it had no sense of time, so a superseded finding ranked the same as the current one.

What I wanted was memory with structure. That's what pushed me to Hindsight's documentation, and it changed how I thought about the problem.

What Hindsight gives me

Hindsight exposes three operations, and the distinction between them matters more than it first appears:

• Retain stores information and extracts what's worth remembering from it.

• Recall retrieves the memories relevant to a query.

• Reflect reasons over those memories to produce a considered answer rather than a list of hits.

Memory lives in banks, and I give each department its own. That was a design decision, not a default: Finance context shouldn't bleed into an HR answer, and an auditor with access to one scope shouldn't get another scope's history through a clever question. If you're new to the idea, Vectorize has a good overview of what agent memory is and why it differs from retrieval.

Here's the retain path. Every time an auditor closes a finding, changes a status, or writes a decision note, the API layer sends it to the matching bank:

from hindsight_client import Hindsight

memory = Hindsight(base_url=settings.HINDSIGHT_URL)

def remember_finding_event(event: FindingEvent) -> None:

memory.retain(

bank_id=f"audit-{event.department}",

content=(

f"{event.audit_ref}: {event.title}. "

f"Status changed to {event.status}. "

f"Rationale: {event.rationale}"

),

context=f"{event.kind} on {event.control_id}",

timestamp=event.occurred_at,

)

Two details in there cost me real time. First, the timestamp is the time the event happened, not the time I ingested it. When I backfilled old audits with the ingestion time, everything looked like it happened on the same day, and "what did we know last spring?" stopped meaning anything. Second, I write the content as a sentence with the rationale included. Retaining a bare status flip (open -> accepted) gave recall nothing to work with. The why is the part auditors want back.

Choosing what to retain

The hardest design question wasn't how to remember. It was what. If you retain every click and every chat message, recall gets noisy, and the agent confidently surfaces trivia. I settled on three categories:

Findings and their status changes, with the stated rationale.

Decisions, such as risk acceptances and policy exceptions, with who made them.

Explanations given in chat, but only when an auditor marks an answer as useful or corrects it.

That third rule matters. Without it, the agent remembers its own mistakes as facts. I'd rather it remember nothing than remember a wrong answer, because a stale fact stated with confidence is much harder to catch than a plain "I don't have that."

Corrections go through the same retain call, with the correction written out explicitly, so the newer statement can supersede the older one.

The answer path

At question time the agent does two things. It pulls the structured data it needs from the API, and it asks memory what it already knows:

async def answer(question: str, department: str, user: User) -> Answer:

bank = f"audit-{department}"

history = memory.recall(bank_id=bank, query=question)

live = await audit_api.snapshot(department, user=user)

if needs_judgement(question):

reasoned = memory.reflect(bank_id=bank, query=question)

return compose(question, live, history, reasoned)

return compose(question, live, history)

I use recall for lookup-style questions ("what was finding GY24-011 about?") and add reflect when the question needs a judgement ("is this the same root cause as before?"). The live snapshot always comes from the database, never from memory. Memory is for what happened and why. The database is for what's true right now. Mixing those two up was my second-worst bug, and I'll come back to it.

What it looks like in use

The clearest case is a recurring gap. A reviewer asks:

Why is Customer Support showing another overdue access review?

The agent responds, roughly:

Customer Support had the same gap in the March audit (CS24-021). It was marked accepted pending a tooling change, and the note says the quarterly review was deferred until the new identity system went live. The control is still assigned to the same owner. The current overdue item looks like the same control, not a new one.

Nothing in the current database row says any of that. The row says "overdue." The history came from memory, with the rationale intact and the date attached.

A second pattern I use constantly: asking the agent to compare what it knew at different points. Because events carry their real timestamps, "what was the open risk count in Finance before the June audit?" gets answered from retained events instead of whatever the dashboard shows today.

The interface: where I spent more time than I planned

I assumed the UI would be an afternoon. It wasn't, and the decisions taught me as much as the memory layer did.

Color carries meaning, so I rationed it. Compliance data is mostly status. I kept a restrained blue base and reserved green, amber, and red strictly for compliant, partial, and non-adhering. The moment I used green decoratively, users stopped being able to trust it as a signal. These live as tokens, and nothing else is allowed to use them:

:root {

--surface: rgba(255, 255, 255, 0.06);

--accent: #2f6bff;

/* status only: never decorative */

--status-ok: #22c55e;

--status-partial: #f59e0b;

--status-fail: #ef4444;

}

The glass look has a contrast bill. Translucent panels over a soft background look good and make text hard to read. I keep the blur subtle and check contrast on every card against the actual background rather than trusting my eyes on a good monitor.

Animations have to explain a change. Counters tick up, charts draw in, progress bars fill. That's it. Anything purely decorative got cut, because an animation that delays someone from reading a number is costing them time.

Lessons learned

1. Memory and the database answer different questions. The database says what's true now. Memory says what happened and why. My worst bug was letting the agent answer a current-status question from a remembered event, and it described a finding as open after it had been closed. Live state always comes from the source of truth now.

2. Retain the reasoning, not just the event. A status change without its rationale is nearly useless on recall. Write the why into what you store.

3. Timestamps are data, not metadata. Use the time the event occurred. Ingestion time makes history collapse into a single day.

4. Scope your memory banks like you scope permissions. Per-department banks were an easy decision early and would have been painful to retrofit.

5. Wrong memories are worse than none. Build the correction path before you build the recall path, and be strict about what the agent is allowed to retain from its own answers.

There's also a lesson that isn't about memory at all. In an early build, the same metric appeared in two panels with slightly different values, 65% in one and 62% in the other. It was a plain two-sources-of-truth bug, and every number on the dashboard now comes from one query path. Most of the real work in this project was edge cases like that, empty states, a 0% that means "no data" versus "none compliant," and an audit that's "due" versus "in progress." The happy path is the small part.

If you want to try the memory side yourself, the Hindsight repository on GitHub is where I'd start, and the retain, recall, and reflect split is worth understanding before you design anything around it.

Links and further reading

• Hindsight on GitHub: the open-source agent memory system behind Veriq's audit memory.

• Hindsight documentation: the retain, recall, and reflect operations and how memory banks work.

• What is agent memory?: Vectorize's overview of agent memory and how it differs from plain retrieval.