DEV Community: Mir Arshad Ali Talpur

How to Give Your AI Agent Memory You Can Audit

Mir Arshad Ali Talpur — Tue, 27 Oct 2026 10:12:25 +0000

Giving an agent memory is easy. Knowing what it remembered, and whether that memory changed its answer, is the hard part. Most memory setups are a vector store behind a retrieval call. The agent receives some text, acts on it, and nobody records which memories were used. When the agent says something wrong, you cannot tell whether the model failed or a stale note from three months ago misled it.

This guide shows how to build agent memory that stays inspectable, using ZizkaDB’s search and context functions. The principle is simple: memory should be part of the record, so every answer can be traced to the memories behind it.

A note on scope: the SDK documents search(query, agent?) for semantic search and context_for(agent, task) for a prompt-ready memory block, and log() embeds events when tenant embeddings are configured. I could not find a public reference for the exact return fields, ranking behavior, or limits, so the code below is illustrative. Check the SDK reference before relying on specifics, and print results to see their structure.

Before you start: why unaudited memory is a risk

Memory fails in ways that are hard to see:

Stale facts. The agent recalls a price, policy, or preference that stopped being true.
Wrong recall. A similar but irrelevant memory gets retrieved and treated as relevant.
Cross-contamination. One customer’s information surfaces in another customer’s session.
Poisoned memory. Bad or malicious content was written into memory once and now influences every later answer.
Personal data you cannot find. A user asks for deletion, and copies live in a vector store nobody mapped.
Silent cost. Every retrieved memory is more input tokens on every call.

An auditable memory addresses each one by recording what was stored, what was retrieved, and what the agent did with it.

Step 1: Decide what memory is for

Write down the job of memory in your agent, in one sentence per use. Typical uses:

User preferences and history, so the agent does not ask the same questions twice
Past resolutions, so similar issues get consistent answers
Working notes inside a long task, so progress survives a long session
Organizational knowledge, such as policies and product facts

Each use has different rules. Preferences are personal data. Policies change and need freshness checks. Working notes can expire. Decide per use what is allowed into memory, how long it should live, and who may read it.

Step 2: Store memory as events, not as a side database

Instead of a separate vector store that only the agent can see, write memory as events in the same record as everything else. Each event carries its content, a type, a session, and a link to what caused it.

python

illustrative, check the SDK reference for exact parameters

note = await db.log(
agent="support-bot",
event="memory_written",
data={
"kind": "preference",
"text": "Customer prefers email over phone contact",
"subject_id": "cust_8842",
"source": "user_message",
"valid_until": "2027-04-27",
},
session_id=session_id,
parent_id=user_event_id,
)

Three design choices make this auditable:

Link the memory to its source. The parent link means you can later answer where a memory came from and which conversation created it.

Tag a subject identifier. If the memory is about a person, include a stable identifier such as a hashed customer ID, so erasure can find it later.

Record freshness. Add a validity date or version where the fact can go stale.

Since the SDK says events are embedded when tenant embeddings are configured, check that embeddings are set up for your deployment. Without them, semantic search will not return what you expect.

Step 3: Retrieve memory with a task in mind

Before the agent answers, ask for relevant memory. The SDK offers two functions:

search(query, agent?) returns events that match a query by meaning. Use it when you want a list to inspect or filter.
context_for(agent, task) returns a prompt-ready block of relevant memory for a task. Use it when you want to inject memory straight into the prompt.
python

illustrative

hits = await db.search("customer contact preferences", agent="support-bot")
block = await db.context_for("support-bot", task="Reply to a delayed order complaint")

Print both at first. Read what comes back with a critical eye: is it relevant, current, and about the right customer? Retrieval quality is rarely what you assume on the first try.

Step 4: Record the retrieval itself

This is the step that makes memory auditable. After retrieving, log an event that says which memories were supplied to the model, and link it between the user message and the model call.

python

illustrative

retrieval = await db.log(
agent="support-bot",
event="memory_retrieved",
data={
"task": "Reply to a delayed order complaint",
"memory_event_ids": [h.event_id for h in hits], # check the field name
"memory_count": len(hits),
"approx_tokens": estimated_tokens,
},
session_id=session_id,
parent_id=user_event_id,
)

make the next LLM call a child of `retrieval`

Now the causal chain for any answer runs from the user message, through the retrieval, to the memories used, to the model response. When something goes wrong, zizkadb why shows whether a particular memory fed the answer. I could not confirm whether context_for records this automatically, so check one session in the dashboard, and log it yourself if it does not.

Step 4b: Use the MCP route if you work in an editor

If your agent is an MCP-capable tool such as Cursor, Claude Desktop, or Windsurf, the ZizkaDB MCP server exposes memory as tools. Package listings show log_event to record, search_memory to search by meaning, get_context for a formatted memory block, why for the causal chain, query_events to list recent events, and time_travel for past state. The setup is a short config entry with uvx zizkadb-mcp, as covered in the Cursor guide. Tool names may differ between versions, so check the tools your server lists.

The same caution applies: the agent must actually call the memory tools for anything to be stored or retrieved. A rule in your agent instructions telling it when to log and when to search helps.

Step 5: Control how much memory goes into the prompt

Memory costs money and attention. Every memory you inject is input tokens, and too many dilute the useful ones. Set limits and test them:

A maximum number of memories per call
A relevance cutoff, so weak matches are dropped
A token budget for the memory block
A preference for recent or validated memories where freshness matters

Log how many memories and roughly how many tokens each retrieval added. Over a week, that tells you what memory costs and which tasks gain from it.

Step 6: Handle stale and conflicting memories

Memory that is never updated becomes wrong. Build a simple policy:

Expiry. Use the validity date from Step 2, and skip expired memories at retrieval time.
Supersession. When a fact changes, write a new memory that references the one it replaces, so the old one is not retrieved first.
Conflict handling. If retrieval returns two memories that disagree, tell the model so, or prefer the newer one, and log which was chosen.
Review. Periodically sample stored memories of the kinds most likely to go stale, such as prices and policies.
Step 7: Keep customers’ memories apart

Make sure a retrieval for one customer can never return another’s memory. The SDK supports agent-scoped keys and tenant isolation, and you should also filter by the subject identifier where memory is personal. Then test it on purpose: write a distinctive memory for one test customer, query as another, and confirm it never appears. Keep the test result, since security reviews often ask for it.

Step 8: Plan for deletion

Memory about a person is personal data, and embeddings made from their messages are too. The SDK includes forget(filter_key, filter_value) for erasure by metadata filter, and the project describes it as covering both event logs and vector data.

python

illustrative

await db.forget("subject_id", "cust_8842")

Test it on synthetic data first. Write memories for a test subject, run the erasure, then confirm that search and retrieval no longer return anything about them, and that other customers’ memories and their audit trail are intact. This is the reason Step 2 asked for a subject identifier on every personal memory.

Step 9: See what the agent knew at a given time

When a dispute concerns an old answer, you need the memory as it was then, not as it is now. The SDK’s at(agent, timestamp) reconstructs state at a timestamp, and the MCP tool list includes time_travel. Use it to check which memories existed, and combine it with the retrieval event from Step 4 to see which of them the agent actually received. That separates the agent knew it from the agent was shown it.

Step 10: Test memory like a feature

Treat memory quality as something you measure:

Recall check. Write a few known facts, then ask related questions and confirm they are retrieved.
Precision check. Ask unrelated questions and confirm irrelevant memories are not injected.
Freshness check. Update a fact and confirm the old one stops appearing.
Isolation check. Repeat the cross-customer test after every significant change.
Poisoning check. Try writing a deliberately misleading memory, and confirm you can find it, trace what it influenced, and remove it.
Cost check. Compare tokens per session with and without memory, and compare answer quality.

Run these after changes to embeddings, retrieval settings, or prompts. Also watch the behavioral drift view, since a change in memory settings often shows up as a shift in how many retrieval and model events each session contains.

What the business case looks like

These figures are modeled assumptions, not measured results. Replace them with your own.

Assume a support agent handles 20,000 sessions a month, and 10 percent are repeat customers where memory avoids re-asking questions. If memory saves about 2 minutes of conversation on each, at 40 euro per hour of support cost equivalent, that is 2,000 sessions times 2 minutes, about 67 hours, worth roughly 2,700 euro a month.

Add the investigation side. If 8 incidents a month involve a questionable memory, and tracing it takes 5 hours without a retrieval record against 1 hour with one, at 100 euro per hour, that is about 3,200 euro a month saved.

Then the cost side. If unrestricted memory adds 1,500 input tokens to every call and you trim that to 500 through limits and cutoffs, the saving depends on your price per token and volume, so calculate it from your own usage logs.

The harder figure is risk. A customer’s private information surfacing in another customer’s session, or a deletion request you cannot fully honor, is a serious incident with legal and reputational cost. The tests in Steps 7, 8, and 10 are cheap insurance against it.

Common mistakes to avoid

Keeping memory in a separate store nobody audits. If it is not in the record, you cannot trace it.

Not logging the retrieval. Without it, you cannot tell which memories influenced an answer.

Injecting too much. More memory means more tokens and more distraction, so cap it.

Never expiring anything. Stale facts become confident wrong answers.

Forgetting subject identifiers. You cannot erase what you cannot find.

Skipping the isolation test. Cross-customer leakage is the failure that ends deals.

Assuming retrieval is relevant. Read real results, and measure recall and precision.

Where to go from here

Pick one memory use, such as customer preferences. Store it as events with a subject identifier, retrieve it before the model call, and log the retrieval. Then run the isolation and deletion tests. That gives you a memory you can explain, limit, and remove, which is the version enterprises are willing to approve.

If you tell me what your agent remembers today and where it is stored, I can help map it onto this approach and design the tests.

Want to test ZizkaDB on your agent? Try our open source version here: https://github.com/ZIZKA-AI-SL/ZizkaDB

Interested in a design partnership? Fill in the form on our site or reach me directly at founder@zizka.ai

Sources: ZizkaDB Python SDK wiki · ZizkaDB MCP and Cursor wiki · zizkadb-mcp on PyPI · ZizkaDB repository README

Meet the LLM Accountant: The Role Nobody Has Hired Yet

Mir Arshad Ali Talpur — Mon, 26 Oct 2026 10:38:39 +0000

*Every month, a company that builds on language models receives an invoice from its model provider. It is usually one large number, sometimes with a split by model. Then someone in a meeting asks what the number bought, and nobody can answer.
*
That gap is the reason for a job that does not have a name yet. I am proposing one: the LLM accountant. The nearest established term is AI FinOps, but the work is closer to accounting than to infrastructure. It is about tracing every unit of spend to its cause, reconciling it against the invoice, explaining why it moved, and being able to prove it when someone asks.

Why this role should exist
Cost now comes from behavior, not capacity. Traditional cloud cost follows servers and traffic. LLM cost follows decisions made by software: how much context to send, which model to call, whether to retry, how many tool results to feed back. A prompt edit is a spending decision. A retry rule is a spending decision. Nobody approves these as spending decisions today.

One request can cost fifty times another. Two users can trigger the same feature, and one costs a few cents while the other costs tens of euros because a tool timed out and the agent looped. Averages hide this completely.

The invoice cannot answer the real questions. Which feature, which step, which customer, which prompt version? Provider dashboards give totals. The questions that save money need cost attached to context.

Some customers cost more than they pay. Without cost by tenant, nobody knows which ones, and pricing stays a guess.

Other people are starting to ask. Finance wants forecasts. Customers on usage-based plans want to know why their bill moved. Auditors and regulators ask how you know what an automated decision cost and why it happened. Someone must own those answers.

Nobody owns it today. Engineers see the symptoms, finance sees the total, and product sees neither. The work falls between them and gets done in a panic when the bill jumps.

That is the shape of every accounting role in history: money moving through a system nobody can fully see, and a person whose job is to make it legible.

What the LLM accountant owns
The chart of accounts: how spend is sliced into feature, agent step, model, tenant, and prompt version.
The ledger: a record of every model call with its token counts and context.
The rate card: dated prices per model, kept as history.
Reconciliation: monthly matching of the ledger to the provider invoice.
Variance analysis: explaining why any cost head moved.
Recommendations: a fix and an estimated saving for every finding.
Controls: per-session limits, spend alerts, and budgets.
Evidence: being able to show where any number came from.
What they should calculate, and why
A short list of numbers does most of the work. Each answers a question someone will eventually ask.

CalculationHowWhy it mattersCost per sessionTotal cost divided by sessionsNormalizes spend against trafficCost per completed taskCost divided by successful outcomesThe only honest efficiency number, since cheap failures are not savings95th percentile versus average costCompare the twoA big gap means a few runaway sessions are inflating the billInput versus output shareInput tokens cost against output costTells you whether the problem is context or verbosityCost by feature and by stepGroup the ledgerFinds the one place most of the money goesMargin by tenantRevenue minus model costFinds customers who cost more than they payPremium-model share on simple stepsSpend on large models for classification, extraction, routingShows routing savingsRetry and duplicate rateRepeated calls divided by total callsPure waste, and usually cheap to fixCache hit shareCached input tokens divided by input tokensShows whether caching actually worksReconciliation gapLedger total minus invoiceProves whether the ledger can be trusted

The most useful calculation borrows from classic accounting: the variance bridge. When the bill rises, split the increase into four causes:

Volume: more sessions than last month.
Price: the provider changed prices.
Mix: traffic moved toward more expensive features or models.
Behavior: the same work now uses more tokens, through longer context, extra steps, or retries.
Only per-call data can produce this split. The provider invoice cannot, and it matters because each cause needs a different response. Volume is good news, price is a negotiation, mix is a product decision, and behavior is an engineering bug.

Why ZizkaDB is the right tool for this job
Plenty of tools can show a usage chart. The accountant’s job needs something different: the ability to go from a total to its cause and defend it. ZizkaDB is built for that.

Every number links back to its cause. Each call is recorded as an event tied to the session and the step that caused it. A cost head is not just a figure. You can click from it to the sessions behind it, replay one, and ask why a particular step happened.

The audit trail doubles as the ledger. Events carry tamper-evident checksums, so the record behind a cost number can be shown to a finance team, a customer, or an auditor as unchanged since it was written. A spreadsheet export cannot offer that.

It sees behavior, which is where cost comes from. The drift view shows when the number of model calls per session or the share of tool steps changes. That is the behavior component of the variance bridge, spotted as it happens instead of at month end.

Your cost data stays in your hands. The open source version is self-hostable, so prompts, usage, and customer identifiers need not go to a third party. For cost data that reveals your margins and your customers’ behavior, that matters.

Customer data can be removed cleanly. If a customer asks for deletion, an erasure function covers both event logs and vector stores, which means tenant-level cost records need not conflict with data protection duties.

It was built for questions you will be asked later. The same record that explains a cost spike also answers what the agent did, why, and what it looked like at the time. One trail serves the accountant, the engineer, and the auditor.

To be clear about limits: ZizkaDB is the ledger and the evidence, not an accounting package. The rate card, the reconciliation, and the judgment stay with the accountant. And I could not confirm a built-in cost report, so expect to assemble the aggregation yourself at first, from the token fields you log.

What goes into the ledger is small. One event per model call, with fields like these:

python

illustrative ledger entry data, one per LLM call

{"model": "model-large", "feature": "support_reply", "step": "retrieval_answer",
"tenant": "tenant_a", "prompt_version": "v14",
"input_tokens": 9200, "output_tokens": 410, "cached_input_tokens": 0}
Store tokens, not money, and apply a dated rate card when you analyze, so history stays correct when prices change.

Worked example: dissecting one invoice
This example is modeled, not measured. It shows what the work produces once the ledger exists. Take a product whose provider invoice for October says one line: 12,000 euro.

Head 1: by feature
FeatureCostShareSupport reply5,40045%Document summarization3,00025%Ticket classification and routing1,80015%Voice assistant1,20010%Internal copilot6005%Total12,000100%

First finding: support reply is nearly half the bill. Start there.

Head 2: by step inside support reply
StepCostPlanning and tool selection1,100Retrieval and context assembly2,300Final answer1,500Retries and validation loops500Total5,400

Second finding: the answer itself is under a third of the feature’s cost. Retrieval and context assembly is the largest part, and retries cost 500 euro for work that produced nothing.

Head 3: by model, and input versus output
CutCostShareLarge model8,40070%Small model3,60030%Input tokens7,80065%Output tokens4,20035%

Third finding: input dominates, so this is a context problem more than a verbosity problem. And all 1,800 euro of ticket classification runs on the large model, a task a small model usually handles.

Head 4: by tenant, with margin
TenantModel costMonthly revenueMarginTenant A2,9002,000–900Tenant B1,7004,0002,300Other 38 tenants7,400not shownaverage cost about 195 eachTotal cost12,000

Fourth finding: Tenant A is a quarter of the bill and loses money. That is a pricing or usage-limit conversation, and it was invisible on the invoice.

Head 5: the excess, meaning cost that bought nothing
Excess headCostPremium model on simple steps (excess over a small model)1,500Context bloat (history and documents resent unnecessarily)1,300Retry loops700Duplicate calls400Total excess3,900

That is 32.5 percent of the invoice. Recovering half to two thirds of it is a realistic goal, which is roughly 1,950 to 2,600 euro a month.

The variance bridge
Last month’s bill was 9,500 euro. This month’s is 12,000. Why?

CauseChangeVolume (more sessions)+1,100Price (provider adjustment)-300Mix (more summarization and voice)+600Behavior (more tokens per session)+1,100Total change+2,500

Without the ledger, the answer to why did it rise would have been traffic grew. With it, you can say that 44 percent of the increase was behavior, which is fixable, and then find which prompt change caused it.

Drilling into one number
The accountant sorts support sessions by cost. The average session costs 0.90 euro, but one costs 41 euro. Opening it in session replay shows 23 model calls. Asking why on the first expensive step shows the chain: an order lookup tool timed out, the agent retried, timed out again, and each retry resent a 14,000 token history. The cause is not the model. It is a slow tool with no retry cap. The fix is a retry limit and a trimmed history, and the saving applies to every session that hits the same pattern.

That is the work: from 12,000 euro, to a feature, to a step, to a session, to a cause, to a fix. The ability to follow that path is what the role is paid for.

What it adds up to
In this example, the first close produces four decisions: cap retries, route classification to a smaller model, trim retrieval context, and open a pricing conversation with Tenant A. Together they target the 3,900 euro of excess and the 900 euro monthly loss on one account. The labor is a few engineer-days to build the ledger and a small project per fix, so payback is typically a few months, with the saving recurring. Treat these as modeled figures and measure your own before promising anything.

Common mistakes to avoid
Skipping reconciliation. A ledger that never meets the invoice is an opinion.

Storing money instead of tokens. Prices change, so store tokens and apply a dated rate card.

Judging by cost per call. Cost per completed task is the number that matters.

Cutting cost without watching quality. A cheaper model that causes rework raises total cost.

Reporting without recommending. A variance with no proposed fix gets ignored.

Logging full prompts everywhere. It adds volume and risk, and cost accounting does not need it.

Start this week
You do not need a department. Pick one product feature, record every model call for it with model, tokens, step, and tenant, keep a dated price table, and compare your total with the invoice at month end. When the two numbers meet, you have a ledger. When you can explain a variance, you are doing the job.

If you tell me your provider, your models, and your rough monthly spend by feature, I can help draft your chart of accounts and a first close template.

Want to test ZizkaDB as your ledger? Try our open source version here: https://github.com/ZIZKA-AI-SL/ZizkaDB

Interested in a design partnership? Fill in the form on our site or reach me directly at founder@zizka.ai

Sources: ZizkaDB Python SDK wiki · ZizkaDB connection guide · ZizkaDB repository README

How to Detect and Investigate Agent Drift with ZizkaDB

Mir Arshad Ali Talpur — Fri, 23 Oct 2026 10:16:32 +0000

This is the next how-to guide in the series. The earlier drift article explained what behavioral drift is and why it matters. This one is practical: how to set your agent up so drift is measurable, how to read the dashboard, how to tell a real change from noise, and what to do when the score moves.

It is written for engineers and team leads who ship prompt, model, or tool changes regularly and want to know, within days, whether the agent now behaves differently.

A note on scope: the SDK documentation lists baseline(agent) as the method that returns a behavioral drift signal, and the dashboard has an Agent Behavior view. I could not find a public reference for the exact scoring formula, alert thresholds, or the full return fields of baseline(), so this guide sticks to what the product shows and tells you where to verify. Check the repository docs for those details before you build automation on them.

Before you start: what drift actually measures

Drift compares an agent’s recent behavior with its own established baseline. In the dashboard, the comparison covers the shape of what the agent does: how often each event type occurs, how often one event follows another (the transitions), how long sessions run, and how often errors occur. In a typical reading, the window might be the newest 50 sessions compared against the 890 sessions before them.

Three things follow from that design:

Drift is relative to your own history. There is no universal good score, only change against your baseline.
It measures change, not quality. An agent that got faster and more accurate can still show drift.
It depends on stable data. If your event names or logging change, the comparison breaks, and the score will reflect your instrumentation, not your agent.
Step 1: Make your data drift-ready

Do this before you trust any score.

Keep event types fixed. Use a small, stable set of types such as user_message, decision, llm_start, llm_end, tool_start, tool_end, assistant_response, and error. Put variable details in the payload, not in new event names. Inventing new types per release fragments the baseline.

Name your tools. The SDK provides log_tool and log_tool_result, so the dashboard can show which tool changed instead of only that tool calls changed. The LangChain, LiveKit, and MCP integrations already do this.

python
call = await db.log_tool("support-bot", "lookup_order", args={"order_id": "ORD-8842"})
await db.log_tool_result("support-bot", "lookup_order", result=order, parent_id=call.event_id)

Link events with parent_id. Transitions come from parent and child relationships, so a missing parent link silently distorts the transition view.

Use a stable agent name. Drift is computed per agent, so renaming an agent starts a new history.

Step 2: Let a baseline form

Drift needs history to compare against. Run the agent in real conditions long enough to collect a meaningful volume of sessions before you rely on it. There is no magic number, but a baseline built from a few dozen sessions is fragile, and one built from several hundred is far steadier. If your traffic is low, widen the window rather than reading tiny samples.

Be careful about what the baseline contains. If it spans an earlier version with very different behavior, today’s drift score partly reflects that old behavior. After a deliberate, large change, expect the score to stay elevated until enough new sessions accumulate.

Step 3: Open the Agent Behavior view and choose a window

The dashboard offers windows for Sessions, 24h, 7d, and 30d. Pick based on your traffic:

Sessions compares the most recent batch of sessions against older ones. Good for steady traffic and for checking a release.
24h and 7d suit agents with daily patterns, such as business hours, where comparing by session count could mix very different times of day.
30d shows slow movement that shorter windows can miss.

Use the Recompute button after a window change or after new data arrives. If the window is small, treat the result as a hint, not a verdict.

Step 4: Read the headline, then the health context

The top of the view gives a drift label, a percentage, and a score. In one reading it looked like this: minor drift, 9 percent behavior change, score 0.090, with the note that small shifts in event distribution are worth a quick look.

Read that as triage, not diagnosis. Then read the health context next to it, because it tells you whether the change did damage:

Sessions analyzed, which shows how many recent sessions were compared with how many prior ones. Check this first. A tiny recent window makes everything noisier.
Error rate delta, shown in percentage points. A negative number means fewer errors.
Average session length, shown as a percentage change, with a note on whether it is within a normal range.

A drift score that is up while errors are down and sessions are shorter points toward a change in approach, not a failure. A drift score that is up while errors are up points toward a regression. The combination matters more than any single number.

Step 5: Read the biggest changes

This is where the diagnostic value sits. The view ranks the events and transitions that moved the most, in percentage points, between the recent window and the baseline. Each row shows whether it is an event or a transition, the before and after share, and the change.

In the earlier example, the largest movers were:

The share of assistant_response events fell from 11.51 percent to 6.78 percent.
The transition from decision to assistant_response fell from 4.13 percent to 0.00 percent.
tool_start, tool_end, and the transitions between llm_end, tool_start, and tool_end all rose by roughly 2 percentage points.

Turn that into a plain sentence before doing anything else: the agent now goes through tools more often and goes from decision to response directly far less often. That sentence is your hypothesis.

Two cautions when reading the list:

A transition dropping to exactly zero deserves suspicion. Behavior rarely changes to precisely nothing. It may mean the code path was removed on purpose, or that something stopped being logged, such as a changed event name or a missing parent link. Check your instrumentation before you assume the agent changed.

Look at shares, not only counts. Shares are percentages of all events, so when one event type grows, others shrink, even if nothing happened to them. Read related rows together.

Step 6: Compare the baseline and recent panels

Below the ranking, side-by-side panels show each period in full: sessions and events analyzed, events per session, average duration, error rate, the event distribution, and the most common decision sequences, meaning parent to child pairs.

Use them to check your hypothesis against raw numbers. In the example, events per session went from 12.3 to 10.6, average duration from 56 seconds to 51 seconds, and error rate from 1.2 percent to 0.6 percent. Fewer events, faster sessions, and fewer errors fit a story of a more direct, more efficient path, which makes the shift look benign, though not yet understood.

Step 7: Pull the signal programmatically

For automation, the SDK exposes the drift signal as baseline(agent):

python
from zizkadb import ZizkaDB

async with ZizkaDB(host="http://localhost:8000") as db:
result = await db.baseline("support-bot")
print(result)

Print the full result first and read its fields before you write any logic against it, since I could not confirm the exact structure. Once you know it, you can run the check in a scheduled job, a release pipeline, or a notebook, and post a summary to your team channel.

Step 8: Decide whether the change was expected

Drift only means something against what you intended. Before investigating, answer one question: did we change something deliberately in this period?

Keep a simple change log with date, change, and owner, covering prompts, models, tools, retrieval settings, routing logic, and upstream provider updates you know about. Then sort each drift finding into one of three groups:

Expected: you changed something, and the shift matches what you meant to change. Confirm it and note it.
Unexpected but explained: you did not intend it, but you can find a cause, such as a provider model update or a changed tool response.
Unexplained: nothing in your change log fits. These are the ones to investigate first.
Step 9: Investigate with replay, lineage, and point-in-time state

When a shift needs explaining, use the other functions in order:

Use Session Replay on a few sessions from the changed period, and compare them with sessions from the baseline. Look for the specific path that changed, such as a verification step missing or an extra tool call.
Use zizkadb why or db.why() on the step where the new path begins, to see what inputs and context led there.
Use db.at() to reconstruct what the agent looked like around the time the shift began, including prompt, tools, and model, and compare with an earlier time.
Check the tool and API error rates for the tools involved, since a tool that started failing often produces a shift to different paths.

Write down the cause category: prompt change, model change, tool behavior, retrieval change, user mix change, or instrumentation change.

Step 10: Decide, act, and record

Each investigation ends in a decision:

Accept. The change is intended or harmless. Note it, and consider whether the baseline should now be understood as starting from this change.
Fix. The change is harmful. Adjust the prompt or tool, add a check, or roll back, then watch the score and the biggest-changes list for the following days to confirm it reverses.
Watch. You cannot tell yet. Set a review date, and widen the window if volume is low.

Record the outcome in the same change log. Over time this becomes the history that tells you which kinds of changes move your agent most.

Step 11: Turn it into a routine and a release check

A single look is not a process. Set a light routine:

Review the drift view after every prompt, model, or tool change, and once a week otherwise.
Pick a few transitions that must never disappear, such as a verification step before a payment, and check them by name each time.
For releases, run your usual scenarios and compare behavior against the baseline before and after. LLM output is not deterministic, so use enough runs to separate noise from signal, and treat the score as a trigger for review, not an automatic pass or fail.
Tighten human review temporarily when drift appears, for example by sampling more of the agent’s actions until you understand the change.

If you run many agents, ZizkaDB’s Fleet feature is described as covering multi-agent monitoring across projects. Check the current docs for how it presents drift across agents.

What the business case looks like

These figures are modeled assumptions, not measured results. Replace them with your own.

Assume a team ships 3 significant prompt or model changes a month at 100 euro per engineering hour. Without drift monitoring, a harmful change is typically found through ticket volume after about 10 days, and cleanup takes around 40 hours, costing 4,000 euro per incident. With drift flagged within about 2 days, the fix is more targeted and takes about 6 hours, costing 600 euro. If one change in three turns out harmful, the monthly saving is roughly 3,400 euro.

Add investigation time. If a team handles 5 behavior investigations a month, starting from a ranked list of changed events instead of a blank page might reduce each from 4 hours to 1, saving about 1,500 euro a month.

The harder number is the incident that never reaches a customer. A regression caught in two days rarely produces complaints, refunds, or churn. That is difficult to model honestly, so measure it on your own releases before counting it.

Common mistakes to avoid

Treating the score as a verdict. It says behavior changed, not whether that is good.

Reading small samples. A window of a handful of sessions produces noise. Check the sessions analyzed count first.

Changing event names between releases. The baseline fragments, and the score reflects your logging.

Ignoring a transition that drops to zero. Check for a removed code path or a logging gap before assuming behavior changed.

Skipping the change log. Without it, you cannot sort expected shifts from unexplained ones.

Forgetting the baseline contains history. After a deliberate large change, the score stays elevated until new sessions accumulate.

Using drift alone. Pair it with error rate, session length, and replay of real sessions.

Where to go from here

Open the Agent Behavior view for your busiest agent today. Read the sessions analyzed count, the health metrics, and the top three changes, then write your hypothesis in one sentence and check it with a replay. After that, start a change log and add a weekly review. The earlier guides on incident investigation, point-in-time reconstruction, and oversight all connect to this, since drift is usually the first signal that sends you to them.

If you tell me how often you ship changes and what the agent does, I can help you choose which transitions to guard and how to set up the review routine.

Want to test ZizkaDB on your agent? Try our open source version here: https://github.com/ZIZKA-AI-SL/ZizkaDB

Interested in a design partnership? Fill in the form on our site or reach me directly at founder@zizka.ai

Sources: ZizkaDB Python SDK wiki · ZizkaDB repository README

How to Use ZizkaDB with LiveKit to Audit Your Voice Agents

Mir Arshad Ali Talpur — Thu, 22 Oct 2026 11:34:26 +0000

This is the next how-to guide in the series. The earlier guides covered text agents, LangChain, and Cursor. This one is for teams building voice agents on LiveKit: support lines, appointment booking, collections, banking call handling. Voice adds a problem text does not have. When a customer says the agent told them something wrong on a call, nobody can scroll back through the conversation, and the transcript, the tool calls, and the decisions behind them live in different places.

ZizkaDB’s LiveKit integration maps one call to one session, so you can open a call and see the transcript, the tool calls, and the causal chain behind what the agent said. Every command and code snippet below comes from the repository’s LiveKit documentation.

A note on scope: this guide covers the open source self-hosted setup. Package versions were current when I checked, but the docs list slightly different patch numbers (0.2.0 and 0.2.1), so confirm the latest on PyPI. Plan details, compliance questions, and recording consent rules are things to confirm with your own legal counsel.

Before you start: what you get

Voice pipelines are noisy. A single call produces speech turns, interruptions, timeouts, state changes, tool calls, and token usage. The integration organizes this into one session per LiveKit call, with these event types:

session_started and session_ended user_message and assistant_response user_transcript, the final speech-to-text result tool_call and tool_result agent_handoff for multi-agent workflows State, interruption, and timeout events usage_updated for token usage error

Each event links to the one that caused it through a parent_id, which is what makes the Why? view and db.why() work for voice the same way they do for text.

Two design facts matter for production. First, no audio is stored in ZizkaDB. Transcript text is copied from LiveKit’s session report at the end of the call. Second, ZizkaDB writes go onto a background queue, so nothing on the logging path blocks the audio path, and a ZizkaDB outage degrades observability, not the call.

Step 1: Start ZizkaDB

You need Docker. This starts the stack without cloning the repo, and the first image pull can take a while:

bash curl -fsSL https://raw.githubusercontent.com/Zizka-ai/ZizkaDB/main/scripts/quickstart-remote.sh | bash

The API runs at http://localhost:8000 and the dashboard at http://localhost:3001. On localhost there is no signup and no API key.

Step 2: Install the integration

One command:

bash pip install zizkadb-livekit

Per the docs, this pulls in zizkadb-sdk>=0.3.0 and livekit-agents>=1.3.0 automatically, so there is no separate SDK install. If you already have a LiveKit agent, check that the version it pins is compatible.

Step 3: Set the environment Variable What it does ZIZKADB_HOST Self-host API URL, default http://localhost:8000 ZIZKADB_AGENT Agent name, which must match the dashboard ZIZKADB_API_KEY Cloud key (zizkadb_live_...) used instead of a host ZIZKADB_EVENT_LEVEL Verbosity: transcript, standard (default), or verbose

You also need your usual LiveKit variables, which are LIVEKIT_URL, LIVEKIT_API_KEY, and LIVEKIT_API_SECRET.

Step 4: Wire the observer into your agent

This is the full wiring from the docs:

python import os from livekit.agents import AgentServer, JobContext, AgentSession, Agent from zizkadb import ZizkaDB from zizkadb_livekit import ZizkaDBLiveKitObserver, pop_observer, register_observer

server = AgentServer()

async def on_session_end(ctx: JobContext) -> None: observer = pop_observer(ctx) if observer is None: return try: await observer.ingest_session_report(ctx) finally: await observer.aclose() # flushes queued events, closes the connection

@server.rtc_session(agent_name="support-voice", on_session_end=on_session_end) async def entrypoint(ctx: JobContext): db = ZizkaDB(host=os.getenv("ZIZKADB_HOST", "http://localhost:8000")) observer = ZizkaDBLiveKitObserver( db, agent=os.getenv("ZIZKADB_AGENT", "support-voice"), session_id=ctx.room.name, ) register_observer(ctx, observer)

session = AgentSession(...) # your STT / LLM / TTS

Attach before session.start() so no turn is missed.

observer.attach(session, job_ctx=ctx)
observer.queue_session_started(room=ctx.room.name, job_id=ctx.job.id)

await session.start(agent=Agent(instructions="..."), room=ctx.room)
await ctx.connect()

Three details are worth understanding, because they are where voice integrations usually go wrong:

One observer per call. The observer is registered and looked up per call using the job context. A worker serves many calls at once, so a shared module-level observer would let calls overwrite each other’s transcripts. Keep it per call as shown.

Attach before starting the session. Call observer.attach(...) before session.start(), or early turns can be missed.

Close it at the end. In the session end handler, ingest_session_report back-fills the full transcript, and aclose() flushes queued writes and releases the connection. Do not skip the try and finally, or the last events of a call may not be written.

If you want the quickest route, the repository includes a runnable voice worker with a local browser interface at http://localhost:8080 in examples/livekit-agent/. Run that first to see the whole flow before adapting your own agent.

Step 5: Make a test call and verify

Place a short test call. Then open the dashboard, go to Activity, choose your agent (for example support-voice), and open Sessions. You should see one session for the call, with the same event types as a text agent.

If nothing appears, check these in order: the stack is running, the host URL is reachable from where your worker runs, the agent name matches exactly, the observer is attached before the session starts, and the end of call handler actually runs and closes the observer.

Step 6: Pick the right verbosity level

Voice pipelines emit many more events than text agents, so events are tiered. A level logs its own tier and everything quieter. Errors and session start and end are always logged.

transcript: turns, tool calls and results, handoffs, errors, and session boundaries. Lightest, good for cost and privacy sensitive setups. standard (default): adds pipeline detail such as state changes, interruptions, timeouts, and token usage. verbose: adds partial transcripts, turn detector predictions, speech events, and a bounded raw copy of each LiveKit payload. Use it for debugging, not as the default.

You can set the level per observer with level="verbose" or globally through ZIZKADB_EVENT_LEVEL. A sensible pattern is standard in production, verbose in staging, and transcript where privacy rules require minimizing what you keep. Because verbose stores raw payloads, treat it as more sensitive.

Step 7: Investigate a call end to end

When a customer disputes what the agent said, use the same workflow as any other incident:

Open the call’s session and read the sequence, from the first user turn to the last assistant response. Find the first step where the call went wrong. In voice, look closely at user_transcript events, since many failures start with a misheard word, not a bad decision. Open the Why? (causal) tab on that step, or run zizkadb why , to see what led to it. Check tool_call and tool_result events for a failed or slow tool that the agent routed around. Look for interruption and timeout events near the failure, which often explain a clipped or repeated answer.

The speech-to-text layer deserves special attention. If the transcript says the customer said a different amount than they did, the agent’s reasoning may have been sound on bad input. Recording that distinction is the difference between fixing the model and fixing the transcription setup.

Step 8: Add manual events for what LiveKit cannot see

The observer captures what flows through LiveKit. For things outside it, such as a database write, a payment authorization, or a human approval, use the manual helpers log_tool_call, log_tool_result, and log_error, and link them into the chain with the appropriate parent. Without that link, why() stops at the edge of what LiveKit knew.

Step 9: Handle privacy and consent for voice

Voice calls raise questions text does not. Decide these before production:

Audio. ZizkaDB does not store audio, only transcript text. If your own stack records audio, that is separate and has its own consent and retention rules. Transcripts are personal data. Callers say names, account details, and health or financial information aloud. Redact or limit what your agent logs where you can, and use the transcript level where minimizing helps. Consent and notice. Rules on informing callers about recording and AI use vary by country and sector. Confirm yours with counsel before launch. Erasure. Plan for deletion requests. ZizkaDB has db.forget() for erasure by metadata filter, so tag sessions with a stable subject identifier from the start, such as a hashed customer ID, so you can find everything belonging to one person. Step 10: Test the failure modes

Before relying on the trail, test these on purpose:

Stop ZizkaDB mid call and confirm the call continues normally. The docs say failures are swallowed with a warning, so verify that on your setup. Run several concurrent calls and confirm each gets its own correct transcript. End a call abruptly, for example by dropping the connection, and confirm the session still closes and the transcript is back-filled. Check a call with a tool failure and confirm the error shows in the session.

Each of these is a case where silent gaps would otherwise only show up during a real dispute.

What the business case looks like

These figures are modeled assumptions, not measured results. Replace them with your own.

Assume a voice support line handles 300 disputed or escalated calls a month, with a reviewer at 50 euro per hour. Without a connected record, reviewing a call means pulling the recording, listening through, and matching it against tool logs, about 30 minutes each, costing 7,500 euro a month. With the transcript, tool calls, and causal chain in one session, review takes about 8 minutes, costing about 2,000 euro a month. That saves roughly 5,500 euro a month.

Add engineering time. If your team handles 10 voice incidents a month at 100 euro per hour, and the average falls from 5 hours to 1.5 hours with a connected trail, that is about 3,500 euro a month more.

The setup cost is small. A single LiveKit agent integration is plausibly a few hours of work, since it is one package and a short wiring block, so payback depends mostly on how often calls get questioned.

The larger value is in regulated settings. If your voice agent gives information about money, health, or contracts, the question you will eventually face is what exactly did the agent say, and why. A recorded chain answers that. Without one, you rely on recordings and memory.

Common mistakes to avoid

Using one shared observer across calls. Transcripts overwrite each other. Register one per call.

Attaching after the session starts. Early turns are missed.

Skipping aclose. Queued events at the end of a call may never be written.

Blaming the model for a transcription error. Check the user_transcript events before changing prompts.

Running verbose by default. It stores raw payloads, which adds volume and sensitivity.

Ignoring consent and retention. Voice data has rules of its own, so settle them before launch.

Never testing an outage. Prove that a ZizkaDB failure does not affect calls.

Where to go from here

Run the example worker in examples/livekit-agent/, place a test call, and open its session in the dashboard. Then wire the observer into your own agent, test the failure cases, and decide your verbosity and privacy settings. After that, the earlier guides on drift monitoring, point-in-time reconstruction, and audit preparation all apply to voice, since they build on the same record.

If you tell me your LiveKit setup, your STT and LLM providers, and what the agent is allowed to do on a call, I can help adapt these steps, including the privacy settings and the test plan.

Want to test ZizkaDB on your voice agent? Try our open source version here: https://github.com/ZIZKA-AI-SL/ZizkaDB

Interested in a design partnership? Fill in the form on our site or reach me directly at founder@zizka.ai

Sources: ZizkaDB LiveKit integration docs · ZizkaDB connection guide

How to Add an Audit Trail to a LangChain Agent in 15 Minutes with ZizkaDB

Mir Arshad Ali Talpur — Wed, 21 Oct 2026 09:10:36 +0000

This is the next how-to guide in the series. The earlier guides explained what each ZizkaDB function does and how to design for audits and oversight. This one is hands-on: take a LangChain agent, connect it to ZizkaDB, and get to the point where you can ask why the agent did something and see the chain. Every command below comes from the ZizkaDB connection guide, so you can copy it as is.

It is written for developers who already run a LangChain agent, or are about to, and want auditability from day one instead of after the first incident.

A note on scope: this guide uses the self-hosted open source stack on localhost. The managed cloud works with the same SDK, using a live API key instead of a host. Package versions below were current in the repository when I checked, so confirm them on PyPI before you pin anything.

Before you start: what you are building

The goal is a record where every step links to the step that caused it. ZizkaDB does this with an explicit parent_id on each event. A user message is the root, an LLM response points to that message, and a tool call points to the response. Walking back along those links is what why() does, and it is what separates an audit trail from a flat log.

By the end you will have:

A running ZizkaDB stack on your machine Your LangChain agent logging events through a callback handler Named tool calls, so the dashboard shows which tool ran A working why() query on a real agent step Step 1: Start the stack

You need Docker. The recommended path needs no repo clone, and the first image pull can take 5 to 10 minutes:

bash curl -fsSL https://raw.githubusercontent.com/Zizka-ai/ZizkaDB/main/scripts/quickstart-remote.sh | bash

If you would rather work from a clone:

bash git clone https://github.com/Zizka-ai/ZizkaDB.git && cd ZizkaDB bash scripts/quickstart.sh

When it is up, the API runs at http://localhost:8000 and the dashboard at http://localhost:3001/login. On localhost there is no signup and no API key, because a dev key is built in.

Step 2: Check that the stack works before touching your agent

Run the built-in demo first. If something fails here, it is a stack problem, not an agent problem:

bash pip install zizkadb-sdk zizkadb demo

You should see a small chain printed, with a tool call linked to an LLM response linked to a user message. If you get a connection refused error, the stack is not running, so repeat Step 1.

Step 3: Install the LangChain integration bash pip install "zizkadb-sdk>=0.3.0" "zizkadb-langchain>=0.1.3"

Optionally set defaults so you do not repeat them in code:

bash export ZIZKADB_HOST=http://localhost:8000 export ZIZKADB_AGENT=my-bot

Pick the agent name with care. The same name must be used in your code and in the dashboard, and it is how you will filter later.

Step 4: Attach the callback handler

This is the whole integration for a basic LangChain call:

python import asyncio from langchain_openai import ChatOpenAI from zizkadb import ZizkaDB from zizkadb_langchain import ZizkaDBCallbackHandler

async def main(): async with ZizkaDB(host="http://localhost:8000") as db: handler = ZizkaDBCallbackHandler(db=db, agent="my-bot") llm = ChatOpenAI(model="gpt-4o-mini") await llm.ainvoke("Hello", config={"callbacks": [handler]})

asyncio.run(main())

The handler goes in the callbacks list of the config you pass to LangChain. For a real agent, pass the same config to your agent executor or graph invocation, so every LLM and tool call inside the run goes through the handler.

If you are starting a new project, you can scaffold one instead:

bash zizkadb init my-agent --template langchain Step 5: Verify in the dashboard

Open the dashboard, go to Activity, and look for my-bot. It should appear within about 30 seconds. Click an event and open the Why? (causal) tab to see its chain.

The connection guide also offers two quick checks. In Settings, Send test event logs to a separate dashboard-connection-test agent, so it never pollutes your real data. And Settings with ?agent=my-bot has a Test agent action.

If nothing shows up, check these in order: the stack is running, the host URL matches, the agent name matches exactly, and the handler is actually in the callbacks of the call that runs.

Step 6: Ask why from the terminal

Pick an event ID from the dashboard and run:

bash zizkadb why

Or from Python:

python (await db.why(event_id)).print()

You should see the chain walking back from the step you chose to the user message that started it. Try it on a tool call or an odd answer. This is the moment the integration proves its value, because you are now explaining a real decision from your own agent, from a recorded chain instead of from memory.

Step 7: Name your tools so the data is useful

An event type like tool_call says that a tool ran. Naming it says which one. The connection guide states that the LangChain, LiveKit, and MCP integrations already log tool names this way. For tools or API calls you log yourself, use these helpers:

python call = await db.log_tool("support-bot", "lookup_order", args={"order_id": "ORD-8842"}) await db.log_tool_result("support-bot", "lookup_order", result=order, parent_id=call.event_id)

on failure:

await db.log_tool_result("support-bot", "lookup_order", error=exc, parent_id=call.event_id)

await db.log_api("support-bot", "POST", "https://api.stripe.com/v1/refunds", status=200, duration_ms=312)

Per the guide, this is what lets the dashboard, reports, and drift detection show per-tool call counts and error rates, and say things like refund calls up 12pp instead of only tool calls up. It also means log_api strips the query string, fragment, and credentials from URLs before logging, which matters for secrets. A 4xx or 5xx status counts toward that API’s error rate.

Pass parent_id whenever you log by hand. A missing parent link is the most common reason a why() chain looks incomplete.

Step 8: Cover parts the handler does not see

The callback handler sees what LangChain sees. Anything outside it, such as a database write, a call to an internal service, or a human approval, will not appear unless you log it. Use db.log() with an explicit parent_id for those:

python user = await db.log(agent="my-bot", event="user_message", data={"text": "Why is my order late?"}) await db.log(agent="my-bot", event="tool_call", data={"tool": "lookup_order"}, parent_id=user.event_id)

Check coverage with a simple test. Run one scripted conversation, open its session in the dashboard, and compare it with what you know the agent did. Any step you can name that is missing from the timeline is an instrumentation gap. Fix those before relying on the record for anything important.

Step 9: Keep secrets and personal data out of events

Logged content is stored. Before going further, decide what must never be logged: API keys, credentials, and unnecessary personal data. Redact before you log, since changing a stored event afterwards would break its integrity checksum. If you handle personal data, tag events consistently so erasure with db.forget() can find everything that belongs to one person, using a metadata filter.

For telemetry, the project collects an anonymous install ping from the MCP server, which you can opt out of with export ZIZKADB_TELEMETRY=false.

Step 10: Move toward production

Localhost uses an auto-injected dev key, which is not suitable for a server. The self-hosting steps in the guide for a VPS are:

In infra/.env, set ENV=production, DEPLOYMENT_MODE=self_hosted, and SELFHOST_ADMIN_TOKEN to a long random value. Run docker compose -f infra/docker-compose.yml up -d. Run bash infra/deploy-selfhost.sh. Open the dashboard, enter the admin token, and create API keys in Settings. Use those keys in your SDK and MCP configuration.

Then check the wiki pages on self-hosting and production deployment for backups, TLS, and upgrades, which this guide does not cover. If you would rather not run infrastructure, the managed cloud takes the same SDK with a live key.

What the business case looks like

These figures are modeled assumptions, not measured results. Replace them with your own.

Assume a team running one LangChain agent has 10 incidents a month at 100 euro per engineering hour. Without a causal record, each takes about 5 hours of reading scattered logs and guessing at sequence, costing 5,000 euro a month. With why() and replay on a connected trail, each takes about 1 hour, costing 1,000 euro a month. That saves roughly 4,000 euro a month.

The setup is a one-time cost. For a single LangChain agent, the handler attach plus a few hand-logged steps is plausibly a few hours, and the stack setup is under an hour apart from the image download. At 100 euro per hour, that is a few hundred euro, so the payback is measured in days if the incident volume is real.

The self-hosted edition is free under AGPL-3.0. The managed cloud is listed at 29 euro a month for Pro and 69 euro for Team, so choose based on whether you would rather pay for hosting or run it yourself.

The larger value arrives later, when a customer, an auditor, or your own risk team asks why the agent did something and you can show the chain instead of reconstructing it.

Common mistakes to avoid

Mismatched agent names. The name in code and in the dashboard must be identical, or events seem to vanish.

Forgetting the handler in the call that runs. The callback must be in the config of the invocation that executes the agent, not only on a spare instance.

Missing parent links on hand-logged events. Without parent_id, a why() chain stops early.

Logging secrets. Redact before logging, because stored events are checksummed.

Trusting an unchecked trail. Run one scripted conversation and compare the timeline to what really happened.

Using the dev setup in production. Replace the auto dev key with real keys and an admin token before exposing anything.

Where to go from here

Run the quickstart, attach the handler, and ask why on one real step. That is the whole integration, and it takes minutes. After that, add named tools and hand-logged steps for anything LangChain does not see, then work through the earlier guides for drift monitoring, point-in-time reconstruction, and audit preparation, since they all build on the same record.

If you tell me which agent framework and model provider you use, I can help adapt these steps and test the coverage of your trail.

Want to test ZizkaDB on your agent? Try our open source version here: https://github.com/ZIZKA-AI-SL/ZizkaDB

Interested in a design partnership? Fill in the form on our site or reach me directly at founder@zizka.ai

Sources: ZizkaDB connection guide (CONNECT.md) · ZizkaDB repository README

How to Track and Cut Your Cursor Token Costs with ZizkaDB and MCP

Mir Arshad Ali Talpur — Tue, 20 Oct 2026 08:15:03 +0000

This is the next how-to guide in the series. The earlier guides focused on auditability for banks and enterprise teams. This one is for Cursor users: solo developers and small teams who watch the usage meter climb and cannot say where the spend went.

The hook is MCP. Cursor supports Model Context Protocol servers, and ZizkaDB ships one, so connecting it takes about 30 seconds and requires no changes to how you work. Once it is connected, the same event record that gives enterprises an audit trail gives you a breakdown of token cost by model, session, and step, and a way to cut it based on your actual usage.

Part 1: Connect ZizkaDB to Cursor What you need Cursor, with MCP support. uv, which provides uvx. If you do not have it: bash curl -LsSf https://astral.sh/uv/install.sh | sh One of two ZizkaDB options: the managed cloud at db.zizka.ai, or the open source self-hosted stack. Self-hosting needs Docker. Option A: Self-hosted (open source, data stays on your machine)

Start the stack with the quickstart script. The first image pull may take 5 to 10 minutes.

bash curl -fsSL https://raw.githubusercontent.com/Zizka-ai/ZizkaDB/main/scripts/quickstart-remote.sh | bash

Prefer to clone the repo? Use this instead:

bash git clone https://github.com/Zizka-ai/ZizkaDB.git && cd ZizkaDB bash scripts/setup-local.sh

When it is running, the API is at http://localhost:8000 and the dashboard is at http://localhost:3001/login.

Now add this to ~/.cursor/mcp.json for all projects, or .cursor/mcp.json inside a single project:

json { “mcpServers”: { “zizkadb”: { “command”: “uvx”, “args”: [“zizkadb-mcp”], “env”: { “ZIZKADB_HOST”: “http://localhost:8000" } } } }

On localhost no API key is needed, because a dev key is injected automatically. If you changed DEV_API_KEY in infra/.env, the MCP server’s key must match it.

Option B: Managed cloud Sign up at db.zizka.ai. In the dashboard, create an agent (for example cursor-agent) and copy the API key. It is shown only once. Paste this into ~/.cursor/mcp.json or .cursor/mcp.json: json { “mcpServers”: { “zizkadb”: { “command”: “uvx”, “args”: [“zizkadb-mcp”], “env”: { “ZIZKADB_API_KEY”: “zizkadb_live_xxxx” } } } }

Two details from the README matter here. Signing up alone does not log anything, because you must create an agent and key and put the key in ZIZKADB_API_KEY. And the agent name must match: if your key belongs to cursor-agent, log events with agent=”cursor-agent”. If you use many agent names, create a tenant-wide key in Settings.

Reload and verify Reload MCP in Cursor, or restart it. Open Cursor’s MCP settings and confirm the zizkadb server shows as connected, with its tools listed. Run a short Agent prompt, then check the dashboard. Events should appear within seconds.

If you want to test ZizkaDB before involving Cursor, run pip install zizkadb-sdk && zizkadb demo. It logs a small example chain, so you can confirm the stack works first.

Two honest notes on what gets captured

MCP is called by the agent. The README refers to log_event calls, which means events reach ZizkaDB when Cursor’s agent calls the ZizkaDB tools. It is not a passive recorder of everything Cursor does. If your sessions are empty, check whether the agent is actually calling the tool, and consider adding a Cursor rule that tells it to log. The repository includes a .cursor folder and an AGENTS.md that show how the maintainers do this.

Token counts only exist if they are logged. Cost analysis depends on the model name and token counts appearing in the event data. Check one real event in the dashboard and confirm it has both. If counts are estimated or missing, treat any cost numbers as approximate, and compare them with Cursor’s own usage page.

Privacy

Self-hosted means prompts and code context stay in your own Postgres. On the managed cloud they go to db.zizka.ai. Either way, keep API keys and credentials out of prompts and rules files, because logged content is stored. You can opt out of the anonymous install ping with export ZIZKADB_TELEMETRY=false, which also works as an env entry in the MCP config.

Part 2: Cut your Cursor costs

The integration only takes minutes. The savings come from what you do with the data.

Step 1: Work normally for one week to set a baseline

Do not optimize on day one. At the end of the week, record total tokens and estimated cost, the split by model, the split of input versus output tokens, average cost per session, and your ten most expensive sessions. Without a baseline you cannot prove any change helped.

Step 2: Break the cost down by model

Rank models by spend, then ask what share of each model’s spend needed that model. A common Cursor pattern is leaving a premium model selected for everything, including renames, boilerplate, and explaining a function. Sort spend into three types: necessary (hard reasoning, big refactors), discretionary (a cheaper model would do), and wasteful (retries and loops).

Judge by cost per completed task, not cost per call. A cheaper model that needs three attempts can cost more than a premium model that succeeds once. Also check whether Max mode or large context settings are driving up the cost of each call.

Step 3: Find the outlier sessions

A few sessions usually account for a large share of spend. Open the top ten with Session Replay and look for these causes:

Context bloat: a long thread that resends a growing history on every turn. Heavy attachments: @-mentioning whole folders when a few functions were relevant. Agent retry loops: the agent repeats a failing test or command many times. Wandering sessions: an unclear goal that leads to dozens of exploratory steps. Verbose output: whole files regenerated for small changes. Rules and MCP overhead: long rules files and many MCP servers add instructions and tool descriptions to every request.

For expensive steps where the cause is not obvious, use zizkadb why or db.why() to see what inputs led to the call. It often reveals that a tool returned far more text than you expected.

Step 4: Separate input cost from output cost

If input dominates, your problem is context. If output dominates, the model is writing too much. Agent style coding is usually input heavy, so context management tends to be the biggest lever. Prompt caching can reduce the cost of repeated context on some models and plans, but verify that before counting on it.

Step 5: Apply one fix at a time Route by difficulty: a cheaper model for simple edits and questions, the premium model for hard reasoning. Start fresh chats when a task is done, with a short note carrying over any state you need. Narrow the context to specific files, functions, or line ranges. Trim your rules: keep them short and move rarely needed guidance out of always-on rules. Prune MCP servers you are not using in this project. Stop loops early: interrupt, fix the underlying error, or rephrase the task. Ask for diffs, not rewrites. Plan first with a cheap model on larger tasks, so the expensive one does not wander. Step 6: Re-measure, and watch quality too

After a week, compare against your baseline: total cost, cost per session, share of spend on the premium model, and number of retry loops. If a cheaper model causes more rework or failed tests, the saving is false, so track completed tasks and time spent fixing output as well.

Step 7: Use drift detection to catch cost regressions

Drift is useful for cost as well as safety. If a new rule, a newly added MCP server, or a model switch leads to more LLM calls per session or more retries, the drift view shows which events and transitions moved. A rise in llm_start and tool_start against your baseline is an early sign something is burning tokens. In the API this is db.baseline(). Drift shows that behavior changed, not whether it is good, so confirm with cost per session.

Step 8: Make it a ten minute weekly habit

Review the cost breakdown at a fixed time each week, set a soft budget, and look at the top three sessions whenever you exceed it. For small teams, review per project to share efficient habits, not to police anyone. Revisit your model routing rule monthly, since new models and price changes shift the trade-offs.

What the business case looks like

These figures are modeled assumptions, not measured results. Replace them with your own pricing and usage.

Take a team of 4 developers spending about 150 euro each per month, so 600 euro in total. Suppose the breakdown shows 30 percent of spend is discretionary work on a premium model and 10 percent is waste from loops and context bloat. Routing discretionary work to a cheaper model might cut that portion by half to two thirds, saving 10 to 20 percent of the total. Stopping loops and trimming context might remove half the waste, saving about 5 percent. Together that is 15 to 25 percent, or roughly 90 to 150 euro a month.

Time often matters more. If each developer loses 2 fewer hours a month to stalled or looping sessions, at 60 euro per hour, that is 480 euro a month across the team. For a solo developer on 150 euro a month, the same percentages mean 20 to 40 euro, which is modest, but you stop being surprised by the bill and learn which habits cost money.

Self-hosting is free under AGPL-3.0, and the MCP server is MIT licensed. The managed cloud is listed at 29 euro a month for Pro and 69 euro for Team, so at small usage the cloud fee can exceed the savings. For solo developers, self-hosting is usually the better fit.

Common mistakes to avoid

Skipping the verification step. An empty dashboard usually means the agent never called the tool, or the agent name does not match the key.

Optimizing without a baseline. You cannot prove a saving you did not measure.

Judging by cost per call. Cost per completed task is the number that matters.

Downgrading models blindly. A cheaper model that causes rework can cost more overall.

Changing five things at once. You will not know which change helped.

Logging secrets. Keep credentials out of prompts and rules, since logged content is stored.

Where to go from here

Paste the config, reload MCP, run one prompt, and confirm an event appears. That takes minutes. Then work normally for a week and open your ten most expensive sessions. The record you build for cost analysis is also the foundation for replay, lineage, and drift, so if your needs grow from cost control to auditability, nothing is wasted.

If you tell me which models you use in Cursor and what you spend per month, I can help you draft a model routing rule and estimate the savings for your case.

Want to test ZizkaDB with Cursor? Try our open source version here: https://github.com/ZIZKA-AI-SL/ZizkaDB

Interested in a design partnership? Fill in the form on our site or reach me directly at founder@zizka.ai

Sources: ZizkaDB repository README · zizkadb-mcp README

How to Design Human Oversight for AI Agents with ZizkaDB

Mir Arshad Ali Talpur — Mon, 19 Oct 2026 13:31:38 +0000

This is the next how-to guide in the series. The earlier guides covered individual functions, instrumentation, audit preparation, incident investigation, and selling to enterprises. This one covers a requirement that most teams get wrong in a quiet way: human oversight. Many teams put a person somewhere in the loop and consider the box ticked. Regulators, and your own risk team, will ask a harder question: could that person actually understand what the agent was doing, and could they have stopped it?

It is written for CTOs, heads of risk, and product leads at banks, insurers, and vertical AI companies who deploy agents that influence decisions about people or money.

A note on scope: ZizkaDB records what agents and reviewers do and lets you reconstruct it. The approval gates and escalation rules themselves live in your agent runtime and your operating procedures. This guide shows how to design both so they produce evidence.

Before you start: what oversight actually means

Article 14 of the EU AI Act, which applies to high risk systems, asks for oversight measures that let the people assigned to the task do things like the following:

Understand the system’s capabilities and limitations Monitor its operation and notice anomalies Stay aware of the tendency to over-trust automated output Interpret the output correctly Decide not to use the output, or override it Intervene or stop the system

Article 26 adds that deployers must assign oversight to people with the necessary competence, training, and authority. Whether your agent counts as high risk depends on its use case, so confirm that with legal counsel. The design below is good practice either way, because it is also what customers and auditors tend to ask for.

The key idea is that oversight is only real if it is informed, timely, and effective, and you can only prove those three properties if the review itself leaves a record.

Step 1: Classify actions by consequence

List what each agent can do and sort the actions into tiers:

Tier 1, reversible and low impact: drafting, summarizing, internal lookups. Sampling review is enough. Tier 2, customer visible or moderately consequential: sending a message, changing a limit, recommending a decision. Review before or shortly after, depending on risk. Tier 3, irreversible or high impact: moving money, declining credit, closing an account, filing a regulatory report. A human decision before the action executes.

Write the tiers down and get risk and legal to sign off. This table becomes the backbone of everything else.

Step 2: Choose the oversight mode per tier

Three modes are common, and it is worth being explicit about which applies where:

Approval before action. The agent proposes and a human approves. Use for Tier 3. Intervention window. The agent acts after a short delay unless a human stops it. Use where speed matters and actions can be cancelled. Sampling and after the fact review. Humans review a sample of completed actions. Use for Tier 1 and some Tier 2.

Avoid choosing approval everywhere. Reviewers who see hundreds of trivial requests stop reading them, which defeats the purpose.

Step 3: Make the approval gate a first class event

An approval is a decision, so record it as one. Your agent should emit events when it requests review and when the review is resolved, linked into the same causal chain as the action itself.

python

illustrative pseudocode, check the SDK docs for exact calls

def request_approval(proposal, reviewer_group): req_id = emit("decision", { "action": "approval_requested", "proposal": proposal.summary, "tier": 3, "reviewer_group": reviewer_group, }) return req_id

def resolve_approval(req_id, reviewer_id, outcome, reason): emit("decision", { "action": "approval_resolved", "reviewer": reviewer_id, "outcome": outcome, # approved | rejected | modified "reason": reason, "request_ref": req_id, })

Use the same fixed event types and put the specifics in the payload, so drift baselines stay stable. Capture who decided, what they decided, when, and any reason given. Where the reviewer changes the proposal, record both versions.

Make the gate fail safe: if the approval service is unavailable, a Tier 3 action should wait or be refused, not proceed silently.

Step 4: Give reviewers the context to decide

A reviewer who sees only a recommendation cannot judge it. For each review request, decide what the reviewer sees, and build it from the record:

The agent’s proposal and a short rationale The key inputs and retrieved context behind it, drawn from the causal chain The tool results that mattered Known limitations relevant to this case, such as low retrieval confidence or a tool that returned an error A link to the full session replay for anyone who wants to dig in

Keep the default view short and the depth one click away. The goal is informed review in minutes, not a forensic exercise for every case. Be careful about what you surface: show data the reviewer is entitled to see, and keep tenant and privacy boundaries intact.

Step 5: Assign oversight to named, trained roles

Define who reviews each tier, what training they need, and who has authority to pause or override the agent. Record role assignments with effective dates so you can later show who was responsible on a given day, which pairs naturally with at() for historical state.

Cover cover and escalation: what happens when the reviewer is unavailable, disagrees with the agent repeatedly, or sees something unusual. Write down who can stop an agent entirely and how that is done.

Step 6: Build a stop mechanism and test it

An ability to intervene means little if stopping the agent takes an engineering ticket. Provide a documented way to pause an agent, a tool, or a class of actions quickly, and make sure the pause itself emits an event with who triggered it and why.

Test it as a drill. Pick a scenario, trigger the stop, and measure how long it takes and whether the in-flight sessions end in a clean state. Keep the drill record as evidence.

Step 7: Measure whether oversight is working

Oversight can look healthy while being ineffective. A reviewer who approves 99.9 percent of requests in a few seconds each may be rubber stamping, or the agent may simply be excellent. You need data to tell them apart. Track per reviewer group and per action type:

Approval, rejection, and modification rates Time spent per review How often reviewers open the replay or deeper context Disagreements later proven right, such as an approved action that led to a complaint Queue age, meaning how long requests wait

Watch for warning patterns: near zero rejections combined with very short review times, reviews clustered at the end of shifts, or approval rates that jump after a prompt change. These are signals to investigate, not verdicts. Deliberately inject known test cases, including some the agent gets wrong, to check whether reviewers catch them.

Step 8: Connect oversight to drift monitoring

Oversight and drift watch different things. Reviewers see individual cases, while drift shows shifts in the overall pattern. Use them together:

Review drift after every prompt, model, or tool change, and flag transitions that should never disappear, such as a verification step before a Tier 3 action. When drift appears, temporarily tighten oversight: raise the sampling rate or move a Tier 2 action to approval until you understand the change. When reviewers report that the agent seems different, check the drift view for corroboration.

Remember that drift indicates change, not whether the change is good or bad. Use error rate, session length, and review outcomes alongside it.

Step 9: Close the loop on overrides

Every rejection or modification is a learning signal and a piece of evidence. For each one, review the session with replay, use why() to see what led the agent astray, and classify the cause: bad input, bad context, tool failure, or reasoning error. Feed recurring causes into prompt fixes, validation rules, or evaluation sets.

Keep a log of what changed as a result. Being able to show that human interventions led to concrete improvements is strong evidence that oversight is effective rather than ceremonial.

Step 10: Prepare the oversight evidence pack

When an auditor or customer asks how oversight works, you should be able to hand over a package without starting from scratch:

The action tiers and mode per tier (Steps 1 and 2) Role assignments, training records, and authority to stop the agent (Step 5) Sample review cases with replay and lineage, using synthetic or consented data (Steps 3 and 4) Stop mechanism drill records (Step 6) Reviewer effectiveness metrics and test case results (Step 7) Drift review records and the actions taken (Step 8) Examples of overrides and the resulting improvements (Step 9)

Verify the SHA-256 checksums on any events you include, so you can show the records are unchanged since they were written. Remember that checksums show integrity, not that a decision was correct. The pack supports your technical documentation and compliance reporting but does not replace your risk management system, training obligations, or any formal assessment.

What the business case looks like

These figures are modeled assumptions, not measured results. Replace them with your own.

Assume 500 escalated decisions a month reviewed by humans, at 60 euro per hour for a reviewer. Without connected context, a reviewer spends about 12 minutes per case gathering inputs from several systems, costing 6,000 euro a month. With the proposal, rationale, and replay assembled from the record, review takes about 4 minutes, costing about 2,000 euro a month. That saves roughly 4,000 euro a month, or 48,000 euro a year, and shortens customer wait times as well.

Add audit and customer evidence requests about oversight. Assume 4 requests a year, each needing 2 people for about 3 days to piece together who reviewed what and when, at 100 euro per hour, costing around 4,800 euro per request. With review events already recorded and linked, the same request takes about half a day, costing around 800 euro, a saving of roughly 16,000 euro a year.

The larger value is risk. An oversight process that cannot show it worked is exposed in every serious incident. Under the EU AI Act, breaches of most obligations can bring fines of up to 15 million euro or 3 percent of global annual turnover, whichever is higher, and records that were never captured cannot be recreated later.

Common mistakes to avoid

Putting a human in the loop and stopping there. Presence is not oversight. Reviewers need context, authority, and time.

Requiring approval for everything. Over-review trains people to click through. Reserve approval for actions where it matters.

Not recording the review. If the approval is not an event in the record, you cannot prove it happened or who made it.

Hiding the stop button. An intervention path that takes days to use will not help during an incident.

Never testing reviewers. Without test cases and effectiveness metrics, you cannot tell oversight from rubber stamping.

Treating the tool as the oversight. ZizkaDB supplies the evidence. People, procedures, and authority supply the oversight.

Where to go from here

Start with one agent and its Tier 3 actions. Define the gate, record the approvals, give reviewers a short context view, and run one stop drill. After a month, look at your review times and rejection rates, because they will show you where oversight is real and where it is only ceremony.

If you tell me your sector and the decisions your agents make, I can help tailor the tiers, the reviewer metrics, and the numbers to your case.

Want to test ZizkaDB on your agent? Try our open source version here: https://github.com/ZIZKA-AI-SL/ZizkaDB

Interested in a design partnership? Fill in the form on our site or reach me directly at founder@zizka.ai

How to Make Auditability a Sales Asset When You Build Agents for Enterprises

Mir Arshad Ali Talpur — Sat, 17 Oct 2026 17:54:12 +0000

This is the next how-to guide in the series. The earlier guides covered what each ZizkaDB function does and how to instrument an agent system. This one is for a different reader: the team that builds agents and sells them to enterprises. If you are a vertical AI company, a consultancy delivering agents to clients, or a product team inside a software vendor, your buyer is not only evaluating whether your agent works. Their security, risk, legal, and procurement teams are evaluating whether they can defend using it.

Most vendors treat that as an obstacle to get past. This guide treats it as something you can prepare, package, and use to close deals faster.

A note on scope: ZizkaDB provides the records, isolation, deployment options, and query functions. Customer-facing reports, contract language, and review processes are things you build around it, so adapt the steps to your own product and legal advice.

Before you start: what enterprise buyers actually ask
Across industries, enterprise reviews tend to converge on the same questions:

Can you show what your agent did for us, and why? (traceability and explainability)
Is our data isolated from your other customers? (tenant isolation)
Where does our data live, and can it stay in our region or our environment? (residency and deployment)
How long do you keep it, and can we have it deleted? (retention and erasure)
How do you know the agent has not changed behavior after an update? (change management and monitoring)
Can you prove the records have not been altered? (integrity)
What happens when something goes wrong? (incident response)
Every one of these maps to a capability covered earlier in this series. The work is in turning them into answers a buyer can verify rather than promises they must trust.

Step 1: Design for per-customer isolation from day one
Treat each enterprise customer as its own tenant with its own scoped API keys, its own records, and its own retention settings. Retrofitting isolation after you have ten customers in a shared pool is painful, and a single cross-tenant leak can end a relationship.

Check the details that reviewers probe: separate keys per customer and per environment, no shared credentials between agents, and tests that confirm one tenant cannot query another tenant’s events. Keep those tests and their results, since you will be asked for them.

Step 2: Offer deployment options that match buyer constraints
Your largest prospects, especially in banking, insurance, health, and the public sector, may refuse a design where their data flows through your shared infrastructure. ZizkaDB supports managed, self-hosted, and VPC deployments, so you can offer a menu instead of a single take it or leave it model.

A practical packaging pattern:

Standard tier: managed deployment with tenant isolation, for buyers with moderate requirements.
Regulated tier: deployment inside the customer’s own cloud account or VPC, for buyers with residency or security mandates.
Decide your supported options before the first negotiation, including who operates what and who is responsible for upgrades. Ambiguity there slows contracts more than technical gaps do.

Step 3: Make retention a per-customer setting
Different customers have different statutory and internal retention requirements, and some will want the record kept longer than you would choose yourself. Configure retention per tenant and document the defaults, the allowed ranges, and how a customer requests a change.

Put the same thought into end of contract. Define what happens to a customer’s records when they leave: export, retention for a stated period, then deletion. Buyers increasingly ask this in due diligence, and a clear answer reads as maturity.

Step 4: Prepare a data subject and deletion procedure
Your customers are data controllers or processors under GDPR, and their own users will exercise erasure rights. Agree in advance how a deletion request travels from the customer to you and how you execute it. forget() erases records across both event logs and vector stores, which is the part vendors usually get wrong, because embeddings generated from a person’s messages are still personal data.

Test the procedure on synthetic data, confirm the audit trail for other users remains intact, and keep the test record. Then write a one page description of the process that a customer’s privacy team can read in five minutes.

Step 5: Give customers a way to inspect what the agent did for them
This is the step that differentiates you most. Enterprise buyers do not only want assurance that you can investigate. They want to be able to verify, or at least to request evidence quickly. Using the dashboards, APIs, and SDKs, decide how each customer can see their own agent activity:

Read access to their own sessions and replays, through your product or through a request process
A defined turnaround for explanation requests on a specific decision, using why() and replay
Exports of their records in a standard format
Start with a request process if building a self-serve view is too much for now. A promise of a verified explanation within, say, two working days beats a vague offer of support. Scope what you expose carefully, since payloads may include your own prompts or other customers’ context, and tenant boundaries must hold in every view.

Step 6: Build a change management story around drift
Enterprises fear silent change. The question to prepare for is how they will know if the agent behaves differently after you update a prompt, swap a model, or your model provider updates theirs.

Use behavioral baselines and drift monitoring to build a concrete answer:

Establish a baseline per customer deployment where volume allows.
Review drift after every prompt change, model upgrade, or tool change, and weekly otherwise.
Set thresholds that trigger review, including specific transitions that must not disappear, such as a verification step before a payment action.
Record each review, even when nothing is found.
Then decide what you share. A short monthly behavior summary for each enterprise customer, covering whether behavior shifted, whether error rate and session length moved, and what you did about it, turns a monitoring feature into a retention tool. Be honest that drift indicates change, not necessarily a problem, and pair it with health metrics when you report it.

Step 7: Be precise about integrity claims
Every event carries a SHA-256 checksum, so you can demonstrate that a stored record is unchanged since it was written. State the claim exactly. Checksums make tampering detectable, they do not make it impossible, and they do not prove the logged content was correct at the time.

Support the claim with the surrounding controls from Step 1, restricted administrative access, and, for stricter customers, periodic export of checksums to storage your own administrators cannot modify. Overstating integrity guarantees is a common way to lose credibility in a technical security review.

Step 8: Write the incident response commitment
Define what you will do and by when when a customer reports an agent incident. Use the investigation process from the earlier guide as the backbone: check drift, reconstruct with replay, explain with lineage, confirm historical state with at(), verify checksums, and deliver a case file.

Turn that into commitments you can sign: acknowledgment time, time to a first evidence-based explanation, and time to a written root cause. Do not promise times you have not tested. Run a drill on a past incident and see what you can actually deliver.

Step 9: Assemble a trust pack
Combine the previous steps into one reusable package that your sales team can send at the start of a security review instead of at the end:

Architecture and deployment options (Step 2)
Tenant isolation description and test summary (Step 1)
Retention and end of contract policy (Step 3)
Deletion procedure and test record (Step 4)
Customer evidence access and explanation turnaround (Step 5)
Change management and drift review process, with a sample monthly summary (Step 6)
Integrity approach and its limits (Step 7)
Incident response commitments (Step 8)
A sample replay and lineage walkthrough on synthetic data
Have your legal counsel review the claims. The pack supports your answers to questionnaires, but it does not replace contractual terms, certifications such as ISO 27001 or SOC 2 where a buyer requires them, or any formal assessment the customer must perform.

Step 10: Demo it, then pilot it
A live walk through is persuasive in a way a document is not. Prepare a ten minute demo using synthetic data that goes from a customer complaint to the replay, to the lineage explanation, to the point-in-time configuration, to checksum verification.

For larger deals, propose a pilot where the customer’s own risk or audit team tries to break your answers: ask for the explanation behind a decision of their choosing and measure how long it takes you. A buyer who has watched you answer their own question has less left to worry about.

What the business case looks like
These figures are modeled assumptions, not measured results. Replace them with your own.

Assume you close 10 enterprise deals a year, and unclear answers on traceability, isolation, deletion, and change management add about 3 weeks to each security and legal review. That is 30 weeks of delay across the year. A prepared trust pack and demo that cut the average by 1 to 2 weeks pulls 10 to 20 weeks of revenue forward. If your average annual contract value is 100,000 euro, a deal that starts 2 weeks earlier is worth roughly 3,800 euro of earlier revenue, so 10 such deals are worth about 38,000 euro in timing alone. The more important effect is on deals that stall completely, where one reassuring answer can decide between a pilot that dies and a contract that closes.

Add the effort side. Without a pack, each review consumes perhaps 40 hours of engineering and leadership time answering questionnaires one by one, costing 4,000 euro at 100 euro per hour, or 40,000 euro across 10 deals. With a maintained pack, that can fall to about 15 hours per deal, saving roughly 25,000 euro a year.

Add retention. Customers who receive a monthly behavior summary and can request verified explanations are more likely to renew, because they can see the controls working. That effect is real but varies widely, so measure renewal rates before and after rather than assuming.

Common mistakes to avoid
Answering questionnaires from scratch every time. Build the pack once and keep it current.

Promising what you have not tested. Turnaround times, deletion, and isolation claims should each be backed by a drill or a test record.

Sharing evidence across tenant boundaries. A single cross-customer leak in a replay or export undermines everything else.

Overstating integrity or compliance. Say what the controls do and what they do not. Reviewers trust precise claims more than broad ones.

Treating auditability as an engineering project only. Sales, legal, and customer success all need to know the process, or the capability never reaches the buyer.

Starting after the first big deal is in review. Preparation is cheapest before a specific customer is waiting on it.

Where to go from here
Start by listing the last five security questionnaires you received and marking the questions you could not answer confidently. Those gaps tell you which steps to do first. Then build the trust pack incrementally, and use each new enterprise review to improve it.

If you tell me your sector, your deployment model, and how many enterprise deals you have in review, I can help tailor the trust pack and the numbers to your situation.

Want to test ZizkaDB on your agent? Try our open source version here: https://github.com/ZIZKA-AI-SL/ZizkaDB

Interested in a design partnership? Fill in the form on our site or reach me directly at founder@zizka.ai

How to Investigate an Agent Incident End to End with ZizkaDB

Mir Arshad Ali Talpur — Tue, 13 Oct 2026 12:39:54 +0000

This is the next how-to guide in the series. The earlier guides covered individual functions, such as lineage, replay, drift, at(), forget(), SHA-256 checksums, and audit preparation. This one walks through a single scenario every team running agents eventually faces: something went wrong in production, and you need to find out what happened, why, and what to do about it. It is written for engineering leads, on-call engineers, and heads of risk at vertical AI companies and banks who want a repeatable process instead of a scramble.

Before you start: what a good investigation produces
A good incident investigation answers five questions, in this order:

Did something actually change, or is this a one-off? (detection)
What exactly happened in the affected session? (reconstruction)
Why did the agent take the step that went wrong? (causation)
What did the system look like at the time? (context)
Can you prove the record is intact and hand it to someone else? (evidence)
Most teams jump straight to causation, guess, and fix the wrong thing. The steps below keep the order honest.

Step 1: Define the incident in one sentence
Write down what was reported, by whom, and when. For example: a customer says the agent quoted the wrong refund amount on a specific date. Note the session ID if you have it, or the customer and time window if you do not.

Resist the urge to write a theory yet. At this stage you only record the symptom.

Step 2: Check whether behavior has drifted
Open the Agent Behavior view and compare recent sessions against the baseline. You are looking for two things: whether the overall drift score has moved, and whether any specific events or transitions shifted around the time of the incident, for example a verification step being skipped more often than before.

Read the drift score alongside error rate and session length. Drift tells you something changed, not whether it is good or bad. Two outcomes are common. If a relevant shift lines up with the incident, you may be dealing with a systemic issue, and the ranked deltas give you a first hypothesis. If nothing moved, the incident is more likely a one-off or an edge case, which changes how urgent the response is.

Step 3: Reconstruct the session with Session Replay
Open the affected session and step through it in order, from the first user message to the final response. Do not stop at the step where the problem became visible. Many failures begin two or three steps earlier, such as a tool returning a stale result that a later decision relied on.

Mark the first step where the session diverges from what should have happened. That is the step you will investigate in the next stage.

Step 4: Explain the step with causal lineage
Run why() on the step you marked. It returns the chain of inputs, retrieved context, tool results, and decision points that produced it. Check each link:

Was the user input interpreted correctly?
Was the retrieved context accurate and current?
Did the tool return what it should have?
Did the decision follow reasonably from those inputs?
This usually narrows the cause to one of a few categories: bad input, bad context, a failing tool, or a reasoning error. Write down which one, and the evidence for it.

Step 5: Confirm the system state with at()
Query the agent’s state at the time of the incident: which prompt version, tools, and model were live. This protects you from a common mistake, debugging against the current system instead of the system as it was. If a prompt was changed, a tool swapped, or a model upgraded since the incident, the current configuration may not reproduce the problem.

Compare the historical state with the current one and note every difference that could plausibly matter.

Step 6: Check scope
One bad session is rarely the whole story. Using what you learned in Steps 4 and 5, look for other sessions that share the same cause: same tool, same prompt version, same transition pattern. Decide whether this affected one customer, a segment, or everyone, and whether any other customers need to be told.

Be honest about the limits of what you found. If you can only confirm the cause in some sessions, say so in the report instead of generalizing.

Step 7: Verify the integrity of the evidence
Before you share anything outside the team, verify the SHA-256 checksums on the events involved. If they match, the records are exactly as written. If any do not, stop and investigate why before relying on that evidence.

Remember what this does and does not show. Checksums show the record has not changed since it was written. They do not prove the logged content was correct at the time.

Step 8: Decide on the fix
Match the fix to the cause you found:

Bad input or context: improve validation or retrieval, and add checks before the agent acts.
Failing tool: fix or replace the tool, and add a fallback so the agent does not route around it silently.
Reasoning error: adjust the prompt or add a human review step for this class of decision.
Change that caused drift: consider rolling back, then reintroduce it with safeguards.
After shipping the fix, watch drift and error rate for the following days to confirm the shift reverses and nothing new appears.

Step 9: Write the case file
Capture the investigation in a short, standard format so anyone can read it later:

Symptom and timeline
Replay findings, including the first divergent step
Lineage explanation and the cause category
System state at the time, and relevant differences from now
Scope and affected customers
Checksum verification result
Fix, owner, and follow-up date
Where the incident involves a customer dispute or a regulated decision, this file is what you hand to the complaints team, auditor, or ombudsman.

Step 10: Learn from it
Add the failing session to your evaluation set so the same failure is caught before release next time. Decide whether a drift alert threshold, a review rule, or a monitoring check should change. Review the process itself: where did the investigation take longest, and what evidence was missing?

What the business case looks like
These figures are modeled assumptions, not measured results. Replace them with your own.

Assume 20 agent related incidents a month at 100 euro per engineering hour. Without a structured process and connected tooling, each takes about 7 hours, mostly spent reconstructing what happened from scattered logs, costing 14,000 euro a month. With drift detection, replay, lineage, and point in time state in one place, each takes about 1.5 hours, costing 3,000 euro a month. That saves roughly 11,000 euro a month, or 132,000 euro a year.

Add the cost of misdiagnosis. If 1 in 5 investigations leads to fixing the wrong thing and costs about 3 hours of rework, that is 4 cases a month, around 1,200 euro. Debugging against the real historical state and the real causal chain removes most of it.

The larger value is usually outside engineering: faster answers for customers, fewer disputes that drag on, and evidence you can hand over with confidence.

Common mistakes to avoid
Jumping to causation. Skipping reconstruction means you explain a step you have not fully seen.

Debugging the current system. The configuration that caused the incident may no longer exist, so check the historical state.

Fixing one session. If you do not check scope, the same cause keeps producing incidents.

Sharing unverified evidence. Verify integrity before the record leaves the team.

Skipping the case file. Without it, the same investigation gets repeated from scratch next time.

Where to go from here
Start by running this process on your next real incident, and time each step. The steps that take longest show you where your current tooling and procedures are weakest. After three or four incidents, you will have a baseline of your own, which is worth more than any modeled number.

If you tell me your sector and roughly how many incidents you handle a month, I can tailor this process and the numbers to your case.

Want to test ZizkaDB on your agent? Try our open source version here: https://github.com/ZIZKA-AI-SL/ZizkaDB

Interested in a design partnership? Fill in the form on our site or reach me directly at founder@zizka.ai

The article is originally published in Medium and can be viewed here : https://medium.com/@MirArshadTalpur/how-to-investigate-an-agent-incident-end-to-end-with-zizkadb-28c2f328f05f?postPublishedType=initial

How to Prepare Your Agentic AI System for an EU AI Act Audit with ZizkaDB

Mir Arshad Ali Talpur — Mon, 12 Oct 2026 11:39:09 +0000

This is the next how-to guide in the series. The earlier guides covered individual functions, such as lineage, replay, drift, at(), forget(), and SHA-256 checksums. This one puts them together around a single goal: being ready when an auditor, regulator, or enterprise customer asks you to prove how your AI system behaves. It is written for CTOs, heads of compliance, and vertical AI teams who would rather prepare now than scramble later.

Before you start: what an audit will actually ask
Auditors rarely ask whether you have an AI system that works. They ask for evidence. In practice the questions fall into five groups:

What did the system do, and can you show the record? (traceability and logging, Article 12)
Can you explain why it did it? (transparency toward deployers, Article 13)
Was a human able to oversee and intervene? (human oversight, Article 14)
Can you show the system’s behavior over time, and how you monitor it? (post market monitoring, Articles 72 and 79)
Is the record reliable, secure, and handled lawfully? (Article 15 and GDPR)
Whether your system falls under the high risk rules depends on its use case, for example creditworthiness assessment. Confirm that classification with legal counsel. The steps below are useful either way, because the evidence they produce is what most audits and customer security reviews ask for.

Step 1: Inventory your agents and classify them
List every agent in production or planned, with its purpose, the data it touches, and who is affected by its decisions. Mark which ones may be high risk under the Act. Start the rest of this guide with the highest risk agent first, not all of them at once.

Step 2: Turn on continuous event logging
Connect the agent to ZizkaDB so every event is captured as it happens: user messages, decisions, LLM calls, tool invocations, and responses. This is the foundation for Article 12 style traceability.

Check completeness before moving on. Run test sessions and confirm each one can be reconstructed from the first message to the final response. Gaps in capture become gaps in your evidence, and they cannot be filled in later.

Step 3: Set retention per tenant
Decide how long each category of record must be kept, based on legal requirements and your own policy, and configure retention per tenant. Do this before the first audit, not during it. Auditors often look back six months or more, so a retention period that is too short quietly destroys evidence you may need.

Step 4: Secure the record
Evidence is only useful if it is trustworthy. Set up tenant isolation and scoped API keys so each agent and team can only reach its own records. Every ZizkaDB event carries a SHA-256 checksum, so you can show that a record is unchanged since it was written. Where your requirements are strict, choose self hosted or VPC deployment, and consider exporting checksums to storage your administrators cannot modify.

Be precise about the claim you make. Checksums make tampering detectable, not impossible, and they work best alongside access control and monitoring.

Step 5: Build the explanation workflow
For any contested or sampled decision, your team should be able to follow the same short procedure:

Open the session with Session Replay and review the full sequence in order.
Use why() on the key step to see the inputs, retrieved context, tool results, and decision points behind it.
Use at() to confirm which prompt, tools, and model were live on that date.
Verify the checksums on the events involved.
Save the findings in a case file.
Write this as a standard procedure so any trained analyst can run it, not only the engineer who built the agent.

Step 6: Define human oversight
Article 14 is about humans being able to understand and intervene, not just about having a person somewhere in the loop. Decide for each agent which decisions require human review, who reviews them, and what they see when they do. ZizkaDB lets reviewers inspect full action chains and system state at any point in time, so interventions rest on evidence rather than screenshots or manual notes. Document who has authority to pause or override the agent.

Step 7: Set up behavioral monitoring
Let the agent run long enough to build a baseline, then use drift detection to compare recent behavior against it. Decide who reviews drift alerts and how often, for example after every prompt change or model upgrade plus a weekly check. Keep the records of these reviews, since they are direct evidence of ongoing monitoring.

Remember that drift tells you something changed, not whether it is good or bad. Pair it with error rate and session length, then use replay and lineage to investigate flagged changes.

Step 8: Prepare for erasure requests
GDPR applies to every agentic system that touches personal data. Define how erasure requests are received and verified, and test forget() on synthetic data first. Confirm that deletion covers both event logs and vector stores, and that the audit trail for other users stays intact afterwards.

Step 9: Build an evidence pack template
Create a reusable template so an audit request does not start from a blank page. It should include:

The agent inventory and risk classification from Step 1
Retention and security configuration from Steps 3 and 4
Sample session replays and lineage reports for representative decisions
Point in time configuration records from at()
Drift review records from Step 7
Erasure procedure and test results from Step 8
Be clear internally about what this pack is. It is evidence that supports your technical documentation and compliance reporting. It does not replace your formal risk management system, quality management, or any conformity assessment by a notified body.

Step 10: Run a mock audit
Before a real request arrives, pick a date from three months ago and a decision from that date, then ask a colleague outside the build team to play the auditor. Can they get from the question to a complete, verified answer using only the procedure and the pack? Note where it took too long or where evidence was missing, fix those gaps, and repeat each quarter.

What the business case looks like
These figures are modeled assumptions, not measured results. Replace them with your own.

Assume 4 audit or customer evidence requests a year, each needing 2 engineers and 1 compliance person at about 100 euro per hour. Reconstructing evidence by hand from scattered logs, deployment history, and memory takes around 5 working days, costing about 12,000 euro per request, or 48,000 euro a year. With a prepared pack and direct queries, it takes about 1 day, costing about 2,400 euro per request, or roughly 9,600 euro a year. That is a saving of around 38,000 euro annually.

Add enterprise sales. If 10 deals a year stall for about 2 weeks on questions about logging, oversight, and data handling, a prepared and demonstrable answer can pull a meaningful share of that time back. Earlier revenue usually matters more than the labor saved.

The larger number is exposure. Under the EU AI Act, breaches of most obligations can bring fines of up to 15 million euro or 3 percent of global annual turnover, whichever is higher. Logs either existed at the time or they did not, and that cannot be fixed after the fact.

Common mistakes to avoid
Starting logging late. Evidence for the period before you instrumented the agent does not exist.

Treating the tool as the compliance program. ZizkaDB supplies evidence. You still need governance, risk management, and legal review.

Skipping the procedures. If nobody owns the explanation workflow or the evidence pack, the data sits unused when it matters.

Never testing. A mock audit finds gaps cheaply. A real one finds them expensively.

Where to go from here
Auditability is far cheaper to build before the questions arrive than after. Start with one agent, get the logging, security, and explanation workflow right, then expand.

If you tell me your sector and how many agents you run, I can tailor the evidence pack and the numbers to your case.

Want to test ZizkaDB on your agent? Try our open source version here: https://github.com/ZIZKA-AI-SL/ZizkaDB

Interested in a design partnership? Fill in the form on our site or reach me directly at founder@zizka.ai

Opensource Contribution Needed

Mir Arshad Ali Talpur — Sun, 11 Oct 2026 06:52:33 +0000

Hello all,

ZizkaDB is growing quickly as an open-source project, and we are looking for more contributors with strong community skills. We would love to have your contributions, and potentially welcome you as a maintainer later on.

How to Use SHA-256 Checksums to Prove Your Audit Trail Has Not Been Tampered With

Mir Arshad Ali Talpur — Fri, 09 Oct 2026 14:37:25 +0000

This guide walks through how to use the tamper evident checksum built into ZizkaDB, based on SHA-256, in the order a CIO, head of risk, or vertical AI team would actually do it. The checksum sits underneath lineage, replay, and point in time retrieval, because all three assume the record is true. These steps are how you prove it.

Before you start: what SHA-256 is
SHA-256 is a cryptographic hash function. You give it any piece of data, a sentence, a log entry, an entire file, and it returns a fixed length fingerprint, 64 characters long, that represents that data.

Three properties make it useful for auditing.

It is deterministic. The same input always produces the same fingerprint. Anyone, at any time, can recompute it and get the same answer.

It is extremely sensitive. Change a single character in the input, one digit in a transaction amount or one word in a decision, and the fingerprint changes completely.

It is one way. You cannot work backwards from the fingerprint to the original data, and in practice you cannot craft a different input that produces the same fingerprint. SHA-256 is part of the SHA-2 family standardized by NIST and is widely used in banking, software signing, and certificate infrastructure.

Think of it as a tamper seal on a package. The seal does not stop anyone from opening the package, but if someone does, you can see it.

Step 1: Decide what needs to be provable
A log is only worth as much as your ability to trust it. An auditor, regulator, or court asks not only what the record says, but whether it could have been changed since.

List the situations where your records may be questioned: customer disputes, supervisory reviews, security questionnaires from enterprise buyers, internal investigations. For agentic AI this matters more than usual, because if an agent made a contested decision, the log of that decision is the evidence. This list becomes your acceptance criteria for the steps below.

Step 2: Enable continuous event logging
Connect your agent to ZizkaDB so every event is captured as it happens: user messages, decisions, LLM calls, tool invocations, and responses. Every event recorded gets a SHA-256 checksum computed from its content, stored alongside the event.

Check completeness before moving on. Run test sessions and confirm each one can be reconstructed from first message to final response. Checksums protect what is recorded, so gaps in capture are gaps in protection.

Step 3: Lock down the surrounding controls
A checksum makes tampering detectable, not impossible. That is what tamper evident means. Prevention comes from the controls around it, so set these up now:

Tenant isolation, so each team or business line only accesses its own records
Scoped API keys, so each agent can only write what it should
Restricted write and administrative access to the database
Self hosted or VPC deployment where data residency or security requirements demand it
The strength of the guarantee depends on how checksums are stored and protected. If an attacker could rewrite both an event and its checksum, a standalone checksum would not catch it.

Step 4: Add an external layer for stricter requirements
If your requirements are strict, as they often are in banking, add protection the database administrator cannot override. Options include periodically exporting checksums to storage that the administrator cannot modify, or anchoring checksums externally on a schedule. Treat the checksum as one strong layer, not the only layer.

Step 5: Verify integrity on a schedule
Verification is simple: recompute the checksum from the stored event and compare it with the one on record. If they match, the event is exactly as it was when written. If they do not, something changed, and you know which event.

Do not wait for an incident. Run verification routinely, for example weekly, and assign an owner for reviewing any mismatch. Not every integrity problem is malicious. Bad migrations, storage faults, and buggy scripts also alter data, and checksums flag them regardless of cause.

Step 6: Build a procedure for when someone questions a record
Write a standard procedure so any analyst can follow it, not only the engineer who built the system:

Pull the session or events in question.
Run checksum verification on them.
Record the result in the case file.
Hand the evidence, with the verification result, to the auditor, ombudsman, or counsel.
This turns “trust us” into “here is the verification result.”

Step 7: Prepare your answer for security reviews
Enterprise buyers in banking, insurance, and health ask how you protect the integrity of audit data. Prepare a short answer that names the standard, explains that every event is checksummed, describes the surrounding controls from Steps 3 and 4, and offers a demo of verification. A concrete answer built on an algorithm they already trust shortens the conversation.

Step 8: Map it to regulation, with the right expectations
The EU AI Act expects high risk systems to maintain logs that support traceability and post market monitoring (Article 12), and expects technical robustness and cybersecurity (Article 15). In banking, DORA and supervisory record keeping expectations point the same way. None of these name SHA-256 specifically, and a checksum alone does not make a system compliant. It gives you a concrete, verifiable control supporting the claim that your logs are reliable, which is the claim regulators are testing.

Step 9: Run a pilot and measure
Run verification alongside your existing processes and measure before and after:

Time to establish that a questioned record can be trusted
Time spent answering integrity questions in security reviews
Time spent on manual spot checks for tampering
If nothing moves, this is mostly insurance for you right now, and that is fine.

What the business case looks like
These figures are modeled assumptions, not measured results. Replace them with your own.

Assume 10 situations a year where record integrity is questioned, at 100 euro per hour. Establishing trust manually, through access logs, change history, and written attestations, takes about 3 days each, costing around 24,000 euro a year. With checksum verification it takes about an hour each, costing about 1,000 euro a year.

Assume 10 enterprise deals a year where integrity comes up, and an unclear answer adds about 3 days of back and forth per deal. That is roughly 24,000 euro in effort and, more importantly, delay. A direct answer removes most of it.

Manual spot checks for tampering might take 2 days a quarter, around 6,400 euro a year. Automated verification reduces that to a small review cost.

Together that is in the range of 50,000 euro a year for a mid sized team. The larger value is that a record you can prove is intact is worth far more in a dispute than one you can only claim is intact.

What checksums do not do
They do not stop deletion of whole records unless you also track sequence or counts. They do not prove the logged content was true when written, only that it has not changed since. They do not replace access control, backups, or monitoring. They are one control in a layered design.

Why ZizkaDB uses SHA-256
It is a standard that auditors and security teams already trust, so nobody has to evaluate a custom invention. It is cheap, adding negligible overhead per event. And it is independently verifiable: anyone with the stored event can recompute the checksum with open tooling, which fits ZizkaDB’s open source and self hosted approach, so you do not have to trust the vendor to verify the record.

Where to go from here
The whole premise of auditability is that someone else can rely on your record. Lineage explains why an agent acted, replay shows what happened, and at() shows what the system looked like. SHA-256 checksums are the quiet foundation that lets you say, with evidence, that the record is exactly what was written at the time.

If you tell me your vertical and what integrity or audit requirements your customers ask about, I can tailor these numbers to your case.

Want to test ZizkaDB on your agent? Try our open source version here: https://github.com/ZIZKA-AI-SL/ZizkaDB

Interested in a design partnership? Fill in the form on our site or reach me directly at founder@zizka.ai
The article was originally published in medium and can be viewed here:https://medium.com/@MirArshadTalpur/how-to-use-sha-256-checksums-to-prove-your-audit-trail-has-not-been-tampered-with-b58ed34f9721?postPublishedType=initial

DEV Community: Mir Arshad Ali Talpur

How to Give Your AI Agent Memory You Can Audit

illustrative, check the SDK reference for exact parameters

illustrative

illustrative

make the next LLM call a child of retrieval

illustrative

Meet the LLM Accountant: The Role Nobody Has Hired Yet

illustrative ledger entry data, one per LLM call

How to Detect and Investigate Agent Drift with ZizkaDB

How to Use ZizkaDB with LiveKit to Audit Your Voice Agents

Attach before session.start() so no turn is missed.

How to Add an Audit Trail to a LangChain Agent in 15 Minutes with ZizkaDB

How to Track and Cut Your Cursor Token Costs with ZizkaDB and MCP

How to Design Human Oversight for AI Agents with ZizkaDB

How to Make Auditability a Sales Asset When You Build Agents for Enterprises

How to Investigate an Agent Incident End to End with ZizkaDB

How to Prepare Your Agentic AI System for an EU AI Act Audit with ZizkaDB

Opensource Contribution Needed

How to Use SHA-256 Checksums to Prove Your Audit Trail Has Not Been Tampered With

make the next LLM call a child of `retrieval`