A law firm runs a set of AI agents over its case files. One agent takes in new documents. A second agent reads each file and adds notes. A third agent redacts, exports, and disposes of files when a matter closes.
One morning the review agent deletes a file. The file is a deposition transcript in an active case: Halvorsen v. Meridian Logistics. The review agent is only allowed to read and annotate. It is not allowed to delete anything.
The delete still happens.
Weeks later an auditor asks a simple question: prove that the deletion of this file was authorized. The firm opens its activity log. The log has one line for the event:
records:delete agent=review file=halvorsen-deposition 2026-08-06T09:51:59Z
The line says what happened. It does not say whether the agent was allowed to do it. So the firm cannot answer the auditor. The log records the action, but it does not record the authority behind the action. An agent audit trail must record more than events.
Why a normal log fails
Most application logs answer one question: what happened? They record an actor, an action, a target, and a time. For human users that is often enough, because a human signs in once and carries a session. The permission check happened at the door.
AI agents break this model. An agent acts many times inside one run. It calls tools, it reaches other agents, and it acts on data on behalf of a person who is not in the loop for each step. The permission that matters is not "did someone sign in." It is "was this specific action, by this specific agent, allowed at the moment it ran."
A normal log cannot answer that, for three reasons:
- It records the action, not the decision. There is no field that says "allowed" or "denied," and no field for the rule that was checked.
- It records an actor name, not a verified identity.
agent=reviewis a label the application wrote. It is not proof that the caller was the review agent and held the right permissions. - It cannot tell an allowed action from a denied one. If the review agent's illegal delete and a lawful delete by the disposition agent both produce the same line, the log has erased the one difference that matters.
Look again at the two log lines below. One delete was allowed. One was not. The log cannot tell you which is which.
records:delete agent=disposition file=cormorant-memo 2026-08-06T09:51:59Z
records:delete agent=review file=halvorsen-deposition 2026-08-06T09:51:59Z
The disposition agent is allowed to delete. The review agent is not. But the two lines have the same shape and the same fields. Neither line carries the permission that was in force. To an auditor, both deletions are equally unprovable.
The diagram below shows the blind log at the center of the problem. Two deletes go in. The log flattens them into the same shape, and the authority is gone.
What the auditor actually wants
An auditor wants a record that answers four questions for every sensitive action:
- Who took the action. This is a verified identity, not a label.
- On whose authority. This is the person or agent the action was performed for.
- Under what permission. These are the exact rights that were in force, and the decision that was made.
- Can you prove the record was not changed. This shows the row is the original, not edited after the fact.
A normal log answers none of these with confidence. It gives a name, an action, and a time. The rest is missing. So the honest answer to the auditor is "we cannot prove it," which is the answer no regulated business can give.
Three fixes that do not work
Teams reach for the same three fixes when they hit this problem. Each one seems reasonable. Each one fails.
Fix 1: log more fields. Add the agent's role, the file's status, and a note to each line. This adds detail, but it does not add proof. The application still writes the fields itself, so the log still says agent=review without checking that the caller really was the review agent or that the review agent held delete rights. A richer log of unverified claims is still unverified.
Fix 2: check the permission once, at the start of the run. Verify the human's role when the run begins, then let the agents act. This fails because the agents do different work with different rights. The review agent should never delete, even in a run a partner started. One check at the door cannot bind each action inside the run.
Fix 3: trust the caller's claim. Let each agent tell the system what it is allowed to do, and record that. This is the worst fix, because it lets any agent widen its own authority by claiming more. The record then documents a lie. An auditor trusts a record that the actor could not forge, not one the actor wrote about itself.
The pattern across all three is the same. They try to describe the action better. The real fix is different: record the decision, made by something the agent cannot control, and make the record impossible to change without detection.
The principle
An agent audit trail must record the authority, not just the action. For every sensitive action it must capture:
- the verified identity of the agent that called,
- the person or agent the action was performed for,
- the exact permissions in force at that moment,
- the decision, allowed or denied, with the reason,
- and a value that proves the row was not edited later.
When the record carries all of this, the auditor's question has a real answer. "Prove this deletion was authorized" becomes "here is the row: the disposition agent held records:delete, the decision was allow, and the hash chain shows the row is unchanged." The review agent's illegal delete becomes a different row: decision denied, reason insufficient scope, action not performed.
The rest of this article builds that record with the real code from a working demo. The demo is small, it runs locally, and it ships an end-to-end script that proves the whole story headless.
The fix: record the decision, not the claim
The fix has one core idea. Before the system performs a sensitive action, it asks a separate authority to make a decision. That authority verifies the caller, checks the exact permissions in force, and returns allow or deny. The system then writes one row that captures the decision, the verified identity, and the permissions. It performs the action only on allow.
The demo uses four tools, and each one owns one job. This split matters, so here is the honest version of who does what:
- Kinde issues and verifies each agent's identity and scopes. Kinde does not ship the audit layer.
- The kinde-convex-agent-auth component makes the authorization decision and returns the result. It keeps its own append-only decision audit.
- Convex stores the replayable, hash-chained record the app builds from each decision, and it drives the user interface.
-
Langfuse traces how the run executed: spans, timing, tool calls. It is not the audit record. A run's
correlationIdlinks its Langfuse trace to its authority rows.
Keep these apart. The app builds the authority record from the component's decision and stores it in Convex. Kinde is the identity layer. The trace is not the authority record. Each tool answers one question, and no tool answers another's.
The authorization call
Every sensitive action runs through one call: authorize. The application passes the agent's token, the run identifier, the action, and the target. The component does three things:
- It verifies the token. It reads the scopes the token carries and resolves the caller to a registered agent.
- It works out the effective scopes. These are the scopes the agent holds, the scopes its delegation allows, and the scopes in the token, combined by intersection. The result is the set of rights in force for this action.
- It returns a decision. If the effective scopes include the scope the action needs, the decision is allow. If not, the decision is deny, with a reason and the missing scope.
An invalid token is a different case. A bad or expired token throws an error, which the application rejects with a 401. A valid token that lacks the right scope is not an error. It is a lawful deny, and the system records it as one. The difference matters: a deny is a normal, recorded outcome, not a crash.
The denial, step by step
Here is the review agent's illegal delete, walked through the fixed system. The demo seeds named case files, and the run acts on the first two active ones. The review agent targets the first, Halvorsen v. Meridian Logistics.
The review agent holds two scopes: records:read and records:annotate. It does not hold records:delete.
- The application calls
authorizewith the review agent's token and the actionrecords:delete. - The component verifies the token. The caller is the registered review agent. Its effective scopes are
records:readandrecords:annotate. - The action
records:deleteneeds the scoperecords:delete. The effective scopes do not include it. - The decision is deny. The reason is
insufficient_scope. The missing scope isrecords:delete. - The system does not delete the file. Halvorsen v. Meridian Logistics stays active.
- The system writes one row: agent identity, the action, the decision deny, the reason, the scopes in force, the missing scope, and the run identifier.
The lawful delete by the disposition agent runs through the same call and returns allow, because the disposition agent holds records:delete. It targets the second file, Project Cormorant — internal strategy memo, and the file is deleted. It produces a row too, with decision allow and the scopes that permitted it.
Now the two deletes are different rows. One says allow. One says deny. The auditor can tell them apart, because the authority is in the record.
The authority record in enforced mode. Each row shows the verified machine-to-machine subject (the raw Kinde identity, not a friendly name), the action, the file, and the scopes in force. The scopes identify the agent: records:read + records:annotate is the review agent, so its delete is denied; the disposition agent's scopes include records:delete, so its delete is allowed. The denied file stays active; the allowed file is deleted.
One row per action
The row the app builds from the component's decision is the authorized-action record. It carries the fields an auditor needs:
- the organization the action belongs to,
- a sequence number, unique and in order within the organization,
- the run identifier, which also links to the Langfuse trace,
- the time,
- the agent's verified identity,
- the authority root, the person or agent the action was performed for,
- the effective scopes in force,
- the action, and the target it acted on,
- the decision, and on a deny, the reason and the missing scope,
- and two hash values that chain the row to the one before it.
The last two fields make the record tamper-evident, which is the next piece.
The hash chain
A record that anyone can edit is not proof. If a person can open the table and change a deny to an allow, the auditor cannot trust any row. So the rows form a chain.
Each row carries the hash of the row before it, called prevHash, and its own hash, called rowHash. The system builds rowHash from the row's own fields and the previous row's hash. The first row uses a fixed starting value. Every later row depends on every row before it.
This has one useful result. If a person changes any field in any past row, that row's hash no longer matches, and every row after it no longer matches either. A single edit breaks the chain from that point on, and a check finds the exact row where the break starts.
The demo has a Verify integrity action. It walks the chain, recomputes each hash, and reports either "chain verified, N rows" or the sequence number where the chain breaks. This shows the rows are unchanged, rather than promising it.
After clicking **Verify integrity, the check walks the chain and reports "Chain verified — 2 rows, unbroken." A changed row would report the sequence number where the chain breaks instead.
The diagram below shows the fixed path. The action goes through authorize first. The decision and the identity go into the row. The row chains to the row before it.
Why an auditor would accept this record
A custom log format does not help an auditor. Auditors work from known frameworks, and they trust records that match practice they already recognize. The design in this article aligns with established standards on every point.
The fields align with NIST 800-53
The NIST 800-53 catalog is the control set most audit programs build on. Its audit family, the AU controls, defines what an event record should hold and how to protect it. The authorized-action row lines up with three of them:
- AU-3, content of audit records. AU-3 lists the fields an event should carry: the type of event, when it happened, where, the source, the outcome, and the identity involved. The row carries each of these. AU-3(1) adds "access control or flow control rules invoked," which is exactly the effective scopes and the decision the row records.
- AU-9, protection of audit information. AU-9 requires that audit records resist change. The hash chain supports this goal: an edited row breaks the chain, so a naive change cannot pass without detection. The chain gives tamper-evidence, not access control, so it is one part of an AU-9 story, not the whole of it.
- AU-10, non-repudiation. AU-10 requires that an actor cannot later deny an action. The row supports this by binding a verified identity to each decision. Full non-repudiation also needs a signature on the row, which this demo does not add. See the limits below.
The authority model follows RFC 8693
RFC 8693 is the OAuth 2.0 Token Exchange standard. It defines how one party acts on behalf of another, and it gives the act claim to name the acting party in a delegation. The row follows this shape: it names the actor and the authority root.
Be precise about what this demo records. Each agent acts on its own machine-to-machine identity, so the row's authority root is the agent itself, not a human. The component also supports a user-rooted delegation, where a person delegates a subset of scopes to an agent and the row roots the authority in that person. The demo does not exercise that path. The effective-scope intersection still follows the standard's rule either way: a delegated actor cannot hold more than the party it acts for.
The row shape maps to OCSF
The Open Cybersecurity Schema Framework, OCSF, is a vendor-neutral schema for security events. Its "API Activity" class covers create, read, update, and delete calls, the same class OCSF uses for AWS CloudTrail events. Its "Actor" object names the user, role, or process behind an activity. The row uses the same shape: an actor, an action, a target, an outcome, and a time. A security team can read the row into its existing tools without a custom parser.
The tamper-evidence maps to known practice
Two standards support the hash chain:
- RFC 8785, JSON Canonicalization Scheme (JCS). A hash is only stable if the input is serialized the same way every time. JCS defines one deterministic way to serialize JSON, so the same row always produces the same bytes and the same hash. The demo canonicalizes each row with JCS before it hashes. Many do-it-yourself audit logs skip this step and get unstable hashes.
- Chain of custody, as used by AWS CloudTrail. CloudTrail protects its log files with a hash chain, where each file references the hash of the one before it. The demo uses the same primitive at the row level. This pattern also supports common audit programs, such as SOC 2 and PCI DSS, which require that log records be protected from change.
What this demo does not do
An honest record states its own limits. The demo uses a SHA-256 hash chain inside one Convex deployment. It does not sign rows with a private key, and it does not anchor the chain outside the system. So a person who controls the store could rewrite the whole chain and recompute every hash, and the internal check would still pass. A production system that must resist that attacker would go further:
- Merkle inclusion proofs, so a single row can be proven without replaying the whole chain.
- External root anchoring, where the chain's current hash is published to a separate system on a schedule, so the party that runs the store cannot rewrite history unseen.
- Separation of duties, so the person who can change the system cannot also control the roots.
Standards such as SCITT describe this kind of transparency for supply-chain and attestation records. The demo does not implement them. It implements the core chain, and it names the next step. An auditor accepts a record that states its own limits, and rejects one that overclaims.
Building it
The demo is a small monorepo. The web app holds the Convex functions and the user interface. A separate agents package holds the agent graph. A third package is the only way the agents reach the app. This section shows the real code for the parts that matter: the mode switch, the authorization call, the provenance row, the hash chain, and the trust boundary.
The stack, and the job of each part
- Next.js serves the web app and the demo page.
- Convex holds the database and the server functions. It stores the records, the blind log, and the provenance chain, and it drives the interface in real time.
- Kinde issues and verifies each agent's identity and scopes. Each agent is a separate Kinde machine-to-machine application, so each has its own identity.
- kinde-convex-agent-auth is the authorization component. It verifies the caller, makes the decision, and gives back the result the app records.
- Mastra runs the agent graph: intake, then review, then disposition.
- Langfuse traces how the run executed. It is not the audit record.
The demo page. The scenario states the problem in plain language, the legend names the three agents and what each may do, and the "Demo control" flips the one server mode. Login is optional — an anonymous visitor gets the full page.
The two modes, decided on the server
The demo has two modes. Broken mode writes the blind log and checks nothing. Enforced mode verifies the agent and records the decision. The mode is a server value. A request can never choose its own mode, because that would let an agent pick the mode that lets its action through.
The resolveAuthzMode function reads the mode from one global setting, then from the deployment environment, and never from the request:
// apps/web/convex/authzMode.ts
export async function resolveAuthzMode(ctx: QueryCtx): Promise<AuthzMode> {
const row = await ctx.db.query('demoSettings').first();
if (row !== null) {
return row.authzMode;
}
return process.env.AUTHZ_MODE === 'enforced' ? 'enforced' : 'broken';
}
The demo page has a toggle that flips this one global value, so a reader can switch modes and watch the result change. The toggle sets the global mode. It does not attach a mode to a request. The action path always reads the resolved global value.
The single action entry point
Every record action goes through one function, performAction. It reads the mode first. In broken mode it writes the blind row. In enforced mode it runs the full path: verify, start the run instance, authorize, then record.
// apps/web/convex/agentActions.ts
const mode = await ctx.runQuery(internal.authzMode.readAuthzMode);
if (mode === 'broken') {
return await ctx.runMutation(internal.agentActions.performBroken, {
orgCode: args.orgCode,
actorAgentId: args.actorAgentId,
action: args.action,
recordId: args.recordId,
title: args.title,
kind: args.kind
});
}
// ENFORCED. An invalid or missing token throws. A denial does NOT throw —
// it is returned as decision.allowed === false and recorded.
if (args.token === undefined || args.token.length === 0) {
throw new Error('enforced: missing bearer token');
}
The comment states the rule that matters. A bad token is an error. A lawful deny is not an error.
Verify, then authorize
Enforced mode does three things before it records anything. It verifies the token, it starts a run instance for the agent, and it authorizes the action.
// apps/web/convex/agentActions.ts
// 1. Verify the token to resolve the registered agent (throws if invalid).
const verified = await agentAuth.verifyCaller(ctx, args.token, {
expectedOrgCode: args.orgCode
});
if (verified.agentId === null) {
throw new Error('enforced: token maps to no registered agent');
}
// 2. Start the run instance, only after token verification.
const instanceId = await ctx.runMutation(internal.agentAuth.startInstance, {
agentId: verified.agentId,
runId: `${args.correlationId}:${verified.agentId}`,
actingForSubject: args.actingForSubject ?? verified.subject,
orgCode: args.orgCode
});
// 3. Authorize this action for the instance. authorize() (never authz.can)
// threads the verified caller in, so the decision is bound to this agent.
const {caller, decision} = await agentAuth.authorize(ctx, args.token, {
instanceId: instanceId as CanArgs['instanceId'],
action: args.action,
enforceTokenScopes: true,
requireOrgCode: true,
...(args.recordId === undefined ? {} : {resource: args.recordId})
});
Two options in the authorize call carry the weight. enforceTokenScopes: true makes the decision use the scopes the token actually carries, so an agent cannot claim a scope its token does not hold. requireOrgCode: true binds the decision to the organization, so a token for one tenant cannot act on another.
The call returns caller and decision. The caller is the verified identity and its scopes. The decision is allow or deny, with a reason and the missing scope on a deny. The app records both.
One row, allow or deny
The commit step performs the action only on allow, then writes one provenance row either way. A deny performs nothing, and it still produces a row, because a denied attempt is part of the audit trail.
// apps/web/convex/agentActions.ts
const resourceId: string = args.allowed
? await applyAction(ctx, args.orgCode, args.action, args.recordId, args.title, args.kind)
: (args.recordId ?? '');
await appendProvenanceRow(ctx, {
orgCode: args.orgCode,
correlationId: args.correlationId,
ts: Date.now(),
actorAgentId: args.actorAgentId,
actorSub: args.actorSub,
authorityRootKind: args.authorityRootKind,
authorityRootSub: args.authorityRootSub,
delegationId: args.delegationId,
effectiveScopes: args.effectiveScopes,
action: args.action,
resourceType: 'records',
resourceId,
decision: args.allowed ? 'allow' : 'deny',
...(args.denyReason === undefined ? {} : {denyReason: args.denyReason}),
...(args.requiredScopes === undefined ? {} : {requiredScopes: args.requiredScopes})
});
Compare this with the broken path, which writes a row with no authority at all:
// apps/web/convex/agentActions.ts — broken mode
await ctx.db.insert('activityLog', {
orgCode: args.orgCode,
actorAgentId: args.actorAgentId,
action: args.action,
resourceType: 'records',
resourceId,
ts: Date.now()
});
The broken row has an actor label, an action, a target, and a time. It has no verified identity, no scopes, and no decision. That is the blind log from the start of this article.
The blind log in broken mode. Both deletes land as the same shape — an actor label, the action, the file, a time — each marked "no authority recorded." Nothing on either row says which delete was allowed. The authority panel below stays empty.
The hash chain
The provenance row carries two hashes, prevHash and rowHash. The append function canonicalizes the row with RFC 8785 JCS before it hashes, so the same row always produces the same bytes:
// apps/web/convex/provenance.ts
async function computeRowHash(
rowBody: Record<string, unknown>,
prevHash: string
): Promise<string> {
const canonical = canonicalize(rowBody); // RFC 8785 JCS, never JSON.stringify
if (canonical === undefined) {
throw new Error('provenance: canonicalization failed');
}
return await sha256Hex(canonical + prevHash);
}
The append reads the last row inside the same transaction, so the chain stays gapless and ordered even under concurrent writes:
// apps/web/convex/provenance.ts
const last = await ctx.db
.query('provenance')
.withIndex('by_org_seq', (q) => q.eq('orgCode', input.orgCode))
.order('desc')
.first();
const seq = last === null ? 0 : last.seq + 1;
const prevHash = last === null ? GENESIS : last.rowHash;
The check reverses the process. It walks the rows in order, confirms each sequence number, confirms each prevHash, recomputes each rowHash, and reports the first row that does not match:
// apps/web/convex/provenance.ts — verifyChain
let prev = GENESIS;
for (let i = 0; i < rows.length; i++) {
const row = rows[i];
if (row.seq !== i) {
return {ok: false, brokenAtSeq: row.seq, reason: `seq_gap (expected ${i})`};
}
if (row.prevHash !== prev) {
return {ok: false, brokenAtSeq: row.seq, reason: 'prev_hash_mismatch'};
}
// ... recompute the row hash from its stored fields ...
if (recomputed !== row.rowHash) {
return {ok: false, brokenAtSeq: row.seq, reason: 'row_hash_mismatch'};
}
prev = row.rowHash;
}
return {ok: true, length: rows.length};
The Verify integrity button on the demo page calls this function. A clean chain returns the row count. A changed row returns the sequence number where the break starts.
The trust boundary
There is one more rule that makes the record trustworthy. The agents cannot reach the database directly. If an agent could write to Convex, it could write its own provenance row and forge the decision. So the agents run in a separate package that has no access to Convex at all. The agents reach the app only over HTTP, with a bearer token.
The client the agents use makes this explicit. The token and the delegation are required inputs, with no default:
// packages/api-client/src/index.ts
export interface LockerClientOptions {
/** Bearer token for the agent's Kinde M2M identity. Required — no default. */
agentToken: string;
/** HMAC-signed delegation grant scoping what the agent may do. Required. */
delegation: string;
/** Base URL of the app's HTTP API. Optional; falls back to same-origin. */
baseUrl?: string;
}
The agent also never states its own authority. It asks the app to perform an action. The app derives the authority from the verified token, not from the request body:
// packages/api-client/src/index.ts
// The agent NEVER states its authorization here — in enforced mode the app
// derives authority from the verified token/delegation, not this body.
export interface ActionRequest {
orgCode: string;
actorAgentId: string;
action: RecordAction;
correlationId: string;
recordId?: string;
title?: string;
kind?: string;
}
A rule that lives only in a comment is not a rule. So the build enforces the boundary in continuous integration. A script fails the build if any file in the agents package imports Convex or the app:
// scripts/check-boundaries.mjs
const FORBIDDEN = [
{label: 'convex', test: (s) => /(^|\/)convex(\/|$)/.test(s)},
{label: 'apps/web', test: (s) => /(^|\/)apps\/web(\/|$)/.test(s)},
{label: '_generated', test: (s) => s.includes('_generated')}
];
// ... scan every source file under packages/agents for these imports ...
if (violations.length > 0) {
console.error('✖ Boundary check FAILED — agents may not import the app or Convex.');
process.exit(1);
}
The CI workflow runs this check on every pull request, next to the type check, the lint, and the tests:
# .github/workflows/ci.yml
- name: Boundary check (packages/agents must not import convex/apps/web/_generated)
run: npm run boundary-check
- name: Test
run: npm test
The boundary is a gate, not a convention. The build fails if an agent ever reaches past the HTTP client.
Broken and enforced, side by side
The same run, in the two modes, on the same two files, produces two very different records. The review agent targets Halvorsen v. Meridian Logistics. The disposition agent targets Project Cormorant — internal strategy memo.
Broken mode, the blind log:
| agent | action | file | authority recorded? |
|---|---|---|---|
| review | records:delete | Halvorsen v. Meridian Logistics | none |
| disposition | records:delete | Project Cormorant memo | none |
The two rows look the same. The log cannot tell the illegal delete from the lawful one.
Enforced mode, the authority record:
| agent | action | file | decision | reason | performed? |
|---|---|---|---|---|---|
| review | records:delete | Halvorsen v. Meridian Logistics | deny | insufficient_scope | no |
| disposition | records:delete | Project Cormorant memo | allow | (none) | yes |
Now the rows are different. The review agent's attempt is a recorded deny, and Halvorsen v. Meridian Logistics is still there. The disposition agent's delete is a recorded allow. The auditor can read the authority from the record.
The two screenshots above are the contrast, on the same two files. In broken mode the blind log fills and the authority panel stays empty. In enforced mode the authority panel fills — one DENY, one ALLOW — while the denied file survives.
Three things to take away
- Record the decision, not the action. A log of actions cannot answer an auditor. A record of decisions, each with a verified identity and the exact permission, can.
- Make the authority impossible to forge. The agent must not verify itself, and it must not write its own record. A separate component makes the decision, and a separate store holds the row behind a trust boundary that the build enforces.
- Make the record tamper-evident, and state its limits. A hash chain catches a changed row. Name the stronger steps you did not take, such as external anchoring, so the record is honest about what it proves.
Try it
The demo runs locally, and the static page opens without an account.
- Live page: https://evidence-locker-demo.vercel.app
- Source: https://github.com/kinde-starter-kits/evidence-locker-demo
- Authorization component: kinde-convex-agent-auth
The deployed page shows the scenario, the toggle, the mode banner, existing rows, and the Verify integrity check. It does not run the agents, because the agent run is a server-side worker that does not fit a serverless function. To watch a full run, use the source:
npm ci
cd apps/web && npx convex dev # keep running; seeds the deployment URL
npx convex run seed:seedLocker '{"orgCode":"orgA"}'
npm run dev # http://localhost:3000
Flip the toggle to Broken, run the agents, and read the blind log. Then flip to Enforced, run again, and read the authority record. Click Verify integrity and watch the chain check itself.
To prove the whole story headless, in both modes, with no browser and no live Kinde, run the end-to-end script:
npm run e2e
It seeds one org, runs the agents in broken mode and then enforced mode, and asserts each step: the blind log cannot tell the two deletes apart, the review delete is denied and the file survives, the disposition delete is allowed, and the hash chain verifies. That is the difference between a log that records what happened and a record an auditor would accept.








Top comments (0)