I killed my in-app AI chat, made the agent a first-class user, and made every move it makes auditable

Disclosure: I build Sensefold. It captures what you run into during the day (articles, X and Reddit threads, YouTube videos with transcripts, PDFs, screenshots, images, and ChatGPT, Claude, Gemini, and Grok conversations) and makes them available to you and whatever AI you use. This post covers why I deleted my own chat feature and what building the replacement forced me to get right.

I had a chat box. I removed it.

The first version of the product had the standard feature every "AI knowledge" app has: a chat box over your saved items. Ask a question, get an answer citing the sources it used. It worked. People used it briefly, then went back to ChatGPT or Claude and pasted things in by hand.

The pattern was obvious once I looked: nobody wanted another chat. People already had one, sometimes three, complete with memory, history, and established habits. What they wanted was for that chat to know what they had read, watched, saved, and said last week.

So I deleted the chat and replaced it with a remote MCP server. Paste one URL into Claude, ChatGPT, Claude Code, Codex, or Cursor, sign in, and the agent you already use can search and read your library, and write back to it if permitted. The product stopped being a place you go and became something your agents carry with them.

That shift changes who the user is. In the chat-box version, the human was the only actor, and the AI was just a feature. In the MCP version, the agent is an actor in its own right, running in its own session and operating on your data while you are not looking. That raised the question that shaped everything that followed: when the agent did something at 2 a.m., how do I know, and how do I undo it?

The answer I landed on has two parts:

1. The agent is a first-class user. It gets its own identity, its own permissions, and its own row in the audit log. It is never "the human, but through an API key".
2. Everything it does is auditable and reversible. If a human cannot see it and undo it, the agent is not allowed to do it.

Here is what that meant in practice.

1. Identity: one connection, one principal

The lazy way to let an agent into a product is to hand it the user's API key. That works, but it breaks two things at once: you cannot revoke one agent without revoking all of them, and your logs cannot distinguish an agent's writes from a human's.

So every agent connection is its own principal. For clients that can open a browser (Claude, ChatGPT, Claude Code, Codex, Cursor), that means OAuth 2.1 with PKCE and dynamic client registration. The user pastes one URL:

https://api.sensefold.app/mcp

The client discovers the authorization server, registers itself, and opens a browser. The user approves access and picks a permission tier on the consent screen. Each approval becomes its own connection row in user settings. Reconnecting the same client revokes the previous row rather than stacking a duplicate. Revoked rows stay visible as history and grant nothing.

Headless and CI clients that cannot open a browser get an Agent key instead: created with a specific tier, shown once, and independently revocable and rotatable. Same principle, different transport.

The point is not the OAuth spec. The point is that "which agent did this" is a question the database can answer.

2. Permissions: shape the tool list, then check anyway

MCP has a useful property here: the server decides which tools a client can even see. I use that as the first layer of least privilege across three tiers:

Tier	Tools the client sees	What it adds
read_only	6	search, list, read one item, check quota (plus search/fetch aliases for ChatGPT)
edit	10	save a link, save a note, update a note's text, replace tags
full	11	move an item to the recycle bin

A read-only connection cannot call update_note because the tool is omitted from tools/list. But filtering the list is only the visible half. The server still validates every call against the connection's tier, because a client can send any method name it likes. If it tries an unpermitted method, it gets a tier-denied error rather than a silent no-op.

Two details turned out to matter more than the tiers:

• Read tools carry readOnlyHint. Claude Code and similar clients use MCP tool annotations to decide when to prompt the user. Marking search and read tools as read-only means the agent can look things up without a confirmation dialog every time, saving the human's attention for writes.
• Recommend the lowest tier by default. The setup guide the agent reads says: start with read_only unless the user explicitly needs writes. Most sessions never need more.

3. Audit: who, what, diff, undo

This is the part people skip, but it is what makes the first half honest.

Every write an agent makes creates a revision. Each item in the library has a History drawer that lists every change, who made it (the human, or which named agent connection), and a field-level diff. There is a Revert button. Account-wide, the same feed powers an Activity page, so "what did my agents do this week" is answered on one screen instead of with a database query.

Deletes are soft. The delete_item tool moves an item to the recycle bin where a human can restore it, and no tool deletes permanently. An agent that goes wrong can make a mess, but it cannot make an irreversible one.

One rule came out of this that I would apply to any agent-writable system: no write path without a human-visible undo. If you cannot show the diff and offer Revert, do not ship the tool yet.

4. The boring part that keeps the audit log truthful

An audit log is only useful if it reflects what actually happened, and two ordinary failure modes corrupt it: retries and races.

Retries. Agents retry. Networks flake, clients time out, or the model decides to "try again". If save_note and save_link are not idempotent, one intended note becomes three, and the log dutifully records three writes. The protocol contract fixes this: the client generates a UUID before the first attempt and reuses it on every retry. Submitting the same UUID with different content is rejected as an idempotency mismatch rather than silently accepted.

Races. Two agents, or an agent and a human, editing the same note concurrently. update_note, update_tags, and delete_item all require an expectedVersion taken from a fresh read. If the item moved on, the call fails with VERSION_CONFLICT, and the agent's instructions say: re-read, retry once, then stop. Nobody silently overwrites anybody, and the revision history stays linear enough to remain human-readable.

Neither pattern is novel. Both make the difference between "there is an audit log" and "the audit log is true".

5. Don't touch what the human wrote

Sensefold enriches captures automatically: a saved article gets a summary and tags, a YouTube video gets its transcript and a chapter guide, and an image or PDF gets OCR. Notes the user wrote directly are indexed, but never rewritten by that pipeline. I extended the same rule to agents in the runtime instructions the server sends them: never rewrite a user's note unless explicitly asked. That part is guidance the agent reads, not a server-side check; the server-side guarantee is the revision history and Revert underneath it. An agent with edit access can change a note, and the user can revert it, but the default posture is that the human's words are ground truth and the agent builds around them.

This sounds like a product opinion, and it is, but it is also a safety property: the thing most likely to be wrong in your library is what an AI wrote, and the thing you most need to trust is what you wrote yourself.

What I'd tell anyone building an MCP server that writes

• Give each agent connection its own identity. Never share the user's credential.
• Put the permission choice on the consent screen, and default to read-only.
• Shape tools/list by permission, then enforce on every call anyway.
• Version every write. Show who made the change, show the diff, and offer undo. Soft-delete only.
• Make saves idempotent and updates version-checked, or your audit log will lie.
• State the rules clearly in tool descriptions and server instructions. Agents read them; that is the cheapest guardrail you have.

None of this required anything exotic. It required deciding that the agent is a user, and that users get accountability, not just access.

The scope of what flows through this keeps growing. Today it is web pages, social threads, YouTube videos with transcripts, PDFs, images, your own notes, and your AI conversations; TikTok and Douyin capture are on the roadmap, not shipped. Every new source is one more thing an agent can act on without you watching, which is why the identity and audit half had to come first.

If you want to try the real thing, setup for Claude, ChatGPT, Claude Code, Codex, and Cursor is on sensefold.app/for-agents, the tool schemas and error codes are in the MCP tools reference, and the permission and undo model is written up in agent permissions. I'd genuinely like to hear how other developers handle agent writes, especially anyone who found a case where soft-delete and revert weren't enough.