DEV Community: Royal Simpson Pinto

Designing an offline-first planner that repairs your day instead of shaming you

Royal Simpson Pinto — Wed, 09 Sep 2026 09:30:29 +0000

Most planners I have tried ask for too much before they give anything back. You make an account, you sync to a cloud you did not ask for, and then you get judged: streaks broken, tasks glowing red, a little counter reminding you how many days you missed. The tools that are supposed to reduce the friction of a day end up adding a layer of guilt on top of it.

I wanted the opposite. So I built Tiny Day, a cozy daily planner for Android. Your day, made manageable. No account, no cloud, no analytics.

The core idea: gentle and local

Two commitments shaped every decision.

The first is offline-first. Tiny Day is a React Native app built with Expo (SDK 57). There is no application backend at all. You brain-dump your day in plain language, the app shapes it into a timeline, and everything you type lives on your device. That is not a marketing angle bolted on afterward; it is the architecture. Expo Router owns navigation, Zustand stores own the state, and AsyncStorage persists it. When I say planning is deterministic and local, I mean the scheduler runs on your phone with your data and produces the same result every time. No round trip, no server, works on a plane.

The second is gentleness. Tiny Day never vibrates. It has no streaks. When you do not finish something, it never auto-carries it to tomorrow behind your back. The whole emotional posture of the app is captured in two lines it actually says to you: "Your day has been repaired," and at night, "You did enough for today."

How it works

The flow starts in the morning. You do not fill in a form field by field. You brain-dump, in plain language, and lib/parse.ts interprets that text into task cards, pulling out a name, a category, a duration, a priority, and time heuristics. From there lib/schedule.ts does a priority and energy pass and lays out a timeline. Crucially, it protects meals and deliberately keeps free time open. A planner that fills every minute is just a different kind of stress, so leaving gaps on purpose is a feature, not an oversight.

Priority in Tiny Day is three tiers, and they are shown as a glyph plus a label, never color alone:

▲ must gets a reminder plus one follow-up
● should gets a single reminder
○ optional stays silent

Notifications are local only, scheduled on the device through expo-notifications. There are quiet hours, and a privacy mode that shows a generic "Important reminder due now" instead of leaking the task name onto your lock screen. At most one gentle replan prompt. That is the ceiling on how much the app is allowed to nag you.

The Room is the part I am fondest of. It is a layered SVG scene (built with react-native-svg) with a window sky, a lamp glow, a tint overlay, and a tiny character. It has four time states plus a rain variant, it follows the real clock, and it brightens as you complete things and dims into lamplight at night. It is ambient feedback that carries no numbers and no pressure.

Then there is the feature that named the whole design philosophy: "My day went wrong." Real days fall apart. You wake late, a task runs long, you get too tired, an urgent thing lands. Instead of watching the plan turn into a wall of overdue items, you tap what changed. The repair engine in lib/repair.ts moves optionals, shortens flexibles, protects your ▲ musts and any fixed appointments, and inserts rest. Then it shows you exactly what it is about to change before it applies anything. You stay in control; the app just does the tedious reshuffling. And because repair is deterministic and local like the scheduler, it is fast and predictable.

The day closes with an evening review: a mood check-in, gentle stats, and leftovers triage where you decide, per item, whether something goes to tomorrow, to the backlog, or you simply let it go. Nothing moves on its own.

A few things I made sure not to skip. Accessibility is real: 44px and larger touch targets, dynamic type, reduce-motion and high-contrast toggles, and screen-reader labels. There is a Plan tab with tomorrow, an expandable week, a backlog, and a routine builder. And the Profile can load and remove reversible sample data so you can explore the app without polluting your own tasks.

One honest limitation

The natural-language parsing is a heuristic, not a language model. lib/parse.ts reads your brain-dump with rules for durations, categories, priority cues, and time hints. That is exactly why it can run fully offline and stay deterministic, which I consider a fair trade. But it means phrasing the parser has not seen can land in the wrong category or miss a time you clearly implied. The app leans on this by letting you edit any task in place or reschedule it to an explicit future date and time, so a miss is a quick correction rather than a dead end. Still, if you are expecting the free-form understanding of a cloud AI assistant, this is a simpler, more predictable engine, and I would rather be honest about that than oversell it.

There is also a practical packaging note: the current v1.1.0 APK is ARM64-only, which covers most modern phones and tablets but means a 32-bit-only device needs a separately configured build. Tiny Day targets Android 7.0 (API 24) and newer.

Try it

Tiny Day is open source under the MIT license. It is a fully offline React Native app: no account, no cloud, no analytics, and no vibration. If a planner that repairs your day instead of shaming you sounds like the kind of tool you have been missing, the code and the latest Android release are here:

https://github.com/royalpinto007/Tiny-Day

You did enough for today.

Delivering messages with no internet, no servers, and no SIM

Royal Simpson Pinto — Mon, 07 Sep 2026 09:30:29 +0000

Every messenger you use has a hidden dependency: a working network path to a datacenter. Drop into a basement, a packed stadium, a moving train through a tunnel, an exam hall with jammers, or a remote area with no plan, and the app is just a spinner. The people you want to reach are often standing a few meters away, but your message still has to travel to a server on another continent and back. When that path is gone, so is the app.

Kabootar is my attempt to remove that dependency entirely. It is a messenger with no backend at all. Your phone forms a peer-to-peer mesh with other phones nearby, and messages hop device to device over Bluetooth and Wi-Fi until they reach the recipient. No internet, no servers, no SIM. It is built in Flutter, and the routing core is plain Dart.

The core idea: delay-tolerant networking

The insight that makes this work is refusing to assume the recipient is reachable right now. Normal networking is connection-oriented: open a path end to end, then send. If there is no path, there is no delivery.

Kabootar instead treats the network as a delay-tolerant network (DTN). A message does not need a live end-to-end path at the moment you hit send. It needs a chain of carriers that will exist over time. You hand your message to whoever is nearby. They hold onto it, carry it as they walk around, and pass it along to the next phone they meet. Eventually a carrier bumps into the recipient and the message lands, even if that is minutes later and both you and the recipient have long since walked away.

This is store-and-forward, the same shape as a durable, at-least-once message queue, except the queue is running across a swarm of phones instead of inside a datacenter.

How a message actually travels

The routing strategy is epidemic routing: flooding. When you send a message, it spreads to everyone in range like a rumor. Each device that receives it re-broadcasts it onward, so the message replicates through the crowd, taking every path at once. That redundancy is exactly what makes delivery robust in a network where any single link is unreliable and short-lived.

The whole routing brain lives in one place: a framework-free Dart MeshEngine with zero Flutter, radio, or database imports. Every phone applies the same small set of rules to each envelope it sees:

De-dup by message id. A message can arrive by many paths, but it is acted on exactly once. This is what stops flood storms and routing loops.
Learn from a hello. The contact list is built from whoever comes near, via a handshake.
Deliver if it is for me. Save it, show it, and send back an ack.
Receipt on an ack. When the acknowledgement for one of my sent messages makes it back to me, I flip that message to delivered.
Relay and carry otherwise. Decrement the time-to-live, cache it, and re-flood it onward.
Cap everything. TTL, a max-age, and a cache-size bound keep battery and storage in check so a carrier stays honest.

That is the entire mesh in six rules. Delivery receipts are genuinely end to end: the ack epidemically routes back to the original sender the same way the message went out, so the WhatsApp-style double tick means the message really reached the recipient's device, not just some server.

Because de-dup uses the message id and that seen-set is persisted to SQLite, the system survives restarts. A phone that reboots mid-mesh will not re-flood messages it has already handled. At-least-once delivery becomes effectively exactly-once at the edges.

Bluetooth and Wi-Fi as the transport

Under the engine sits a MeshTransport interface, implemented on top of the platform peer-to-peer stacks (Nearby Connections on Android, Multipeer-style discovery on iOS) through flutter_nearby_connections. Phones continuously advertise and scan, form short-lived peer links over Bluetooth and Wi-Fi, and flush their carried messages whenever a link comes up.

The architecture keeps a clean seam. The UI is Material 3 and binds to a single ChatService, which owns state, the hello handshake, and receipt ticks. ChatService talks to the pure MeshEngine for routing decisions, to SQLite for persistence, and to the transport for the actual radios. Because the engine has no framework or hardware dependencies, its behavior is pinned down by tests that run on a laptop with just the Dart SDK. You can watch store-and-forward-across-time play out (recipient offline, a relay carrying, delivery after the recipient returns, sender eventually learning it was delivered) without touching a phone by running dart run tool/engine_check.dart.

Direct chats and private groups are end-to-end encrypted with X25519, Ed25519, and AES-GCM, with signed messages and a safety code to verify a contact. Relays only ever see ciphertext, which matters a lot when your "relay" is a stranger's phone in a crowd.

One honest limitation

Mesh delivery is only as good as the crowd. Delivery needs a chain of carriers to physically exist between you and the recipient over some window of time. In a dense place, a festival, a protest, a stadium, that chain forms readily and messages move fast. In a sparse setting with few phones running the app, the chain may be long, slow, or never complete. There is no server to fall back on, so if the carriers are not there, the message waits. This is inherent to any mesh, not a bug I can code away, and I would rather state it plainly than pretend proximity networking is magic.

There is also a platform wall worth naming: one Flutter codebase runs on both Android and iOS, but a message cannot currently hop across the OS boundary because the two platforms use different peer radios. v1 meshes within an OS family. Forward secrecy is future work too; today's encryption uses long-term static keys.

Why I built it

I wanted to prove that resilient, private messaging does not require anyone's cloud. No account, no backend, no data leaving the device. It is the kind of infrastructure that keeps working in dead zones precisely because it never depended on infrastructure in the first place.

The routing engine is small, provable, and framework-free, which I think makes it a genuinely fun read if you are into distributed systems. It is open source under MIT.

Repo: https://github.com/royalpinto007/Kabootar

What a registry of real AI-agent failures reveals about where agents break

Royal Simpson Pinto — Sat, 05 Sep 2026 09:30:32 +0000

Every week I read the same story in a slightly different shape. An AI agent deleted a production database. An agent emailed the wrong recipient list. An agent ran up a surprise bill because nobody set a spend limit. These incidents get a viral thread, a few hundred angry replies, and then they vanish. The next team wires up an agent with the exact same missing guardrail, and the exact same thing happens again.

Agent failures are undocumented and, because they are undocumented, they repeat. That is the problem I wanted to fix. So I built AgentPostmortem, a public registry of AI agent failures. Real incidents, documented and searchable, at agentpostmortem.com.

The core idea

Aviation has the NTSB. Software has postmortems and status-page retrospectives. AI agents, which are being handed write access to filesystems, inboxes, payment APIs, and cloud consoles, have nothing comparable. There is no shared, structured record of what has actually gone wrong.

AgentPostmortem is that record. It is a community-driven database of incidents where an AI agent caused real harm: deleted data, sent emails to the wrong people, ran up unexpected bills, exposed credentials, or otherwise went off the rails in production. Cases can be submitted anonymously or with attribution. Every case is reviewed before it goes public, and each approved case gets a permanent identifier in the form APM-XXXX so it can be cited and referenced forever.

The goal is not to dunk on any particular model or vendor. It is to turn one-off war stories into a corpus you can actually search before you ship.

The schema

The value of a registry lives in its schema. If every report is a free-form blog post, you cannot compare or aggregate anything. So the submission is structured and validated. The fields I settled on, enforced with a Zod schema on the server, are:

Agent involved, chosen from a known registry of agents (Claude, GPT-4, o1, o3, and others), each tied to its company.
Title, a concise summary, between 20 and 200 characters.
Prompt, the exact instruction given to the agent, optional, up to 2000 characters. This is often the most instructive field, because the failure frequently lives in the gap between what was asked and what was done.
Outcome, a full description of what happened and the consequences, required, at least 100 characters and up to 10,000.
Damage level, a severity rating from 1 to 5.
Estimated cost in USD, an integer, where 0 means the damage was reputational or non-financial.
Tags, at least one, up to eight, so incidents can be grouped by failure mode.
Evidence, up to five screenshots.

Attribution is optional. A submitter can stay fully anonymous, or attach a handle or company name. If they leave an email, it is used only to send a private edit link and is not meant to be stored long-term.

How it works

The stack is deliberately boring so the data outlives any hype cycle. It is a Next.js 14 App Router app on Vercel, with Supabase (Postgres with row-level security) as the database, Cloudflare R2 for screenshot storage, and Resend for delivering edit-link emails.

When you submit a case, it does not appear immediately. The submission goes through moderation. Behind the scenes the API hashes the submitter IP with a fixed pepper for rate limiting and abuse control, runs PII redaction over the text, and stores the case with a pending status. Nothing is published on trust. When a moderator approves a case, it is assigned the next sequential APM number, zero-padded, and only then does it become part of the public feed.

The public side is built for browsing and discovery. There is a feed with hot, new, and this-week views, individual case pages at a stable URL per case number, agent profile pages, tag pages, a search endpoint, a hall of fame for the highest-voted cases, and a stats page. Visitors can vote, which is what powers the ranking.

Crucially, the whole corpus is exportable. There is an export endpoint that dumps every approved case as CSV: case number, title, agent, company, damage level, estimated cost, vote score, tags, author, date, and the full outcome text. That matters to me because a registry you cannot get the data out of is just another silo. If someone wants to run their own analysis of agent failure modes, the raw data is one request away.

One honest limitation

The hardest problem here is not the code. It is verification. AgentPostmortem relies on people to report incidents accurately, and moderation can confirm that a report is coherent, on-topic, and free of obvious abuse, but it usually cannot independently prove that an incident happened exactly as described. Screenshots help, and attribution helps, but a determined person could still submit a plausible fabrication, and an anonymous submitter cannot always be followed up with.

I decided that structured, moderated, imperfect documentation is still far better than the status quo, which is nothing. The severity and cost fields are self-reported estimates, not audited figures, and they should be read that way. Over time I would rather add verification signals, such as linking to a public source or corroborating reports, than pretend the current bar is higher than it is. For now, treat the registry as a well-organized collection of field reports, not a court record.

Why I keep working on it

The number of agents with real write access to real systems is only going up. The institutional memory of how they fail is not keeping pace. A shared, searchable, citable record of failures is the kind of infrastructure that gets more valuable with every case added, and it costs the community almost nothing to contribute to.

If you have watched an agent do something it should never have been allowed to do, that is a case worth documenting. Submissions are open and anonymous submissions are accepted. The code is MIT-licensed and issues and PRs are welcome, including adding new agents to the registry.

Repo: https://github.com/AgentPostmortem/agentpostmortem

Observability for AI agents without instrumentation: read the transcript

Royal Simpson Pinto — Fri, 04 Sep 2026 09:30:29 +0000

Most observability advice starts the same way: wrap your calls, add a tracer, emit spans, ship them somewhere. That is a lot of work to do before you have learned anything, and it only helps the runs you thought to instrument ahead of time. The run you actually want to understand is usually the one that already happened, the one you did not plan for.

Here is the thing I kept forgetting: the transcript is already the source of truth. Claude Code writes every session to disk as it goes, at ~/.claude/projects/<slug>/<session-id>.jsonl. Every subagent delegation, every prompt, every result, every tool call. It is all there whether or not you set anything up in advance. So the whole "add instrumentation" step is skippable. You do not need to trace the run. You need to read the file the run already left behind.

I built two small tools on that idea. They read local files, no API keys, no network, no SDK. One asks "which of these results should I not trust?" and the other asks "what filled up the context window and what can I delete?"

Tool one: flagging runs you should not trust

The first tool is agentrace. Directing agents turned out to be the easy half of the job. The hard half is verification. When you fan out ten background subagents and each returns a confident wall of text, generation is not the bottleneck. Knowing which answers count as proof is. And you cannot verify what nobody reads. By the time the agents report back, the interesting parts are buried in a transcript nobody opens.

So agentrace reads it for you and flags the suspicious results. Every check exists because it actually bit me, over roughly 150 research subagents across two weeks. A few of the ones that earned their place:

error: agents dying on session limits mid-sweep, work silently lost, nobody noticing until the final report came back short.
absence_as_evidence: an agent concluded a company was not hiring because an API returned an empty list. That API returns empty with HTTP 200 for accounts that do not even exist. Absence of data is not evidence of absence.
gave_up: "I was unable to find..." reads like an answer if you skim. It is not one.
hedged_claim: an agent said a company "appears to be" hiring. That hedge quietly became a fact by the time it reached a decision. The hedge was honest; the bug was flattening it downstream.
unverified_urls: twenty URLs cited, none opened. That is autocomplete, not research.
no_output_contract and thin_prompt: the failure that is mine, not the model's. A task with no definition of done cannot be verified, because I never really asked the question.

Run against the exact session that motivated the tool, 152 subagent runs, it flagged 36 of them, with 39 findings total: 7 agents that died on session limits, 17 hedged claims, and 12 prompts where I forgot to specify an output shape.

That last number is the one I keep coming back to. Twelve of the flagged runs were not model failures at all. They were me handing an agent a task with no shape and then being surprised the result had no shape. Most agent tooling assumes the model is the problem. A lot of the time the prompt is.

Tool two: profiling the context window

The second tool is ctxlens, a flamegraph for the context window. Agents get slow, expensive, and frankly dumb when their context fills with junk: the same file read six times, a 12k-token tool result that mattered for exactly one turn, tool schemas re-sent on every single step. Token dashboards tell you the bill. They do not tell you where the bytes went or what to cut.

ctxlens parses the same transcripts and gives a per-turn breakdown. Every message is attributed to a segment: system, tool_definitions, user, assistant, thinking, tool_call, tool_result. Then it computes a waste ratio, total_waste / total_tokens, summing duplicate tokens, tool-result bloat above a cap, stale outputs, and oversized tool definitions. On one session it profiled 12,481 tokens across 14 turns and called out 4,932 of them, 39.5 percent, as waste.

The recommendations are rule-based and specific, not generic advice:

Repeated content: the same file or tool result appearing more than once. One session had Read:file_path=config.py show up six times across turns 2, 5, 7, 9, 11, and 13. Every copy after the first is dead weight.
Stale tool outputs: an older result superseded by a newer one for the same target, still occupying context.
Oversized tool definitions: schemas above budget, paid on every turn.
Tool results dominate and single biggest consumer: callouts when one category or one message is eating the window.

Each recommendation carries a severity and an estimated token saving, so you can act on the expensive ones first. It works offline with a deterministic heuristic tokenizer for relative profiling and CI, and upgrades to exact BPE counts automatically when tiktoken is installed. That fallback matters more than it sounds: it means the tool runs anywhere with no heavy dependency, and you only pay for tiktoken when you need the exact number.

The one honest caveat

These are heuristics over text. That is the whole trick and also the whole limitation. agentrace tells you what to go read; it does not tell you what is true. A checker that cries wolf gets switched off, which is worse than no checker at all, so severity is deliberately conservative. thin_prompt used to fire on any prompt under 200 characters, until I noticed it was flagging perfectly verifiable short prompts and teaching nobody anything. Now a prompt has to be short and never say what done looks like before it fires. On the bundled fixture that dropped findings from 16 to 9 without losing a single true one.

Same honesty applies to ctxlens. A heuristic tokenizer is great for relative comparisons and catching duplication; it is not the exact bill. Treat both tools as a fast way to point your attention, not as a verdict.

What this buys you

No instrumentation. No wrapper. No SDK. The data is already on disk, so you can analyze the run you wish you had traced, after the fact, on any session, including the one still being written (both tools skip a torn final JSON line instead of refusing to parse). That is the part I did not expect going in: the observability was never missing. It was sitting in a .jsonl file the whole time, waiting for something to read it.

Both tools are MIT licensed and up at github.com/royalpinto007. If you run agents at any scale, point them at your ~/.claude/projects directory and see what falls out.

Human-in-the-loop patterns for AI agents: when and how to make an agent stop and ask

Royal Simpson Pinto — Wed, 02 Sep 2026 09:30:29 +0000

I build agents that take real actions. Not chatbots that suggest things, but systems that click buttons, issue refunds, and move data around. Once an agent can actually do something, the interesting question stops being "can it decide correctly" and becomes "what happens when it decides wrong." My answer, over and over, has been the same: put a gate in front of the irreversible actions and let a human hold the key.

That is the whole human-in-the-loop pattern in one sentence. Reads are cheap and reversible, so let them run. Writes are not, so make the agent stop and ask before it commits one. What varies is how you implement the stop, and I have shipped three different versions of it across three projects. Here is what each one taught me.

The pattern: a gate before anything you cannot undo

An autonomous agent works because you trust it on the common, safe cases. But autonomy and safety are not opposites you have to trade off against each other. The trick is to split the action space. The safe majority runs automatically. The risky minority pauses and routes to a person. The person only ever sees the decisions that actually need judgment, so the human cost stays low while the blast radius of a bad automated action stays bounded.

The three implementations below differ in where the gate lives: on your phone, in the agent's request contract, and inside the execution graph itself.

Implementation 1: approval as a phone tap (Greenlite)

Greenlite is the human-in-the-loop layer as a mobile app. It is an Expo / React Native codebase that builds to native Android and iOS, and its whole job is to be the place where escalations land when you are not at a desk.

The flow is: an agent wants to do something risky, your phone buzzes, you see the full context and the proposed action, and you approve or deny in one tap. Under the hood it pulls a feed of pending escalations across agents, each carrying a proposedAction and the reason it escalated. Tapping an item shows the full message and a one-tap Approve / Deny that routes back to the originating agent's approve endpoint. Push notifications come through expo-notifications, so an agent can buzz the phone the moment it needs a decision.

What I like about this design is that the approval contract is generic. An item has a proposedAction, and the backend exposes an /api/approve. Any new agent that learns to escalate plugs straight into the same feed with no bespoke UI work. The gate is a protocol, not a screen.

Implementation 2: write-gating in the request contract (Webhands)

Webhands is a computer-use agent for tools that have no usable API. It drives real dashboards (think a seller center or a supplier portal) through Cloudflare Browser Rendering and returns clean structured data plus a screenshot as proof of what it saw.

Here the gate is baked into the request itself. Every step the agent can take is either a read or a write, and any step marked write: true (clicking "Issue refund", say) is refused unless the request also includes confirm: true. So a scraping recipe that logs in and pulls this week's orders runs freely, because it only reads. A recipe that clicks a "confirm shipment" button comes back as an error until you resend it with confirm: true.

The important property is the default. Reads are safe by default and writes are deliberate by default. You do not have to remember to add a safety flag; you have to deliberately add a go-ahead flag. That inversion is the whole point. When someone forgets, the system fails closed, not open.

Implementation 3: a graph interrupt with a checkpointer (relayg)

relayg is a support triage agent built as a LangGraph state machine. It classifies an incoming ticket, runs pure Python refund policy rules, and then acts. The policy rules decide the gate: refunds under $50 auto-approve, refunds over $200 escalate to a senior agent, and refunds between $50 and $200 pause for human approval.

That pause is the most technically satisfying version of the gate I have built. When a ticket lands in the $50 to $200 band, the act node calls interrupt(), which suspends the graph mid-node. The state is checkpointed to SQLite, so the paused run survives a process restart. Later a reviewer resumes it with Command(resume=...), and execution continues from the exact same checkpoint with the human verdict injected into state. Every action the agent takes (send_reply, issue_refund, escalate) is a tool that appends to an audit.jsonl log, so there is a full record of what happened and who approved it.

This is the difference between "wait for a webhook and stuff the state into a database by hand" and having interrupts as a first-class part of the execution model. The paused run is not a suspended HTTP request holding memory hostage; it is durable state on disk you can resume days later. The interrupt and resume path is covered by the project's pytest suite, because if your safety gate is not tested, it is not a safety gate.

When NOT to gate

A gate has a cost, and the cost is a human's attention. If you gate too much, people start rubber-stamping, and a rubber-stamped approval is worse than no gate because it launders a risky action as a reviewed one. So I do not gate reads, ever. I do not gate actions that are trivially reversible. And I do not gate the high-frequency safe cases that make up the bulk of the work. relayg does not pause on a sub-$50 refund, and Webhands does not ask permission to scrape a page. Gating is for the actions where a wrong call actually costs something and cannot be quietly undone.

One honest caveat

A human-in-the-loop gate protects you from bad automated actions. It does not protect you from a bad human decision, and it does not make the agent's proposal correct. If the agent proposes the wrong refund and the reviewer is tired and taps Approve, the gate did exactly its job and you still got the wrong outcome. The gate buys you a decision point, not a correct decision. That is why the context you show at the gate matters as much as the gate itself: Greenlite shows the full message, Webhands returns a screenshot, relayg keeps an audit log. A gate with no context is just a slower way to say yes.

If you want to see any of these end to end, the code is at github.com/royalpinto007.

Pausing a LangGraph Agent Mid-Run for a Human Approval

Royal Simpson Pinto — Mon, 31 Aug 2026 09:30:29 +0000

I had a support triage agent already running in production. It was called Resolvd, wired up in n8n with Claude doing the classification, and it worked. So why rebuild the same thing in code? Because there was one behavior the visual workflow made awkward, and I wanted to see it done properly: pausing a running agent in the middle of a decision, holding its entire state on disk, and resuming it only after a human says yes or no.

That behavior is the whole reason RelayG exists.

The problem

A triage agent for refunds is mostly boring, and that is good. Someone was double charged $25, you refund it automatically. Someone wants $450 back, that is above your ceiling and a senior agent should look at it. The interesting cases sit in the middle. A $120 refund is too much to hand out on autopilot but too routine to escalate every time. You want a person to glance at it and approve or deny, and you want the agent to wait for that answer without losing its place.

In a workflow tool you approximate this with webhooks, wait nodes, and state you stuff into a database by hand. It works, but the pause is bolted on. I wanted the pause to be a first-class part of how the agent runs.

The core idea: the agent is an explicit state machine

The thing I care about most here is that RelayG is not a vague "LLM in a loop." It is a state machine with three named nodes and edges between them, declared up front:

classify -> policy_check -> act -> END

classify reads the ticket and produces structured labels. policy_check applies pure Python refund rules. act takes the action. That is the entire graph, wired in LangGraph with add_node and add_edge. Because the shape is explicit, I can read the control flow without running anything, and I can unit test each node in isolation.

The state that flows through those nodes is typed. It is a TypedDict carrying the ticket fields, the classification, the policy decision and its reason, and an actions list that accumulates across nodes via an operator.add reducer. The classification itself is a Pydantic model with Literal types for intent and urgency, so "refund_request" is a value the type system knows about rather than a string I hope is spelled right.

One deliberate choice: the LLM only touches classify. It extracts intent, urgency, and a refund amount. Every actual decision about money happens in policy_check, which has no LLM and no I/O:

if amount < AUTO_APPROVE_LIMIT:          # under $50
    return "auto_approve", ...
if amount <= HUMAN_APPROVAL_LIMIT:       # $50 to $200
    return "needs_approval", ...
return "escalate", ...                    # over $200

The model reads text. Deterministic rules decide what happens. That split is the point.

How the pause works

The interesting node is act. When the policy decision is needs_approval, instead of returning a value, the node calls interrupt():

verdict = interrupt(
    {
        "ticket_id": ticket_id,
        "question": f"Approve refund of ${amount:.2f}?",
        "reason": state["policy_reason"],
    }
)
approved = bool(verdict.get("approved"))

interrupt() suspends execution right there, in the middle of the node. LangGraph writes the current state to a checkpointer and hands the interrupt payload back to whoever invoked the graph. The run is genuinely stopped, not blocked on a thread. If the process died at this moment, the paused ticket would still be sitting in storage.

The checkpointer in RelayG is a SqliteSaver over a plain SQLite file, keyed by a thread_id (I use the ticket id). When a human is ready to answer, you resume by invoking the graph again with a Command:

result = graph.invoke(Command(resume=verdict), config)

LangGraph loads the checkpoint for that thread, injects verdict as the return value of the interrupt() call that was frozen earlier, and the act node continues from exactly that line. If approved, it issues the refund and records who approved it; if denied, it sends a decline reply carrying the reviewer's note. Every action, refund, reply, or escalation, is appended to an audit.jsonl log, so after the fact you can see what the agent did and who signed off.

The demo runs five sample tickets through this. Two of them ($120 and $75) hit the approval gate, pause, and get resumed with simulated reviewer verdicts, one approved and one denied. It runs offline by default: with no GROQ_API_KEY set, classify falls back to a deterministic keyword mock, so the interrupt and resume path is fully testable without any API key, and it is covered by pytest.

One honest limitation

The tools are mocks. issue_refund, send_reply, and escalate do not call Stripe or an email provider; they write structured entries to the audit log and return. RelayG is a faithful demonstration of the control flow, the typed state, the checkpointing, and the human-in-the-loop mechanism, but wiring the actions to real systems is left as an exercise. The refund thresholds ($50 and $200) are also hardcoded constants rather than per-customer or per-plan policy, which a real deployment would want.

I am comfortable with that scope, because the part that is genuinely hard to get right in a workflow tool, the durable pause and resume, is the part that is real here.

The tradeoff

Rebuilding Resolvd in LangGraph meant giving up the visual editor and the drag-and-drop integrations that made the n8n version fast to assemble. In exchange I got typed state checked at development time, a graph I can read and unit test as plain Python, and interrupts as the core execution model instead of webhook-and-wait plumbing. For an agent that touches money and needs a human in the loop, that felt like the right trade.

Code, demo, and tests are here: https://github.com/AgentPostmortem/relayg

Read freely, confirm before writing: a safety model for computer-use agents

Royal Simpson Pinto — Sat, 29 Aug 2026 09:30:29 +0000

Every few weeks I hit the same wall. A tool I need to pull data from has no usable API. Seller centers, supplier portals, 3PL dashboards. The data is right there on the screen, but the only way to get it is to log in and click through the UI like a human. So I built Webhands: a computer-use agent that operates those dashboards in a real headless browser, returns clean structured data, and refuses any write action unless I explicitly confirm it.

The problem with agents that click

The moment you hand an agent a real browser session, you have given it the power to do everything the logged-in human can do. That includes the dangerous stuff. Issuing a refund. Confirming a shipment. Canceling an order. Reading a page is safe. Clicking "Issue refund" is not, and the difference between the two is one button.

Most automation frameworks treat every action the same way. A click is a click. That is exactly the design choice I did not want, because a scraping run that quietly mutates production state is not a convenience, it is an incident waiting to happen. I wanted reads to be safe by default and writes to be deliberate, with no way to trip into a mutation by accident.

The core idea: recipes and write-gating

You drive Webhands by POSTing a recipe. A recipe is just an entry URL, an optional list of login and navigation steps, and an extraction spec. Here is the shape of a step in the source:

export type Step =
  | { action: "goto"; url: string }
  | { action: "type"; selector: string; text: string; secret?: boolean }
  | { action: "click"; selector: string; write?: boolean }
  | { action: "waitFor"; selector: string; timeoutMs?: number };

Notice the write?: boolean on click. That flag is the whole safety model. Typing, waiting, navigating, and reading are inherently safe. The only action that can change state is a click, so the only action that can carry write: true is a click. If a step is marked as a write, the request is refused unless it also carries confirm: true.

The gate itself is tiny and lives before any browser even launches:

if (hasWriteStep(recipe) && !confirm) {
  return {
    ok: false,
    mode: env.BROWSER ? "live" : "dry",
    steps: [],
    error: "recipe contains a write step; resend with confirm:true to execute",
  };
}

hasWriteStep returns true if any step is a click marked write: true. When that is the case and confirm is absent, the function returns an error and never opens the browser. To actually run the write, you resend the exact same recipe with "confirm": true. Reads never need confirmation because they can never mutate anything.

I like that this check runs before the browser is provisioned. There is no window where a write step has partially executed and then gets caught. The refusal happens up front, deterministically, based purely on the shape of the recipe.

What comes back

On a successful run you get three things. First, data: structured JSON. You can extract it two ways. Give the recipe a list of fields with CSS selectors and it scrapes them directly with no model involved. Or give it a natural language extract.prompt and it hands the page text to Claude and asks for JSON matching your request. Second, screenshotBase64: a base64 PNG of exactly what the browser saw, so you have proof. Third, steps: the log of actions it actually took, like goto ..., type into #email (secret), click #signin.

The extraction path is honest about degrading. If no Anthropic key is set, the prompt-based extractor returns the raw text slice instead of failing, so the pipeline still runs in development. When the key is present it calls the Messages API with a Claude Haiku model, asks for JSON only, strips any code fences, and parses. If parsing fails it returns the unparsed text rather than throwing away the result.

Two modes, so you can build without paying

Webhands runs on Cloudflare Workers with Browser Rendering. There are two modes. In live mode the BROWSER binding is present and it drives a real headless browser via Cloudflare's Puppeteer. In dry mode the binding is absent and it returns the plan it would have run, labelled as a dry result, instead of executing anything. That means you can author and test recipes without a paid binding, then flip to live when you are ready. Even when the binding exists but Browser Rendering is over quota or unprovisioned, the launch is wrapped so it degrades into a clean error instead of crashing.

An honest limitation

The write-gate protects you from executing a mutation you did not confirm. It does not understand what a given click means. The write: true flag is set by whoever authors the recipe. If you mark a genuinely destructive button as a read, or forget to mark it at all, Webhands will happily click it without asking, because from its point of view a click without write: true is just navigation. The safety model is only as good as the labelling. It is a forcing function for deliberate writes, not a classifier that detects danger on its own. I chose that tradeoff deliberately, since guessing intent from a button label is far less reliable than an explicit flag, but it does mean the human writing the recipe still owns the judgment call.

The other honest caveat is that this operates real UIs. Selectors break when dashboards change, logins get challenged, and a page that renders slowly can time out. Screenshot proof and the step log exist precisely because runs against real portals are messier than runs against an API contract.

Closing

"There's no API" becomes "there's an agent." Webhands is my attempt to make that swap safe enough to actually use against production dashboards, by making reads free and writes something you have to ask for twice. The recipe format is small on purpose, the gate is a few lines, and both modes let you iterate cheaply before anything touches a live account.

Code is here: https://github.com/AgentPostmortem/Webhands

Putting a one-tap human gate in front of an autonomous agent

Royal Simpson Pinto — Thu, 27 Aug 2026 09:30:29 +0000

I build agents that take real actions: resolving support tickets, issuing refunds, replying to customers. The moment those agents stop being demos and start touching money or reputation, the same question shows up every time: what happens on the action that is too risky to automate?

You have two bad options. Either you let the agent act on everything and hope its judgment holds, or you make it stop and wait for a human. The second is correct, but it usually means a human parked at a dashboard, refreshing a queue. That does not scale, and in practice it means the "human in the loop" is asleep, in a meeting, or nowhere near a laptop when the agent actually needs a decision.

Greenlite is my answer to that: the human-in-the-loop layer that lives in your pocket. An agent wants to act, your phone buzzes, you see the full context and the proposed action, and you approve or deny in one tap.

The core idea

The insight is that autonomy is only safe to turn on if the escape hatch is fast. If asking for human approval is expensive, you will avoid asking, and you will end up over-automating. So the whole design goal was to make the approval itself trivial: notification to decision in a couple of taps, wherever you happen to be standing.

That reframes the agent's job. The agent handles the safe cases automatically and pushes only the rest to a human. A refund under the auto-limit just goes through. A $900 refund, or a message dripping with high-urgency negative sentiment, gets escalated. You stay in control of exactly the decisions that need a person, and nothing else.

The other design choice was to keep the approval contract generic. An escalation is just an item with a proposedAction, a reason it escalated, and the underlying detail. Any agent that can produce that shape and expose an approve endpoint plugs straight into the same feed. Greenlite is not tied to one agent, it is a control surface for a whole suite.

How it works

Greenlite is one Expo (SDK 51) plus expo-router codebase in TypeScript that builds to native Android and iOS. There are three moving parts: the feed, the decision, and the buzz.

The feed. Pending approvals are escalations pulled from each agent's own server-side API. Today that source is my support agent's /api/approvals endpoint. One thing I want to call out, because it is a deliberate security decision: Greenlite reads from the agent's HTTP API, never from a database directly. The app carries no database credentials. It sends a token in an x-resolvd-token header and gets back a list of approval items. Each item carries an id, the source agent, a title, the full detail, the proposed action, and the reason it escalated. If the fetch fails, the feed simply comes back empty rather than crashing or guessing.

The decision. Tap any item and you see the full message plus the proposed action. Approve or deny is one tap. That decision routes back to the originating agent's approve endpoint, a POST to /api/approve carrying the item id and a boolean. The agent owns what "approved" actually means; Greenlite just relays the human's verdict back to the system that raised the question. Clean separation: the phone is the judgment, the agent is the executor.

The buzz. Registration uses expo-notifications. On first run the app requests notification permission, sets up a high-importance "approvals" channel on Android, and fetches an Expo push token. You store that token server-side to target the device, so an agent can buzz the phone the exact moment it needs a decision instead of waiting for you to open the app and check. That is what turns this from a queue you have to remember to check into something that reaches out to you.

There is also a demo mode. When no backend credentials are configured, the app runs against sample approvals, the $900 refund and the angry customer, so you can explore the whole flow immediately without wiring up an agent first. It made the thing shippable and explorable on day one, which matters for something you want people to actually try.

One honest limitation

The approval source is not yet plural in practice. The contract is generic and designed for many agents to feed one queue, but right now the wired-up source is a single agent's endpoint, and the token model reflects that: it is one shared token in the request header, not per-agent auth or per-user identity. For a real multi-agent, multi-approver deployment you would want scoped credentials, an audit trail of who decided what, and probably signed requests so a leaked token is not a master key to approve anything. The mechanism is right; the trust model around it is still early. I would not put this in front of a production money-moving flow without hardening that layer first.

That trade-off was intentional for where the project is: prove the interaction, prove that notification-to-one-tap-decision feels good and safe, then invest in the auth story once the shape is validated. It is a lot easier to add per-agent tokens to a working approval loop than to design the whole thing around auth you have not tested.

Closing

The thing I keep coming back to is that agent safety is often framed as a model problem, better guardrails, better judgment inside the agent. A lot of it is actually an interface problem. If the human gate is slow, people route around it. If it is one tap on a device you already have on you, keeping a human in the loop stops being a tax and starts being the default. Greenlite is my attempt to make the safe choice the easy one.

Code and setup are here: https://github.com/AgentPostmortem/Greenlite

Why inbox AI only drafts, and what it takes to let it act

Royal Simpson Pinto — Tue, 25 Aug 2026 09:30:29 +0000

Most "AI support" tools stop one step short of being useful. They read the ticket, they draft a nice reply, and then they hand it back to a human who has to re-read the whole thing and click the action anyway. The AI did the easy part. The person still does the work.

I wanted to see how far past that line I could safely go. Resolvd is my answer: an end-to-end inbox operator that triages, drafts, and acts within a policy. It issues the small refund, sends the order status, closes the ticket, and only escalates the cases that genuinely need a person, with the proposed action already attached so the human decision is one click, not a re-investigation.

The core idea: a guardrail between the model and the action

The central design decision is that the language model never decides what happens. It only describes the situation. A separate, deterministic policy layer decides whether the agent may act on its own or must escalate.

That split matters. If you let the model both classify a ticket and choose to issue a $900 refund, your safety story depends on the model behaving every single time. By putting a plain TypeScript function between the classification and the action, the risky decisions are governed by code I can read, test, and reason about, not by a prompt.

So the pipeline has two clearly separated stages: triage (the model's job) and decide (the guardrail's job).

How triage works

A message hits POST /api/inbound, coming from a helpdesk webhook or an email forwarder. The first stage classifies it. When ANTHROPIC_API_KEY is set, Resolvd calls Claude (the Haiku model) with a system prompt that forces JSON-only output: a category (order_status, refund, complaint, or other), an urgency, a sentiment, an optional refund amount, a one-line summary, and a drafted reply.

When there is no key, it falls back to a transparent keyword heuristic. This was deliberate. The heuristic scans the subject and body for words like refund, chargeback, where, track, broken, angry, pulls a dollar amount out with a regex, and infers urgency from negative sentiment or words like urgent and asap. It is not clever, but it means the service runs and demos without an API key, and it keeps the triage stage honest: nothing downstream should depend on triage being magic. If the Claude call throws, the code logs it and drops to the same heuristic rather than failing the ticket.

Either way, triage produces the same typed shape, and the guardrail does not care which produced it.

How the guardrail decides

The decide function takes the triage result plus the raw payload and returns a decision: a status of resolved or escalated, a proposed action, the action actually taken (only set when auto-resolved), a human-readable reason, and the reply to send. Here is the actual logic, in order.

First, a hard override: if sentiment is negative and urgency is high, it escalates no matter the category. The proposed action becomes "Personal apology + offer remedy." An angry customer is never something I want an automated system quietly closing out, even if the category looks routine.

Then it switches on category:

order_status with an order id auto-resolves. This is a read-only lookup, so it is safe to handle without a human. It fetches the status and replies with it. If there is no order id, it escalates with the proposed action "Ask customer for their order number."
refund at or under the auto-limit auto-issues the refund and replies. The limit comes from REFUND_AUTO_LIMIT and defaults to 50. A refund over the limit escalates with "Approve refund of $X" as the proposed action; a refund with no stated amount escalates asking a human to confirm the amount first.
complaint always escalates to a human by policy.
other escalates as "Human review (uncategorized)" because the system could not confidently categorize it.

The dividing line is essentially about reversibility and blast radius. Reading an order's status is harmless. Refunding twenty dollars is bounded and cheap. Refunding nine hundred dollars, or handling an angry complaint, is not something I want a bounded loop doing unsupervised, so those cases go to a person.

Escalation carries the proposed action

The part I care most about is what escalation looks like. A traditional tool escalates by handing a human a raw ticket. Resolvd escalates by handing a human a decision that is already made except for the approval: the category, the reason, the drafted reply, and the specific proposed action ("Approve refund of $900"). A human approves or rejects via POST /api/approve.

That is the difference between "here, you deal with it" and "here is what I would do, say yes or no." The dashboard reflects this too. It shows every ticket, the auto-resolution rate, the action taken or proposed, and the reason for each. You can see at a glance how much the agent handled on its own and exactly why it made each call.

One honest limitation

The actions are not yet wired to real commerce systems. The order lookup is a deterministic stub: it hashes the order id into one of four states (processing, shipped, out for delivery, delivered) so demos stay stable, and the refund path records the decision rather than moving real money. The plumbing for a real deployment (Shopify, an order management system, a payment provider) is a clearly marked seam in the code, not something already connected. So the guardrail logic and the auto-resolve versus escalate flow are real and testable end to end, but the side effects are simulated. I built the decision architecture first on purpose, because that is the part that has to be trustworthy before you ever let it touch production money.

The stack, for the curious: Next.js 14, Supabase for ticket storage, deployed on Cloudflare Workers via OpenNext.

If you have ever watched a support tool "help" by making a person do the same work twice, this is my attempt at the opposite. Code is here: https://github.com/AgentPostmortem/Resolvd

MCP in practice: building, securing, and serving Model Context Protocol servers

Royal Simpson Pinto — Sun, 23 Aug 2026 09:30:29 +0000

The Model Context Protocol is the plug that connects an AI agent to real capabilities. Instead of every client inventing its own way to call your tools, an MCP server advertises a list of tools, resources, and prompts over a standard JSON-RPC 2.0 interface, and any MCP client (Claude Code, Cursor, an agent built on the Claude Agent SDK) can discover and call them. That standardization is the whole point, and it is also where the danger lives: the moment you expose a tool, you are handing a language model a lever it can pull on your systems. Over three projects I built a server, scoped one down to per-client permissions, and wrote a scanner to catch the mistakes. Here is what actually mattered.

Building a server: the transport is simpler than it looks

My first server, casebook-mcp, turns AgentPostmortem (a public registry of documented AI-agent failures) into something an agent can query mid-investigation. The idea: every team debugging an agent incident is rediscovering failure modes someone already wrote up. So the server exposes four tools: search_cases for ranked full-text search, get_case for full case detail, similar_failures to match an incident description against the corpus, and list_tags.

The lesson here was that you do not need a heavy framework. I implemented the transport directly against the 2025-03-26 streamable HTTP spec in stateless mode: a single POST /mcp endpoint that handles initialize, tools/list, and tools/call. No sessions, no Durable Objects, no auth, because the data is public and read-only. It runs on a Cloudflare Worker with the protocol routing in one file, the pure ranking logic in another (unit tested on its own), and a data layer that hits the live agentpostmortem.com API with a five-minute in-memory cache and falls back to a bundled dataset when offline. A light per-IP rate limit of 60 requests per minute keeps it polite. You can smoke test it with a single curl of tools/list, and add it to Claude Code with one claude mcp add --transport http command.

The separation that paid off: keeping the search and similarity ranking as pure functions meant I could test the interesting logic without standing up the transport at all. MCP protocol handling is boilerplate. Your actual value is in the tool implementations, so isolate them.

Scoping a server: not every client should see every tool

Public read-only data is the easy case. The hard case is a server that fronts a company's real systems. Bridgekit does exactly that: it exposes Shopify, Triple Whale, and Postgres to an AI stack, and three of its four tools are reads (shopify_orders, triplewhale_metrics, db_query against an allowlisted table) while one is a write (shopify_tag_order).

The design decision I care about most is that scope is enforced at discovery, not just at call time. Clients are configured in a secret as JSON, each with a name, a list of allowed tools, and an allowWrite flag. When a client calls tools/list, the server only advertises the tools that client is scoped for. A read-only client never even sees the write tool exists. Callers authenticate with a bearer key (or an x-bridgekit-key header), and every attempt is written to an append-only audit log. When a read-only key tries to call the write tool, the call is denied and the denial is logged.

Two things I would tell anyone building this kind of server. First, per-client tool filtering matters more than you expect, because an agent that cannot see a tool cannot be prompt-injected into calling it. Reducing the advertised surface is a security control, not just tidiness. Second, make it demoable safely: Bridgekit's read tools return clearly-labelled sample data when upstream credentials are not configured, so you can show the whole flow without wiring it to a live store.

Scanning a server: assume you got it wrong

After building two servers, I was convinced I would ship a bad tool eventually, and that most people ship them with no security review at all. So I wrote mcp-audit, a scanner and linter for MCP servers. It connects to a server over stdio or HTTP (or lints a static JSON manifest without executing anything, which is what you want for untrusted servers in code review), enumerates every tool, resource, and prompt, and runs 18 rules over that surface.

The rules cover the failure classes I kept worrying about: arbitrary command or shell execution tools (MCP002, critical), destructive tools with no confirmation argument (MCP001), probable prompt-injection text planted in a tool description (MCP020), secrets or system paths exposed as resources like a .env file (MCP030, critical), caller-controlled URL arguments that invite SSRF (MCP041), HTTP transport with no authentication (MCP040), and unconstrained input schemas that let the model pass anything anywhere. Each finding has a stable MCPxxx id, a severity, and a concrete remediation.

It is built for CI. It runs offline, is fully deterministic, and emits JSON and SARIF 2.1.0 so findings show up as annotations in GitHub code scanning. The process exits non-zero when any finding reaches the --fail-on threshold (default high), so a bad audit breaks the build. You can disable noisy rules, remap severities, or ignore specific locations through a .mcpauditrc file. Running it against my own servers is what turned "I think this is fine" into "the scanner agrees this is fine."

One honest caveat

mcp-audit is a static and structural analyzer. It reasons about the shape of your tools: their names, descriptions, and input schemas. It flags a tool named run_shell and a resource pointing at .env, but it cannot know that your innocently named update_record tool quietly runs raw SQL under the hood, because it never sees the implementation. Pattern-matching on descriptions also means it can miss a cleverly worded injection sink or flag a benign one. It narrows the surface an attacker can reach and catches the obvious, dangerous defaults, but it is a first line of defense, not a substitute for reading the code behind each tool.

The through-line

Building the transport is the least of it. The work that matters is deciding which tools exist, who is allowed to see them, and proving to yourself that none of them are a foot-gun before an agent finds out for you. All three projects are open source under github.com/royalpinto007: casebook-mcp, Bridgekit, and mcp-audit. If you are shipping an MCP server, at minimum run a scanner over it first.

Building a streaming investigation assistant that tool-calls an MCP server and cites case IDs

Royal Simpson Pinto — Fri, 21 Aug 2026 09:30:28 +0000

I keep a public registry of documented AI-agent failures called AgentPostmortem: real incidents where an agent got prompt-injected, deleted a database, or spun into a runaway loop, each written up as a case file with an ID like APM-0048. The registry is useful, but browsing it is a chore. If I want to know why refund agents get prompt-injected, I have to guess search terms, open cases, and read them one by one.

So I built casebook-chat: a chat interface that does the searching for me. You describe an incident or ask about a failure mode, and the assistant searches the live registry, pulls the relevant case files, and answers in plain language while citing the real case IDs it used. The whole thing runs as a single Cloudflare Worker.

The core idea

The registry already speaks MCP (Model Context Protocol). It exposes three tools over JSON-RPC at mcp.agentpostmortem.com/mcp. Rather than reimplement search or copy the data into a new database, I wanted the chat model to call those tools directly and ground its answers in whatever they returned.

The three tools map cleanly onto how you actually investigate an incident:

search_cases(query) runs a full-text search over the failure case files and returns ranked summaries with case IDs.
get_case(id) fetches one case in full: outcome, verified facts, unknowns, and lessons.
similar_failures(description) takes a plain-language description of an incident and matches it against known failures.

The model decides which to call and when. Ask "why do refund agents get prompt-injected?" and it searches. Say "my agent deleted a database" and it reaches for similar_failures. Once it has a case ID, it can pull the full detail with get_case.

How it works

The stack is deliberately small. The browser runs a Vite and React app using @ai-sdk/react's useChat, which POSTs the message history to /api/chat. That endpoint is a Hono route inside a Cloudflare Worker, and the same Worker also serves the static UI through Workers assets. One deploy, one origin, no separate backend.

Inside the route, I call the Vercel AI SDK's streamText against Groq's llama-3.3-70b-versatile, passing the three tools defined with the AI SDK tool() helper and zod input schemas. Each tool's execute is just a fetch to the MCP server wrapping the arguments in a JSON-RPC tools/call:

search_cases: tool({
  description: "Full-text search over documented AI-agent failure cases...",
  inputSchema: z.object({ query: z.string() }),
  execute: async ({ query }) => callMcpTool("search_cases", { query }),
}),

Because these are genuine failure investigations, the model often needs more than one hop: search, find a promising case, then fetch its full detail before answering. I allow multi-step tool chains but cap the loop with stopWhen: stepCountIs(5), so a model that keeps calling tools cannot run away and burn the whole budget.

Streaming with tool calls rendered inline

The nice part is the streaming. The result comes back through toUIMessageStreamResponse(), which turns the run into a UI message stream where tool inputs and outputs arrive as typed message parts, not just text. On the client, useChat exposes each assistant message as an ordered list of parts, and I render them in order: text parts become markdown, and anything whose type starts with tool- becomes an inline activity chip.

Each chip is a collapsible component that reads the part's state field to show where the call is. While the model is still forming or running the call (input-streaming, input-available), it shows a spinner and the word "running". When the output lands (output-available) it flips to a checkmark and a one-line summary; on failure (output-error) it shows a cross and the error text. Expand a chip and you see the exact arguments the model sent and the raw result it got back, pretty-printed. So the tool calls appear woven into the answer as it streams, in the order they actually happened, and you can audit every step.

Citations that mean something

Grounding is enforced through the system prompt, not wishful thinking. The investigator is told to answer only from what the tools returned, to cite the real case IDs it used, and never to invent case IDs or facts. If the registry returns nothing relevant, it is instructed to say so plainly and label any general-knowledge answer as such. It also calls tools silently instead of narrating "let me search the registry", so the visible answer stays clean while the chips carry the process.

The MCP client itself is built to never throw. It retries once on a network error or a 5xx, times out each attempt at around eight seconds, and on persistent failure returns a structured "the registry is temporarily unavailable" string that the model is told to relay to the user rather than crash on. It also handles the fact that the MCP server can answer either as JSON or as a text/event-stream, parsing the last JSON-RPC frame out of the SSE body when needed.

One honest limitation

The grounding is only as strong as the prompt. There is no post-hoc verification step that checks the case IDs in the final answer against the IDs the tools actually returned. The system prompt tells the model not to invent IDs, and in practice llama-3.3-70b on Groq follows that well, but "the model was instructed not to hallucinate" is a softer guarantee than "the app refuses to emit an ID the tools did not surface". If I wanted a hard guarantee, I would parse the assistant's cited IDs and cross-check them against the tool outputs before rendering. That is the honest gap between "cites case IDs" and "provably cannot fabricate a case ID".

Two smaller notes: inference runs on the Groq free tier, so rate limits and the occasional slowdown apply, and 429s are caught and surfaced as a friendly "the model is busy" message rather than an error. And the registry data is community-documented, so the answers are only as complete as the casebook behind them.

Closing

What I like about this build is how little glue it needed. An MCP server that already exposes the right tools, an AI SDK that streams typed tool parts, and a single Worker to host both halves. The result is a chat that does not just talk about agent failures but shows its work: every claim traceable to a case file you can open yourself.

Code is here: https://github.com/AgentPostmortem/casebook-chat

Making a knowledge base queryable by any agent over MCP

Royal Simpson Pinto — Wed, 19 Aug 2026 09:30:29 +0000

Every team debugging an AI-agent incident is quietly rediscovering a failure mode that someone else already documented. A support bot approves hundreds of fake refunds because a ticket told it to. A retry loop runs away and spams customers. A stale cache turns into a confident hallucination. These are not novel bugs. They are recurring shapes, and the knowledge about how they played out and what fixed them is usually locked in a blog post or a postmortem doc nobody thinks to search at 2am.

I built AgentPostmortem as a public registry of these documented failures. But a registry a human has to remember to visit is a registry that goes unused during the exact moment it matters: while an agent is actively investigating. So I built Casebook MCP: a remote MCP server that turns the registry into tools any agent can call. Now Claude Code, Cursor, or an agent built on the Claude Agent SDK can ask, mid-investigation, "has anything like this happened before?"

The core idea

Casebook exposes four tools over the Model Context Protocol:

search_cases(query, tag?) for ranked full-text search over the case files, with an optional tag filter.
get_case(id) for the full detail of one case: outcome narrative, verified facts, unknowns, and lessons.
similar_failures(description) which takes a free-text incident description and returns the closest documented failures by keyword overlap.
list_tags() for every failure-mode tag with its description.

The tool that earns its keep is similar_failures. You paste in what is actually happening ("our support bot was tricked by text in a ticket into approving refunds") and get back real precedents, ranked, with the shared keywords, the outcome, and the lessons attached. That is the difference between a lookup table and something an agent can reason against.

How it works

The whole thing runs as a single Cloudflare Worker. I chose to implement the MCP transport directly against the 2025-03-26 streamable HTTP spec in stateless mode rather than pull in a framework. There is one POST /mcp endpoint that handles initialize, tools/list, and tools/call. No sessions, no Durable Objects, no auth, because the data is public and read-only. That decision keeps the server a plain request/response function, which is exactly what a Worker is good at.

The dispatch layer is just a switch over JSON-RPC methods. Here is the shape of it:

async function dispatch(req: JsonRpcRequest): Promise<unknown | null> {
  switch (req.method) {
    case "initialize":
      return {
        protocolVersion: "2025-03-26",
        capabilities: { tools: {} },
        serverInfo: { name: "casebook-mcp", version: "0.1.0" },
        instructions:
          "Use similar_failures when investigating an incident, " +
          "search_cases for topical research, get_case for full detail.",
      };
    case "tools/list":
      return { tools: TOOLS };
    case "tools/call": {
      const name = String(req.params?.name ?? "");
      const args = (req.params?.arguments ?? {}) as Record<string, unknown>;
      return callTool(name, args);
    }
    default:
      if (req.method.startsWith("notifications/")) return null;
      throw { code: -32601, message: `Method not found: ${req.method}` };
  }
}

Notifications (JSON-RPC requests with no id) get swallowed and produce no response body, which is what the spec wants. The request handler also accepts batched arrays, so a client can pipeline initialize and tools/list in one POST.

The ranking logic lives in its own pure module with no I/O, which makes it trivial to unit test. Scoring is deliberately simple and legible: a query token hitting a case title weighs 3, a tag hit weighs 2, and a body hit weighs 1, with ties broken by case number for stable ordering.

export function scoreCase(c: CaseFile, tokens: string[]): number {
  if (tokens.length === 0) return 0;
  const title = new Set(tokenize(c.title));
  const tags = new Set(c.tags.flatMap((t) => tokenize(t)));
  const body = new Set(tokenize(caseText(c)));
  let score = 0;
  for (const tok of tokens) {
    if (title.has(tok)) score += 3;
    else if (tags.has(tok)) score += 2;
    else if (body.has(tok)) score += 1;
  }
  return score;
}

similar_failures uses a related but even simpler measure: tokenize the description, tokenize each case, and rank by the size of the keyword intersection. No embeddings, no vector store.

Data comes from the live public endpoints on agentpostmortem.com (/api/export for the corpus, /api/search for rich detail, /api/tags) behind a 5 minute in-memory cache. When the network is unavailable, it falls back to a bundled dataset of representative case files that ships in the repo and doubles as the deterministic fixture for tests. So the server keeps answering even offline, just from a smaller snapshot.

There is also a companion investigator agent (agent/investigate.ts) built on the Claude Agent SDK query() API. Given an incident description it connects to the MCP server, finds similar failures, pulls the top cases with get_case, and writes a postmortem-draft.md grounded in the documented lessons. It runs on your local Claude Code subscription auth, so no API key appears in the code, and it has a --dry-run mode that stubs the model but still exercises the MCP server end to end, which is what CI uses.

An honest limitation

The similarity matching is pure keyword overlap, not semantic search. If your incident description and a documented case describe the same failure in different vocabulary, for example one says "loop" and the other says "recursion", the overlap can miss it. That is a conscious tradeoff: keyword ranking is transparent, dependency-free, and fully unit-testable, and it keeps the Worker cold-start cheap with no vector database to stand up. For a corpus of curated, tagged case files it works well in practice, but I would not claim it generalizes to fuzzy paraphrase the way an embedding index would. Adding an optional embedding pass is the obvious next step if recall becomes the bottleneck.

The other honest note: the rate limit is a simple in-memory per-IP counter (60 requests per minute) scoped to a single Worker isolate. It resets when the isolate recycles. That is intentionally lightweight for a public read-only endpoint, not a hardened quota system.

Try it

Add it to Claude Code with a single command once it is running:

claude mcp add --transport http casebook http://localhost:8787/mcp

Then your agent can consult a growing body of real agent failures instead of rediscovering each one from scratch. Code, the four tools, the investigator agent, and the bundled dataset are all here: github.com/AgentPostmortem/casebook-mcp.