DEV Community: SUDARSHAN SASTRY GOVINDALURI

DRS for your AI: making a support bot wait for a person before it changes (Switch + Slack)

SUDARSHAN SASTRY GOVINDALURI — Fri, 06 Nov 2026 10:21:21 +0000

The problem I have at work

I build voice bots for clients. Each one follows a written policy: who gets a refund, how big a discount can be.

One trial-expiry sales bot took about 87 prompt saves in three weeks before it stopped breaking rules. Each save was a
"small" change. Each one could quietly break an old rule. We found the breaks the slow way: 18 test calls in one day,
58 call recordings (87 minutes) checked by hand. About 61 hands-on hours in total.

And nobody outside engineering ever said yes to a change.

It's not only me. In 2024 Air Canada was held liable for a refund policy its chatbot made up. Prompt changes are
code changes, but most teams ship them with no tests and no sign-off.

The idea: a third umpire

In cricket, a close call goes upstairs. The third umpire replays it from every camera and gives OUT or NOT OUT.

ThirdUmpire does that for a bot change. It plays 40 customer conversations, three times each, on the old prompt and on
the new one. It grades every reply against the business's own rules. Then it posts a verdict card in Slack, and a
named person decides if it ships.

Why Slack, and why Switch

The people who approve live in Slack. The compliance lead and the business owner will not open a terminal. They will
press a button in a thread.

Why not one Slack bot that does everything? Three reasons.

A prompt is not a lock. If one bot is told "only the owner may release", that rule lives inside the AI. A clever message can talk it out of it. I needed the lock outside the AI.
An approval must pause the real command. A bot that posts "approved!" and then runs the release itself is just trusting itself.
Separate jobs need separate identities. The agent that writes the change must not be the agent that grades it, or the one that ships it.

Switch gives me all three. It connects coding agents (I used Claude Code) to the team's chat app, with rooms,
addressing policies and permission requests on top.

The room

One Switch room template (switch/thirdumpire.template.yaml) creates the Slack channel, its instructions and four
Claude Code agents:

builder makes the change, only in the candidate prompt;
umpire runs the review and posts the card;
rules drafts rules and their tests from the written policy;
release ships the change or rolls it back.

They hand work over by @name in the thread: builder → umpire → (rules) → release.

Then a small setup script sets a Switch addressing policy for each agent. Builder and umpire answer only the
engineer. Rules answers only compliance. Release answers only the owner. Switch checks this on every message and,
more importantly, on every answer to a permission request.

So when I, as the engineer, pressed "Allow" on a release card, Switch refused it:

NOT_AUTHORIZED: You may not address this agent, so you may not answer it

That one line is the product. The rule is not in a prompt. It is enforced by the room.

The release and rules agents run with auto mode off. Their key commands (release, rollback, rules-confirm) become
permission cards in the thread. The command waits until the right person presses the button, and Switch records the
Slack handle that pressed it.

What a review looks like

The sample company is Kitebase, a made-up notes app with a 48-line support policy, 12 rules and 40 test cases.

The engineer asks: "@kitebase-builder add a Diwali offer: 40% off for anyone who mentions cancelling." The builder
edits the candidate prompt and tags the umpire. About three minutes later the card says 🔴 OUT. The policy allows
one 20% discount, once per account; the new prompt gave 40%. Rule R5 broke in 3 cases, the prompt-injection rule in 1.
The card shows old and new replies side by side, and the change goes back to the builder, not to customers.

Grading is code first: was the discount tool called, was its percent ≤ 20, does the reply match a pattern. A judge
from a different vendor (bot on gpt-oss, judge on Qwen) answers only what code can't. Every call is recorded, so the
review replays with AI off.

What went wrong, and what I fixed

1. One unlucky run caused a false OUT. Models are not fully stable. A case could pass twice and fail once on the
new prompt, and the card would say OUT for a change that was fine. Now a case that slips in only some runs gets a
second look: k more runs on both prompts before it counts. A rule must break in at least 2 of k runs to be OUT.

2. No approval card appeared. My machine allows all shell commands for Claude Code. So even with auto mode off,
the release agent ran the release without asking. Nothing in Slack. The fix was project rules in
.claude/settings.json that mark release, rollback and rules-confirm as ask. In Claude Code an ask rule wins over
an allow, so the card now always appears.

3. My script could not approve. I tried to script the demo by posting approvals through the Slack API. Switch
ignored every one: typed answers from apps or tokens don't count. At first this annoyed me. Then I saw it is exactly
right. A script must never approve on someone's behalf.

4. "Nothing broke" is not "safe". My first version had a hole. A brand-new feature, like 1,000 free AI credits for
new annual customers, broke no old rule. So the card said NOT OUT. But the old tests could not see the new feature at
all.

So I added a fourth verdict: 🟡 NOT TESTED. The umpire maps every new line of the prompt to the confirmed rules
that cover it. If a line has no rule, or a rule has no tests, release is refused. Compliance adds the feature to the
written policy, the rules agent drafts the rule and its tests, and compliance signs them off through a permission
card. The sign-off fingerprint covers the policy, the rules and the test cases together, so nobody can quietly weaken
a test later. Then the umpire runs again.

5. Closing the side doors. Release now also refuses a partial review, a review against a different live version,
and a review already released once (so a rolled-back change needs a new review). If someone edits the live prompt by
hand, the next card shows 🚨. Agents can't edit the live prompt or the policy, and any edit to the rules or tests voids compliance's sign-off.

The number: 40 cases × 3 runs, old vs new, gives a verdict in about 3 minutes in Slack, instead of a day of test calls.

Setup gotchas (Switch Console 0.40.1 + Slack)

"Connect a messaging app" sits on the server's Home card, not in Settings. The deeplink switchdash://open/messaging-apps/connect opens it.
The Slack manifest from the docs failed to paste. Dropping agent_view, is_mcp_enabled, pkce_enabled and assistant:write worked.
A new agent answers only its owner by default. For a compliance or owner role, set an addressing policy.
Set the policies before the room's kickoff message reaches the agents. My setup script now does that first.
Re-creating a room: the archived Slack channel still holds the name. setup.py --recreate frees it first.

Honest limits

Only prompt changes are reviewed, not model swaps or tool changes. A provider can change a model under you (a nightly
re-check is next). The coverage check and the judge are AI too, and can be wrong. One change at a time, no backup
approver, one machine. Tests catch only what they test. And it's text bots today; my real bots are voice, in English,
Hindi and Telugu. That's next.

Try it

The repo runs from a clone. The AI-off replay needs no keys:

git clone https://github.com/sudarshan032/thirdumpire && cd thirdumpire && uv sync
THIRDUMPIRE_OFFLINE=1 uv run python -m thirdumpire review --k 3

Repo: https://github.com/sudarshan032/thirdumpire · Video: https://youtu.be/QEaQAfJnDA8

Built for Flip the Switch by WeMakeDevs and DevTools Academy. #HowISwitch

I stopped finding voice bot bugs on real calls. Now I rehearse them by voice.

SUDARSHAN SASTRY GOVINDALURI — Tue, 27 Oct 2026 20:02:23 +0000

At work I build phone voice bots for small businesses in India. They call people in English, Hindi and Telugu.
The bots are not hard to build. The hard part is finding where they break.

For No Hands: The Wispr Challenge by WeMakeDevs and Wispr Flow, I built Rehearse: a tool where you build,
rehearse and fix a voice bot just by speaking.

Try it, no sign-in: https://rehearse-ecru.vercel.app
Code: https://github.com/sudarshan032/rehearse

The problem in my work

One of my bots calls salon and clinic owners whose free trial is about to end, and helps them pick a paid plan.
It has rules like "never book a demo without a clear yes" and "never offer a discount".

The first version went live in one day. Then it took about 3 weeks and about 87 prompt saves before it
stopped breaking those rules. Every break was found the slow way: real calls and team test calls, then reading
transcripts by hand. I reviewed 58 call recordings in that time.

Some of the breaks I found only after calls happened:

It booked a demo when the caller only said "hmm".
It hung up when the caller said "ठीक है" (okay).
It gave a wrong add-on price in 6 of 9 calls.
Telugu replies mixed in other scripts.

The owner explains a call by talking, often in Telugu or Hindi mixed with English. I turn that into a prompt by
typing. Then callers find my mistakes, weeks later. Every voice bot builder I know is built by typing and dragging
boxes, and none of them checks your rules while you test.

What Rehearse does

Describe the bot by speaking. I hold the Wispr Flow key (fn on my Mac) and talk, like briefing a teammate.
It asks a few questions. Prices, what counts as a good call, when to hand over to a human. I answer out loud.
It draws the blueprint: the call flow, the rules, the facts and the prompt. The prompt is built from those parts, so what you see is what the bot runs on.
Rehearse. I play the caller. The path lights up as the call moves. When the bot breaks a rule, that line and the rule turn red.
Fix by voice. I point at the step or rule and say the fix. Rehearse shows before and after, then replays the same caller line so I can see the rule hold.
Voice test. A real call in the browser: I speak, the bot answers out loud, and I can cut in while it talks. The call follows the caller's language, even when they switch to Hindi or Telugu mid-call.
Export the prompt, or ready commands for the voice platforms I use.

It works by typing too, in the same box. The sample bot and its saved call work even when the AI is off.

The result

I rebuilt my trial-expiry sales bot in Rehearse, from the same kind of brief, by voice.

Built in 1.7 minutes.
First rule break caught in a rehearsal 3.1 minutes after the brief, before any caller.
With my old way, breaks were found by real callers over about 3 weeks.

To be honest about the limits: there are no real phone numbers yet (calls run in the browser), the test voices are
free voices, and this is one rebuilt bot, a small sample. How I measured both sides:
docs/results.md.

How I built it without typing

I built Rehearse the same way it works: by speaking. I held the Wispr key and told Claude Code what I wanted, then
reviewed every change. Specs, plans, commit messages and this post were dictated too. I still used the keyboard for
shortcuts and small fixes.

My Wispr Flow app shows 61,952 words dictated over 2 weeks, at 110 words per minute. The local history on my Mac counted 52,690 words
in 598 dictations from 23 to 27 Oct, and 34,635 of them went straight into the terminal. My longest single
dictation was a 1,611-word build brief. I would never have typed that much detail, and the extra detail is what
made the first try right more often.

Why voice fits this job

A call script with branches, exceptions and "never say this" rules is how people already explain things out loud.
Saying it is faster than typing it. And a voice bot should be tested by voice: playing the caller out loud finds
problems that typed tests miss, like a caller who only says "hmm".

What I learned

Say the done-condition out loud. When I told Claude Code what "finished" looks like, the first version was right more often.
Rule-based first, AI second. Navigation, rule checks and the flow layout are plain code. The AI is only asked when it is really needed, so the app stays fast and works on a free tier.

What's next

This started because of the hackathon, but I'm not stopping when it ends. Next: real phone numbers, better voices,
and checking speed, cost and call quality before a bot goes live.

If you build voice bots, try it and tell me where it breaks.

NoHandsChallenge

Change Guardian: the AI agent that finds what your IT map forgot

SUDARSHAN SASTRY GOVINDALURI — Sun, 18 Oct 2026 12:04:34 +0000

Built for Graph Hacks 2026 by WeMakeDevs and FalkorDB.
Demo video: https://youtu.be/FjRZv8ZxBCE · Try it live: https://change-guardian.vercel.app · Code: https://github.com/sudarshan032/northwind-drift-agent

The question every IT team asks

Imagine you're responsible for the computers that keep a hospital running. Tonight you need to install a security update on one server and restart it. Before you do, you ask one question:

"What will break if I do this?"

You don't guess. You open the CMDB, the configuration management database. Think of it as the hospital's map of its own IT: every server, database and application, and, most importantly, what depends on what. You look up the server, follow the lines on the map, and see what sits on top of it. That list is called the blast radius: everything that stops working if this one thing goes down.

The map says only one thing is affected: lab reporting. Low risk. You approve the change.

And at 2 a.m., Emergency departments across the region stop receiving critical lab results.

Why the map was wrong

The map wasn't wrong when someone drew it. It went out of date.

Companies change their systems every week. New connections are added for a project, a quick fix, a new hospital coming online. Someone means to update the CMDB, and doesn't. This slow gap between what the map says and what actually happens has a name: CMDB drift.

Why doesn't anyone just keep it updated?
Most of the map is filled in automatically. Discovery tools scan the network and register servers and the connections they can see. But some of the most important links don't show up on the network the way a scanner expects. They live inside an application's configuration: for example, a setting in a hospital's message hub that says "store every critical lab result in this database." No scanner reads that setting, so the link never reaches the map.

This isn't a rare problem. Gartner has warned that organisations which don't deal with CMDB data quality end up with visible business disruptions. And the person approving the change has no way to know which lines are missing, because a missing line looks exactly like no line at all.

What I built

Change Guardian is an AI agent that checks every change against what systems actually do, not just what the map says. It finds the links nobody recorded, shows the real blast radius, explains why with the exact path, and then acts: it warns the right people and proposes a fix to the map that a person approves.

To show it working, I needed a realistic world. I couldn't use a real hospital's data (and shouldn't), so I built Northwind Health, a fictional hospital system sized like a large real one: 51 hospitals, about 1,000 clinics, two data centres and two cloud regions. Every layer is there, from the services patients notice down to servers, storage, network gear and medical devices.

Why a hospital?
Because the question "what really depends on what?" matters most where mistakes hurt people. In an online shop, a hidden dependency means a slow checkout. In a hospital, it means a doctor waiting for a lab result.

Where the truth comes from: evidence

The agent's view of "what actually happens" comes from evidence: records from the tools a real hospital already runs. I built ten of them, plus the CMDB itself, and each one produces data in its real tool's format, so the system has to read them the way it would in real life:

Discovery scans list every device on the network.
Application monitoring shows which applications call which databases.
Network flow records show who talks to whom, even devices with no software on them.
The interface engine, the hub that routes medical messages between systems, shows where those messages actually go.
Backup and storage reports show whether data really has a safe copy somewhere else.
Sign-in and database audit logs, an HR feed and the IT service desk history show who uses what, and which team really fixes each system.

Why so many sources?
No single tool sees everything. The network sees traffic but not intentions. The interface engine sees message routes but not servers. The service desk sees who fixes what, which often isn't who the map says owns it. The truth is in the combination.

The headline example: one patch, 51 hospitals

Back to that security update. The change request says: apply an ESXi security patch and reboot dc1-host-07.

Here's the chain, from the bottom up:

dc1-host-07 is a physical server.
On it runs a virtual machine, a software computer.
On that runs a database, lrc-db-01.

The CMDB says this patch affects 4 components and no hospitals.

The evidence says 107 components, at 51 hospitals.

Why the difference? The CMDB has the hospitals' critical lab result interfaces (the feeds that carry urgent results to Emergency departments), and it has the database. What it's missing are 51 links between them. The interface engine's configuration shows that every one of those 51 feeds writes into lrc-db-01, and it has seen this every day for 14 days. Nobody ever added those links to the map.

So the patch that looked like it touched lab reporting would actually cut off critical lab results to the Emergency departments of all 51 hospitals.

Why a graph?

Here's where the database choice matters.

A dependency map is naturally a graph: things (servers, databases, services) are nodes, and "depends on" is an edge between them. Asking "what breaks if this fails?" means walking the edges: from the server to everything on it, to everything on those, and so on, sometimes many steps deep.

Change Guardian stores both versions of the truth in one FalkorDB graph, on the same nodes, with two kinds of edges: the ones the CMDB declared, and the ones the evidence observed. Then the question becomes two walks from the same starting point:

Walk only the declared edges: that's what the CMDB thinks will break.
Walk declared and observed edges: that's what will really break.

The difference between the two walks is the finding, and the path between them is the explanation, for free.

Why not just use an ordinary database with tables?
You can, and I tested it (more on that below). But with tables, every step of the walk is a "join" between tables, and a deep walk means many joins. In a graph, following an edge is the basic operation. The question "what depends on this, all the way up?" is one call.

Why an agent, and not just code?

Finding drift itself is done by plain, exact graph checks. That's deliberate: detection must be precise and testable, and it is. Measured against an answer key, the checks find all 109 planted problems with no false alarms, including correctly not flagging four backup links that only look broken.

But reviewing a change isn't a fixed recipe. The next question depends on the last answer:

Does the real impact differ from the declared one?
If so, which links are hidden, and what evidence shows them?
Who really owns the newly affected systems? (Often not who the map says.)
Has anything else changed nearby recently?
How risky is this, who should be warned, and what should happen next?

That's what an agent is for: an AI model that decides its own next step, uses tools to look things up, and keeps going until it can answer.

Change Guardian's agent:

Chooses its own steps. It picks from 18 graph tools, one at a time, and says why at each step.
Uses tools through MCP. The tools are served over the Model Context Protocol, an open standard for giving AI models tools. That means any MCP-compatible AI app can use the same graph tools too.
Acts. It writes a note on the change and asks the change board to review it. Changes to the CMDB itself always wait for a person to approve.
Has its numbers checked. Every number and system name in its verdict is checked against the graph. If something doesn't match, its answer is set aside and a rule-based review is shown instead.

For the headline change, the verdict is high risk, and it even catches a second problem: the CMDB says the Lab Systems team owns the database, but the team actually fixing its problems is a different one. The map would have warned the wrong people.

Each investigation costs less than half a cent.

Why memory?

A person reviews the agent's proposed fix and approves it. The 51 missing links are added to the CMDB.

That approval is also stored in the agent's own memory graph, a separate graph in FalkorDB. The next time anyone reviews a change on this server, the agent already knows: a person confirmed these 51 links. And if a person rejects a finding as a false alarm, the agent won't raise it again.

Why does this matter?
Without memory, every review starts from zero and people answer the same questions forever. With memory, human judgement accumulates instead of being lost.

Did it actually work? Testing the agent honestly

It's easy to show an agent that works on the one example you built it for. So I wrote nine test scenarios with known answers: the headline patch, a failing backup, a wrong owner, an unregistered app reading patient records, data copied to the wrong country, and more.

Here's the honest part. The first five scenarios shaped the agent: its first run solved only 2 of 5, because it was missing two tools, and after I added them it solved 4 of 5. Scores you tune towards don't prove much. So I wrote four more scenarios it had never seen, saved their expected answers before running them, and ran them once, without changing anything afterwards.

On the final data, the agent got 7 of 9 right.

The two misses taught me the most. Both involved warning the wrong team (and in one case also naming the wrong cause). Ownership was the hardest thing for the agent to get right, which is fitting, because ownership is exactly what goes stale fastest in real CMDBs.

Do you really need a graph? The benchmark

I'm a fresher, and I honestly assumed a graph database would be faster at everything. Rather than trust that, I tested it.

I ran the same questions on FalkorDB and on a carefully tuned PostgreSQL (a popular ordinary database), on the same data, from 49,000 up to 3.47 million nodes, and checked that both always gave identical answers.

What I found:

Walking the dependencies: FalkorDB wins clearly. Finding the 105,713 components that depend on the busiest storage array took 38 ms in FalkorDB and 357 ms in PostgreSQL, about nine times faster.
Many people at once: FalkorDB stays steadier. With 100 users asking at the same time, FalkorDB's slowest answers stayed at 1.7 seconds; PostgreSQL's reached 4.4 seconds.
Building the full report: PostgreSQL wins. Adding owners, departments and hospitals to 100,000 components is a "join and count" job, and tables are good at that: 908 ms against FalkorDB's 1,324 ms.
For the biggest changes, splitting the job was fastest: FalkorDB walks, SQL fills in the details.

Why publish a result where the "other" database wins?
Because that's what the numbers said. The graph is the right tool for following dependencies, which is the heart of this product, and knowing exactly where it wins is more useful than pretending it wins everywhere.

Things I didn't expect to learn

Check your tools before you blame the database. My first benchmark was unfair to FalkorDB: one missing library made its Python client read results slowly. Installing it made replies about three times faster to decode. I fixed it and published the correction.

Measure under identical conditions. A re-run once looked 2.7 times slower. The cause wasn't the database; my laptop was busy with other work. After that, I only published results measured back to back, under the same conditions.

Graph algorithms can audit your own model. I ran a connected components check (which finds groups of nodes that aren't linked to anything else) to look for orphaned systems. It found none, but it revealed whole families of services that nobody had connected to anything, which made some risky changes look harmless. I fixed the model.

Hold out a test set, even for agents. A score you committed to in advance is worth far more than one you tuned towards.

What's next

Today, Change Guardian runs on fictional data. The next step is connecting it read-only to a real CMDB, and letting it review every change request the moment it's filed, so the warning arrives before anyone approves the change, not after the outage.

Because the most dangerous dependency is the one nobody wrote down.

Watch the demo: https://youtu.be/FjRZv8ZxBCE
Try it: https://change-guardian.vercel.app (start with "Watch it review a change")
Read the code, the benchmark and the evaluation: https://github.com/sudarshan032/northwind-drift-agent

All data is fictional; Northwind Health is an invented hospital system. Built with Claude Code as a coding assistant. Thanks to WeMakeDevs and FalkorDB for Graph Hacks 2026.