DEV Community: DOS AI

A check that cannot fail is not a check

DOS AI — Sun, 16 Aug 2026 16:30:19 +0000

Two of my guardrails reported clean for weeks. Neither of them was capable of reporting anything else.

That is a different failure from a broken check. A broken check throws an error and you go fix it. These returned the exact output you get when everything is fine, so I read them as confirmation and moved on.

The grep that searched for a string nobody has ever written

We have a house rule against em dashes in published text. To enforce it on myself I grepped my own diffs:

grep -c $'—\|–' file.md

Zero. Every time. The rule looked healthy for months.

On macOS, BSD grep in basic mode treats \| as two literal characters instead of alternation. So that pattern searches for one literal five-character sequence, —\|–, which appears in no file I will ever write. It returns 0 for a file stuffed with em dashes and 0 for an empty file. Those two zeros look identical.

The version that works:

grep -c -e '—' -e '–' file.md

The interesting part is not the escaping rule. It is that I treated a zero as evidence without once asking what a non-zero would have required.

The analytics tag with nobody to send the hit

Most web analytics snippets have a "defer" or "manual pageview" mode. You turn it on when your app is a SPA, because you want to control when a view is counted instead of letting the loader fire on script execution.

Turning it on has a second effect that is easy to forget. The loader stops sending the initial pageview too. From that moment the counter reports only what your code explicitly sends.

If your code never sends anything, the outside view is indistinguishable from a working install:

the script loads, 200
the counter appears as "installed" in the vendor dashboard
no console errors
no failed requests, because a request that is never made cannot fail

You get an empty dashboard, and an empty dashboard has a much more likely explanation available: nobody visited. That explanation is wrong and it is very comfortable.

The 20-second probe, using Yandex Metrica as the example, works the same way for most vendors:

open any page with ?_ym_debug=1
console must print: PageView. Counter <id>.

If that line is missing, nothing is being counted, whatever the dashboard says.

The fix is two lines and the comment above them matters more than the code:

// The counter runs deferred because this is a SPA and route changes are
// not document loads. Deferred means the loader sends NOTHING on its own.
// If you ever remove `defer`, remove this call in the same commit, or every
// view is counted twice.
export function trackPageView(url: string, referer?: string) {
  window.ym(COUNTER_ID, "hit", url, referer ? { referer } : undefined);
}

Two settings that must agree, living in two files, with no error on either side when they disagree. That is the shape of the problem.

What both of these have in common

Neither check had a failing case I had ever observed.

I had never seen the grep print a number greater than zero. I had never seen the analytics dashboard show a number at all. In both cases the absence was the expected output of the healthy state, so it carried no information, and I kept banking it as good news.

Three habits came out of this, and they are cheap:

Feed the check something bad on purpose. Before trusting a linter, a grep, a hook, or an alert, hand it input that must trip it. If you cannot make it fail, you do not have a check. You have a constant.

Be suspicious of zero specifically. Zero errors, zero matches, zero events. Every one of those is also what a disconnected pipe returns. A pipeline that reports "3 problems found, 3 fixed" tells you it ran. A pipeline that reports nothing tells you nothing.

Verify at the far end, not at the near end. "The script returns 200" is a fact about your server. "The console prints PageView" is a fact about the thing you actually wanted. They are not the same fact, and only the second one is worth anything.

The pattern generalises past these two. Alerts wired to a metric nobody emits. Retry logic behind a condition that is never true. A test asserting a mock. In each case the system reports health, because reporting health is the only thing it can do.

I write about the ugly parts of running an assistant product on top of WhatsApp and Telegram at dosai.pro. The public API and MCP server are documented at dosai.pro/docs/guide/08-developers/80-public-api.

Automating a phone number you do not own: four invariants

DOS AI — Sun, 16 Aug 2026 11:04:42 +0000

If your product connects to WhatsApp through the linked-device protocol, your code operates a phone number that belongs to someone else. A small business owner scans a QR code and hands you their working line, the one printed on their van and saved in every customer's contacts.

The cost of a mistake is not a 500 in your dashboard. It is a ban, and a ban is not something you can refund. Only the messenger lifts it, on the owner's appeal, on their timeline.

What surprises most teams is that the anti-fraud system does not only read outgoing text. It watches how your protocol client behaves. A number can be flagged after two messages in a week if the client around it looks wrong.

Below are four invariants I now treat as non-negotiable for any system of this kind. None of them are about content.

1. Reaction is allowed, initiative is not

Replying to an incoming message is fine. Everything your process starts on its own is suspect.

Never let automation tear down a live session, request a fresh QR, wipe stored credentials, or scan the server for numbers on a schedule. Those actions belong to an explicit human click in your UI, and nowhere else.

The dangerous code is always well-intentioned. It reads like this:

// looks caring, behaves like an unstable client
if (Date.now() - lastInbound > SILENCE_THRESHOLD) {
  await restartSession(sessionId);
}

The intent is "the channel has been quiet, let me reconnect just in case". From the outside, a client that re-authenticates a healthy session three times in a morning is indistinguishable from a compromised or emulated one. Silence is usually just a quiet Tuesday.

If you need a liveness check, make it one auth-safe probe per silence episode with a long cooldown, and log the outcome. Anything that can loop must not touch authentication.

2. Any bulk lookup must converge

Sooner or later you will want to resolve identifiers: map contacts, check which numbers exist, backfill a table after a protocol change. That loop talks to the messenger about people who are not your users.

Three properties make it acceptable:

it remembers negative answers, not only successful ones
it has a ceiling per pass
it has a minimum interval between passes

Remembering only successes is the classic bug. Every pass re-asks about the same unresolvable numbers, the query count never drops, and the loop runs forever at full volume.

The convergence probe is arithmetic. Count the lookups per day and compare consecutive days. Falling is healthy. Flat means the loop is asking the same questions forever, and from the provider's side a permanent high-volume stream of questions about third-party numbers looks exactly like scraping a contact database. That is a category they ban for on its own, no matter how few messages you send.

// the fix is one table, not a rewrite
await recordLookupResult(number, result); // including "not found"
const pending = await pickPending({ limit: CEILING, olderThan: MIN_INTERVAL });

3. A process restart is not a reason to touch every session

Deploys are when fleets die together. The container comes up, the boot path iterates over stored sessions, and every number reconnects inside the same second.

Stagger it. A dozen seconds between reconnects plus jitter, and no automatic connection at all for sessions with no stored credentials, since those can only produce a QR request that nobody asked for.

for (const [i, session] of sessions.entries()) {
  if (!session.hasStoredAuth) continue;      // QR is a human decision
  await sleep(i * 12_000 + Math.random() * 4_000);
  void connect(session);
}

4. There are no silent touches

Every disconnect, reconnect and re-auth gets a row in a durable log with a parsed reason, and that log has to outlive the process that writes it.

This sounds like ordinary hygiene until the day you need it. When a number gets flagged, you have one question to answer: what did our client do in the hours before. If your telemetry lived in container stdout, the answer is a guess, and guesses are how teams end up blaming the messenger for their own restart loop.

Write the events to a table. Mount the raw log on a volume. Archive it on rebuild. It costs nothing and it is the difference between a post-mortem and a shrug.

The test to run on your own code

Take any automation you have that touches a session, and ask: if this misfires ten times in a row on a live customer number, what does the provider see?

Ten replies to ten incoming messages, fine. Ten re-authentications, ten QR requests, or ten thousand queries about strangers' numbers, not fine. That question catches the whole class, and it catches it before someone loses their business line rather than after.

I build DOS AI, where these rules are enforced by the platform rather than left to the integrator. Happy to compare notes in the comments if you run a fleet of your own.

Putting an MCP server in front of a real product: four things I got wrong first

DOS AI — Sun, 16 Aug 2026 11:04:31 +0000

If you ship an MCP server for a product that already has a REST API, the tempting move is to wire the tools straight into your service layer. Skip the network hop, call the function, return the object. I did the opposite, and after a full audit of the surface I am glad I did. Here are the four decisions that turned out to matter, three of which I only got right on the second pass.

I build a platform where an AI assistant answers customers in WhatsApp and Telegram, and the owner manages projects, conversations and leads from a dashboard. The MCP server exposes a slice of that to AI agents.

1. Every tool calls your own HTTP route, not your service layer

This looks like pure waste. You are inside the same process, you have the client, you could call listConversations(projectId) directly and save 20ms.

Do not.

Our API keys carry a read-only flag, and the gate that enforces it reads the request method from a header that middleware sets. An in-process call has no request, no header, and therefore no gate. The same is true for role checks, project scoping and rate limits: all of them live on the route. Bypassing the route means reimplementing four security controls in a second place, where they will drift.

// tool definition, simplified
{
  name: "list_conversations",
  method: "GET",
  path: (args) => `/api/projects/${args.project_id}/conversations`,
  project: (args) => args.project_id,
}

The tool layer becomes a projection: build a path, forward auth, shape the result. Nothing else. When we later added a new permission rank, no MCP code changed.

2. A 200 response is not a successful operation

Our send-message route returns HTTP 200 with { ok: false, error: "channel refused" } when the messaging provider rejects the payload. The route did its job, so 200 is correct.

The first version of the tool reported that as success. An agent told a human "message sent" about a message that never left the building. That is the worst class of bug in an agent product, because the agent is confident and the user has no reason to check.

So tools carry an explicit failure predicate:

{
  name: "send_message",
  failWhen: (body) => body?.ok === false,
  failMessage: (body) => `Channel refused the message: ${body?.error ?? "unknown"}`,
}

Rule of thumb: if your route can express failure inside a 200 body, your tool layer must know about it. HTTP status is a transport signal, not a business one.

3. Unwrap your envelope, or your confirmations arrive empty

Most of our routes answer with an envelope: { data: ... }, and a couple with { success: true, data: ... }. Human clients unwrap it without thinking. The tool projection did not.

The result was subtle. update_prompt returned { ok: true } and nothing else, because the interesting part sat one level deeper and got dropped. The agent had no way to confirm what it had written, so it either stayed vague or invented the detail.

const body = await res.json();
const payload = tool.unwrap ? unwrap(body) : body;

Test this per tool. A single shared unwrap is not enough if some routes wrap and some do not, which is exactly the state most real APIs are in.

4. Idempotency keys must be built from meaning only

Our send tool deduplicates repeated calls. The first key looked like this:

const key = `${conversationId}:${text}:${Math.floor(Date.now() / 60_000)}`;

The minute bucket felt harmless. It is not. Two identical calls that straddle a bucket boundary produce two different keys, so the duplicate goes through. The failure is rare, non-deterministic and impossible to reproduce on demand, which is another way of saying it will happen in front of a customer.

const key = `${conversationId}:${sha256(text)}`;

If time belongs in the key, it should come from the domain (a booking slot, a billing period), never from the clock at call time.

The thing I decided on purpose: no destructive tools

There is no delete-project tool, no payment tool, no remove-member tool. Not because they are hard, but because an agent that can read a customer message can be talked into acting on it, and the blast radius of a confused delete is not recoverable by an apology.

A test enumerates the registry and fails if a tool name matches a destructive verb. That test exists so that a future me, in a hurry, has to argue with a red build instead of quietly adding one.

What I would tell someone starting today

Treat the MCP layer as a thin, dumb projection of an API you already trust. Every piece of intelligence you put in it is a piece of intelligence that now exists twice.

Check the three failure modes above with a real agent, not with curl. Curl reads a raw JSON body and you fill in the meaning yourself. An agent reads what your tool returns and reports it to a human as fact, so an empty confirmation or a false success becomes a lie with your name on it.

The spec of what we expose is public if you want to compare notes: dosai.pro/llms.txt. Happy to go into the auth details in the comments.

Your prompt is not a security boundary

DOS AI — Sun, 16 Aug 2026 03:56:19 +0000

If your AI agent owns tools with side effects, one question decides whether it
is safe to ship: what happens when the model confidently calls a money tool on
invented grounds.

This is a writeup of one mechanism that closes that hole, and of where the
mechanism stops working. The context is assistants that talk to real customers
in messengers and can do irreversible things: confirm a payment, issue an
invoice, book a slot, notify the business owner.

Why an instruction in the prompt is not a control

A line like "only confirm payment after you received the receipt" executes
with a probability, not with a guarantee. That is not a quality problem with
the model. It follows from the training objective: be helpful, agree with the
person in front of you.

The conversation goes like this. The customer writes "I already paid, I will
send the receipt later, please confirm". There is no receipt. The model sees a
polite persistent human, sees an instruction that contradicts him, and over a
long context it picks cooperation. It answers "payment confirmed" and calls
the tool.

For tone of voice, probabilistic execution is fine. For money it is not.

One more hope worth killing early: the tool config field that looks like a
predicate. Most function schemas carry something like trigger_type, and
ai_decides literally means "the model decides". That field controls when the
tool is offered, never under which facts the tool is allowed to fire.

A precondition is a declarative fact about the conversation

The idea is small. A function carries a list of facts that the executor checks
against the database before dispatch. Not "the model believes a receipt
exists", but "there is an inbound attachment in this conversation".

Stored as JSONB next to the function:

[
  { "type": "client_sent_media", "within_messages": 10, "media_kinds": ["image", "document"] },
  { "type": "lead_field_filled", "field": "phone" }
]

The type list is deliberately short and covers nearly every real requirement:

type FunctionPrecondition =
  | { type: "client_sent_media"; within_messages?: number; media_kinds?: string[] }
  | { type: "lead_field_filled"; field: string }
  | { type: "function_called_before"; name: string }
  | { type: "min_client_messages"; count: number };

client_sent_media scans the last N messages written by the customer rather
than the last N rows of the thread. An owner who configures "the last 10
messages" means ten customer replies, not ten rows half of which the bot wrote
itself. The window is capped by a constant so that within_messages: 100000
in a config cannot turn the check into a table scan.

function_called_before reads the event log and requires an earlier
successful call in the same conversation. That is how you build chains like
"verify identity first, then modify the booking".

Three implementation properties that carry the whole design

The check lives in exactly one place. It sits in the tool executor, after
argument validation and strictly before dispatch to any handler. Put it inside
the handlers instead and you fix the class one handler at a time, which means
the next money-touching tool ships without a guard.

A block is returned to the model as a tool error with a reason. Not a
silent refusal:

Blocked: the customer must have sent a image/document attachment in their
last 10 messages. This did NOT happen. Do not tell the customer it did.
Ask the customer for what is missing, then call this function again.

The difference matters more than it looks. After a silent refusal the model
assumes the call went through and keeps lying to the customer. An error with a
cause produces self correction inside the same round: the bot goes and asks
for the receipt.

The requirement is appended to the tool description, so the model sees it
before spending a call:

HARD REQUIREMENT: this function is blocked and will refuse to run unless the
customer must have sent a image/document attachment in their last 10 messages.
Do not claim the action happened until the call actually succeeds.

Fail open, on purpose

When the check itself throws, the call goes through. It is not blocked.

} catch (err) {
  logger.error("Precondition check failed, letting the call through", { ... });
}

Here is the reasoning. A precondition defends against model hallucination, not
against an attacker. An attacker has no reach into this layer at all: he
speaks to the bot in words, while the facts come from our own database. So the
failure mode should be chosen by cost. Blocking every function for every
customer because Postgres blinked means breaking live conversations (no
invoice, no booking, no answer) over a hypothesis. The failure goes loudly
into the log, and the decision falls back to the prompt, exactly as it was
before the guard existed.

If this were access control the choice would be the opposite, fail closed. It
is not access control, and pretending otherwise would be worse than having no
guard.

What it does not solve

It does not replace authorization, idempotency or rate limits. It answers one
question: is there a fact in this conversation without which the action makes
no sense.

It does not rescue a badly specified function. If your only guard is
min_client_messages: 2, you moved the problem one message down the road.

It costs nothing where it is not used. A function with an empty precondition
list issues zero queries, the branch returns on an empty array. That property
is what keeps the guard alive past the second release: a check that slows down
every conversation for the sake of one money flow gets removed by whoever is
on call.

The takeaway

An LLM in production behaves like a capable intern. Most of its calls are
good, and nobody lets an intern sign the cheques. Boundaries belong in code,
get verified against data, and get logged in a way that survives a restart.
The prompt owns the quality of the conversation, and nothing beyond it.

All of the above runs in the platform I build, DOS AI: AI assistants for
WhatsApp and Telegram with a built in CRM, configured in plain text. If you
are building your own, our REST API, webhooks and MCP server are public, so
you can plug your agent in and look at the contract from the outside. The
machine readable spec sits at https://dosai.pro/llms.txt and the code samples
are on GitHub: https://github.com/adsytd1/dosai-api

Happy to go deeper on failure modes in the comments.