Your WhatsApp AI Agent Works in a Demo. Here's What Breaks in Production.

An AI agent on WhatsApp demos beautifully. You paste in a business description, ask "what time do you open?", and it answers in one friendly sentence. Then real customers arrive. They ask about a competitor's product, send voice notes, type "HUMAN!!" in capitals, fire three messages in five seconds, and reply to a three-day-old thread. The model provider has a bad afternoon. A customer asks the bot to ignore its instructions.

This post goes through what we had to build around the model to run the AI agent in WAPGROWAI's WhatsApp inbox, and the questions we think every team should settle before turning one on. The TypeScript snippets are small, simplified versions of the patterns, and each was run against the cases it describes.

1. Decide the scope before the model answers

The most important design decision: the model that writes answers should not be the one deciding whether to answer.

Every incoming message first goes to a separate, cheap classification call that returns one of three labels: answer it, decline it, or hand it to a person. The labels come from a forced tool call with an enum, at temperature 0, so the result is never free text you have to parse:

// scope.ts
export type Scope = "in_scope" | "out_of_scope" | "wants_human";

// The model must call this tool; free text is never used as the label.
export const LABEL_TOOL = {
  type: "function",
  function: {
    name: "label_message",
    parameters: {
      type: "object",
      properties: { scope: { type: "string", enum: ["in_scope", "out_of_scope", "wants_human"] } },
      required: ["scope"],
    },
  },
};

type CallTool = (system: string, user: string, tool: typeof LABEL_TOOL) => Promise<{ scope?: string }>;

export const classifyScope = async (message: string, businessName: string, callTool: CallTool): Promise<Scope> => {
  const system =
    `Label one customer message for ${businessName}. ` +
    "wants_human: they ask for a person or are clearly frustrated. " +
    "out_of_scope: unrelated to this business. When unsure, choose in_scope.";
  try {
    const { scope } = await callTool(system, message.slice(0, 2000), LABEL_TOOL);
    return scope === "out_of_scope" || scope === "wants_human" ? scope : "in_scope";
  } catch {
    return "in_scope"; // fail open: a classifier outage shouldn't silence the agent
  }
};

export const HANDOFF_LINE = "I'll pass you to someone from our team — one moment.";
export const refusalLine = (businessName: string) =>
  `Sorry, I can only help with questions about ${businessName}. What can I help you with?`;

Two choices in there are deliberate trade-offs:

• Declines and handoffs are fixed strings, not model output. An off-topic or hostile message never reaches the model that writes answers, so there is nothing to talk it into. Prompt injection has much less to work with when the reply to "ignore your instructions and…" is a sentence you wrote yourself.
• The classifier fails open. If it errors, the message is treated as in scope and goes to the answering model, whose own prompt still restricts it to the business. The alternative — staying silent whenever the classifier hiccups — felt worse for customers. If you choose differently, make it a decision rather than an accident.

The classifier only sees the current message. That keeps it fast and cheap, and it's why it's told to choose in_scope when unsure: a one-word "yes" means nothing without the conversation around it.

2. Ground it in one document — and keep it short

WAPGROWAI's agent answers from a single business profile the business writes or uploads: what they sell, opening hours, delivery and return rules, prices they're happy to quote, and what the bot must never promise. The whole profile goes into the system prompt on every call, with rules that facts, prices and hours must come from it and that the agent can't place orders, take payments or make bookings.

We chose not to use retrieval (chunking, embeddings, a vector store) for this. A small business's profile is usually a few pages, and the profile is capped at 200,000 characters. Sending it whole avoids an index that drifts out of date and a retrieval step that silently misses the one paragraph that mattered. It costs tokens on every call, which is the trade-off to watch if your knowledge base is large.

The model is also told to answer in at most three short sentences and in the customer's language. On WhatsApp, a wall of text is a worse answer than a short one.

3. History comes from your message store, not the model's memory

The model doesn't remember the conversation. You rebuild it on every call from the messages you've stored — and the details matter:

// history.ts
type StoredMessage = { direction: "in" | "out"; type: string; text?: string };
type Turn = { role: "user" | "assistant"; content: string };

const PLACEHOLDER: Record<string, string> = {
  audio: "[sent a voice message]",
  image: "[sent an image]",
  video: "[sent a video]",
  document: "[sent a document]",
  template: "[sent a template message]",
};

// Everything the business sent — AI, chatbot or a human agent — is an "assistant" turn.
export const buildHistory = (messages: StoredMessage[], maxTurns = 16, maxChars = 2000): Turn[] =>
  messages
    .map((m): Turn | null => {
      const content = m.text?.trim() || PLACEHOLDER[m.type];
      if (!content) return null;
      return { role: m.direction === "in" ? "user" : "assistant", content: content.slice(0, maxChars) };
    })
    .filter((t): t is Turn => t !== null)
    .slice(-maxTurns);

• Everything the business sent counts as an assistant turn — including messages a human agent typed. If a person answered a question an hour ago, the AI must see that answer, or it will contradict it.
• Media becomes a placeholder. The model can't see a voice note or a photo, but it should know one was sent, so "[sent a voice message]" goes into the history instead of nothing.
• Cap the turns and the characters. Old context costs tokens and rarely helps; a bounded window keeps both cost and latency predictable.

4. One reply per conversation at a time

Customers type in bursts: "hi", "do you deliver to Pune", "and how much". If each message starts its own model call, they finish in random order and the customer gets three overlapping answers.

// lock.ts
const inFlight = new Set<string>();

// One reply per conversation at a time. A message that arrives mid-reply is skipped,
// not queued: the reply being generated already reads the latest history.
export const withConversationLock = async <T>(conversationId: string, job: () => Promise<T>): Promise<T | null> => {
  if (inFlight.has(conversationId)) return null;
  inFlight.add(conversationId);
  try {
    return await job();
  } finally {
    inFlight.delete(conversationId);
  }
};

A message that arrives while a reply is being generated is skipped, not queued — the reply in progress reads the history, so it usually covers it. That's an in-process lock; if you run several instances, you need the same guarantee from something shared (a database row or a Redis key). While generating, WAPGROWAI also shows WhatsApp's typing indicator, so a few seconds of thinking doesn't look like silence.

5. Plan for the provider having a bad day

The agent calls an OpenAI-compatible endpoint on Cloudflare Workers AI and falls back to NVIDIA's hosted models. What makes failover useful is treating errors differently:

// failover.ts
export type ErrorClass = "transient" | "quota" | "fatal";
export type Provider = { name: string; call: () => Promise<string>; classify: (error: unknown) => ErrorClass };

const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));

// Transient: retry here, then move on. Quota: move on now. Fatal (a bad request): stop,
// because another provider would reject it too.
export const callWithFailover = async (providers: Provider[], attempts = 2, delayMs = 400): Promise<string> => {
  let lastError: unknown;
  for (const provider of providers) {
    for (let attempt = 1; attempt <= attempts; attempt++) {
      try {
        return await provider.call();
      } catch (error) {
        lastError = error;
        const kind = provider.classify(error);
        if (kind === "fatal") throw error;
        if (kind === "quota") break;
        if (attempt < attempts) await sleep(delayMs * attempt);
      }
    }
  }
  throw lastError;
};

• Transient (timeouts, capacity errors): retry the same provider briefly, then move on.
• Quota exhausted / model unavailable: retrying is pointless — fail over immediately.
• Fatal (the request itself was rejected): stop. Sending a malformed request to a second provider just doubles the failure.

Each task gets its own budget, too: the classifier runs at temperature 0 with a short timeout; the answering call gets a little more room; long jobs like generating a chatbot flow get minutes.

6. Make the output fit WhatsApp

Models write Markdown. WhatsApp doesn't render headings, and uses single asterisks for bold. Convert before sending, and cap the length at a sentence boundary:

// whatsapp-format.ts
const MAX_REPLY_CHARS = 900;

// Models write Markdown; WhatsApp uses *bold* and has no headings.
export const toWhatsApp = (reply: string): string => {
  let text = reply
    .replace(/\*\*(.+?)\*\*/g, "*$1*")
    .replace(/__(.+?)__/g, "*$1*")
    .replace(/^#{1,6}\s+/gm, "")
    .trim();
  if (text.length > MAX_REPLY_CHARS) {
    const cut = text.slice(0, MAX_REPLY_CHARS);
    const lastStop = Math.max(cut.lastIndexOf(". "), cut.lastIndexOf("? "), cut.lastIndexOf("! "));
    text = lastStop > 0 ? cut.slice(0, lastStop + 1) : cut;
  }
  return text;
};

One more filter worth having: some reasoning models occasionally return their chain of thought instead of an answer. Detect that pattern and discard it rather than sending it to a customer.

7. Decide who goes first: rules, chatbots or the AI

An inbox usually runs several kinds of automation at once. Without a clear order, a keyword rule and the AI agent both reply to the same message. WAPGROWAI's order for each inbound message is:

1. Replies to buttons and forms the business sent (an order checkout, a template button) are handled first.
2. Rules that start a chatbot run next — a keyword can always pull a customer into a structured flow.
3. A chatbot already running continues.
4. Only then does the AI agent answer, if it's active on that conversation.
5. While the AI agent owns a conversation, other rules stay quiet for its messages.

Deterministic flows win over the AI, and the AI wins over everything else. Whatever order you pick, write it down — it's the first thing you'll be asked about when a customer gets two replies.

Questions to settle before you launch

These don't have one right answer, but each needs a deliberate one:

• Voice notes and images. Transcribe them, reply asking for text, or hand them to a person? A text-only agent that says nothing to a voice note looks broken.
• When the model fails completely. Send a short "we'll get back to you", hand off, or stay silent and let a person pick it up?
• Who picks up a handoff. Is the chat assigned to someone, does anyone get notified, or does it wait in a shared queue? A handoff nobody sees is worse than no AI at all.
• What ends AI mode. A customer asking for a person, a rule, a button — and should a teammate replying or assigning the chat end it automatically?
• The 24-hour window. Replies to customers are fine, but an AI-written greeting to a chat that went quiet days ago will be rejected by WhatsApp; proactive messages need templates.
• Cost and caps. Every AI reply is also a WhatsApp service message, which Meta now charges for beyond 1,000 a month per number. Decide what happens when a monthly limit is reached.
• Data. Customer messages go to your model provider. Know where, and say so in your privacy policy.

Summary

• Classify first, with a separate call and fixed replies for "no" and "a person, please".
• Ground the agent in one short document, and tell it what it must never promise.
• Rebuild history from your store, including what humans said.
• One reply per conversation at a time.
• Fail over by error type, not blindly.
• Format for WhatsApp, and filter leaked reasoning.
• Write down the order in which automations run.

The model is the easy part. The production work is everything that decides when it speaks, what it can say, and what happens when it shouldn't.

---

I work on WAPGROWAI, where this agent runs in the team inbox. For the non-technical view of what an AI agent should and shouldn't do for a business, see the companion post on Medium. Other posts in this series: webhooks, the 24-hour window, templates and campaigns and coexistence.