Grocery delivery apps have a retention problem. Users download the app, order once, forget their password, and go back to the shop. The interface that every phone already has — a messaging app — never gets used for commerce because wiring up a conversational AI agent to a messaging channel is traditionally a stack of three or four vendors: a messaging gateway, an LLM provider, a transcription service, and a phone validation API.
This walkthrough builds FreshCart, a WhatsApp-based grocery delivery assistant powered by an AI agent named Mia. A customer messages a WhatsApp number, browses products, builds a cart, places an order, tracks delivery, handles substitutions, and reorders previous deliveries — all inside a single chat thread. Voice notes work too: say "add bananas, eggs, and sourdough" and Mia transcribes, parses, and adds each item to the cart.
The whole system runs on four Telnyx products through a single unified API — no third-party LLM, no external transcription service, no separate phone validation vendor.
The canonical code example lives in the Telnyx code examples repo:
https://github.com/team-telnyx/telnyx-code-examples/tree/main/whatsapp-grocery-delivery-nodejs
What This Example Builds
A Node.js Express server with a WhatsApp webhook handler, an agentic AI tool-calling loop, and a 3-panel real-time dashboard:
| Component | Purpose |
|---|---|
| WhatsApp webhook | Receives inbound messages (text, interactive replies, voice notes) |
| AI agent (Mia) | 12-tool agentic loop — browse, cart, order, track, substitute, reorder |
| Dashboard | Product catalog, WhatsApp conversation mirror, delivery pipeline |
| SSE stream | Real-time event push to the browser dashboard |
Why This Matters
WhatsApp has 2 billion monthly active users. For many markets — grocery, pharmacy, food delivery — WhatsApp is where customers already are. Building a full commerce experience inside WhatsApp means no app download, no account creation, no forgotten passwords.
The technical pattern here — an agentic tool-calling loop over a messaging channel — is the foundation for any conversational commerce application:
- Grocery and food delivery — Browse, order, track, substitute, reorder
- Pharmacy — Prescription refills, drug interaction checks, delivery scheduling
- Restaurant ordering — Menu browsing, customization, delivery tracking
- Retail — Product search, size/color selection, order tracking, returns
- Service booking — Appointment scheduling, rescheduling, cancellations
The system prompt and tool definitions are the only things that change.
Products Used
telnyx_products: [WhatsApp Business API, AI Inference, Number Lookup, Audio Transcription]
WhatsApp Business API — Sends and receives messages via POST /v2/messages/whatsapp. Supports text, interactive list messages (for product browsing), and quick-reply buttons (for substitution decisions and delivery confirmations).
AI Inference — Powers the agentic tool-calling loop via POST /v2/ai/chat/completions with OpenAI-compatible function calling. The model (GLM-5.3-Flash) receives the conversation history and tool definitions, decides which tool to call, and the server executes it.
Number Lookup — Validates and normalizes customer phone numbers at registration via GET /v2/number_lookup/{phone}. Returns E.164 format so the app never stores mismatched phone formats.
Audio Transcription — Transcribes WhatsApp voice notes via POST /v2/ai/audio/transcriptions using the Whisper model.
Architecture
WhatsApp Message (text / voice note / button tap)
|
v
+-----------------------+
| /webhooks/whatsapp | Express webhook handler
+----------+------------+
|
v
+-----------------------+
| AI Inference | Agentic tool-calling loop
| (GLM-5.3-Flash) | 12 tools available
+----------+------------+
|
+-------+-------+
| |
v v
[Tool execution] [WhatsApp reply]
- browse_products - sendText()
- add_to_cart - sendList()
- place_order - sendButtons()
- track_delivery
|
v
+-----------------------+
| SSE -> Dashboard | Real-time 3-panel view
+-----------------------+
The Tool-Calling Loop
The core of the AI agent is a loop that runs until the model returns a final text response instead of a tool call:
async function processInbound(db, ssePublish, fromNumber, text) {
const messages = await getConversationHistory(db, fromNumber);
messages.push({ role: "user", content: text });
for (let turn = 0; turn < MAX_TURNS; turn++) {
const response = await chatCompletion(messages);
const choice = response.choices[0];
if (choice.finish_reason === "tool_calls") {
for (const toolCall of choice.message.tool_calls) {
const result = await executeTool(db, ssePublish, fromNumber, toolCall);
messages.push({
role: "tool",
content: JSON.stringify(result),
tool_call_id: toolCall.id,
});
}
} else {
const reply = choice.message.content;
await sendText(fromNumber, reply);
await storeMessage(db, fromNumber, "assistant", reply);
break;
}
}
}
The 12 AI Tools
| Tool | Purpose | WhatsApp Output |
|---|---|---|
browse_products |
Browse by category or search | Interactive list message |
add_to_cart |
Add items (supports bulk from voice) | Text confirmation |
view_cart |
Show cart contents and total | Formatted text |
update_cart |
Change quantity or remove items | Text confirmation |
get_delivery_slots |
Available delivery windows | Interactive list |
place_order |
Checkout from cart | Text with order number |
track_delivery |
Delivery status and ETA | Status timeline |
reschedule_delivery |
Change delivery window | Quick-reply buttons |
handle_substitution |
Accept/reject replacements | Quick-reply buttons |
reorder |
Reorder past delivery items | Quick-reply buttons |
register_customer |
New customer + Number Lookup | Text |
send_whatsapp |
Send formatted message | Varies |
Voice Note Ordering
When a customer sends a WhatsApp voice note, the webhook detects audio and routes it to Telnyx Audio Transcription:
const resp = await fetch("https://api.telnyx.com/v2/ai/audio/transcriptions", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ url: audioUrl, language: "en", model: "whisper" }),
});
const data = await resp.json();
return data.text;
The transcription feeds into the AI agent as text. "Add bananas, eggs, and sourdough" becomes three separate add_to_cart calls.
Phone Number Validation
New customers register with their phone number. The server calls Telnyx Number Lookup to validate and normalize to E.164:
const nlResp = await fetch(
`https://api.telnyx.com/v2/number_lookup/${encodeURIComponent(phone)}`,
{ headers: { Authorization: `Bearer ${apiKey}` } }
);
const nlData = await nlResp.json();
if (nlData.data.valid_number === false) {
return res.status(422).json({ error: "Invalid phone number" });
}
const e164Phone = nlData.data.phone_number;
This prevents the common bug where local format (0626781895) and international format (+31626781895) create duplicate customer records.
Run It
git clone https://github.com/team-telnyx/telnyx-code-examples.git
cd telnyx-code-examples/whatsapp-grocery-delivery-nodejs
cp .env.example .env # add your keys
npm install
node server.js
The server starts on http://localhost:3000. Expose it for webhooks:
ngrok http 3000
Configure the webhook URL in the Telnyx Portal under your WhatsApp Messaging Profile -> Webhook URL -> https://<id>.ngrok.io/webhooks/whatsapp.
Open http://localhost:3000 in a browser to see the 3-panel dashboard. Register with your phone number, then message the WhatsApp number from your phone.
The Dashboard
The 3-panel dashboard updates in real time via Server-Sent Events:
Left — Product Catalog. Nine categories, 70+ products with prices and dietary tags.
Center — WhatsApp Phone Mockup. Live conversation mirror with typing indicators and tool activity.
Right — Delivery Pipeline. Order cards progressing through Confirmed -> Picking -> Packed -> Out for Delivery -> Delivered. An Advance button lets you manually push orders through stages, sending WhatsApp notifications at each step.
Going to Production
- Database — The app includes a Telnyx Edge SQL adapter for serverless deployment
- Webhook security — Add signature verification for inbound webhooks
- Conversation limits — Cap conversation history sent to AI Inference
- Error recovery — Send retry messages on inference failures
- Multi-instance — Move session state to Redis for horizontal scaling
Frequently Asked Questions
Do I need a WhatsApp Business Account? Yes. You need a Telnyx account connected to a WABA with an approved phone number and a messaging profile.
Can customers order items not in the catalog? Yes. The AI creates ad-hoc products when items aren't in the seed catalog.
How does voice note ordering work? Voice notes are transcribed via Telnyx Audio Transcription (Whisper), then fed to the AI agent as text. The agent parses items naturally and calls add_to_cart for each one.
Resources
- WhatsApp API docs
- AI Inference docs
- AI Inference function calling
- Number Lookup docs
- Audio Transcription docs
- Node.js SDK
- Telnyx Portal
- Full code example
If you build something with this pattern — pharmacy, restaurant, booking agent — let me know in the comments. The agentic loop is more general than the grocery example.
Top comments (0)