Artifacts
- GitHub Repository with Cloudflare Workers code.
- JSON config of ElevenLabs Agent in GitHub Repository
Intro
I've been using Claude by voice more and more lately — it's just faster than typing when I'm thinking out loud. Last week it hit me: every agent I've actually built myself only takes text. No voice, nothing. So I went looking and found ElevenLabs. Their voice models are top-notch, and on top of that, they let you build full conversational agents, not just turn speech into text.
That was enough to make me want to actually build something instead of just reading the docs.
I paired it with Cloudflare, because I believe it's the fastest and easiest way to test a backend.
There's one problem that shows up in almost every voice-agent demo: you give the agent a tool that's actually useful, and now it's also a tool that can hurt you.
"Look up my order" is safe. "Refund my order" sounds almost the same but is a completely different kind of risk — if you wire both the same way because they feel similar, you've built a bot that can move money on a hallucination.
I like to surf, so I wanted this demo to be as meaningful as possible.
So here's what I built: a voice concierge for a fake surf shop.
It can check order status, answer questions about products, and open a refund case — but it can't actually finish a refund itself.
The stack is ElevenLabs Conversational AI + Cloudflare Workers.
Quick disclaimer: this uses real public product names from a real German surfboard brand, WAU Eco Surfboards, purely for demo realism. It is not affiliated with, endorsed by, or operated by them.
The architecture
Two pieces:
- The storefront + backend API — one Cloudflare Worker (Vite + TypeScript static assets, plus a D1 database) serving the storefront and embedded widget, and exposing the actual order-lookup and refund-case endpoints Surfie's tools call.
- The agent — an ElevenLabs Conversational AI agent ("Surfie") with a system prompt, a knowledge base, and two webhook tools.
visitor (browser load) caller (voice)
| |
| v
| +-------------------------+
| | ElevenLabs Agent |
| | "Surfie" |
| | prompt + workflow graph |
| | + knowledge base |
| +-------------------------+
| |
| tool calls over HTTPS
| +-------------+--------------+
| | |
| v v
| GET /orders/{id} POST /initiate-refund
| (no auth) (Bearer token)
| | |
| +-------------+--------------+
| |
v v
+------------------------------------------------------------------------+
| Cloudflare Worker (site.demolabs.fyi) |
| |
| +------------------------+ +----------------------------+ |
| | index.html | | index.ts | |
| | (storefront, embeds | | (Hono API) | |
| | the widget) | | | |
| +------------------------+ +----------------------------+ |
+------------------------------------------------------------------------+
|
v
+-------------------------+
| D1: wau-refunds |
| (refund cases) |
+-------------------------+
Setting up Cloudflare
I did all of this on Cloudflare's Free plan — Workers, D1, and a custom domain all fit inside the free tier, which makes it a genuinely cheap way to test out an architecture like this before committing to anything.
Onboard your domain. If your domain isn't already on Cloudflare, you need to add it as a zone first — see Cloudflare's Onboard a domain guide. I bought mine directly through Cloudflare, so it was already set up as a zone in my account with nothing extra to configure.
Create the Worker. I set mine up by hand the first time, before I had a repo to point at. Since then I've published the code, so you don't have to: Cloudflare's Deploy to Cloudflare button below clones the repo into your own GitHub account, provisions whatever Cloudflare resources it needs (the D1 database, in this case), and wires up CI/CD so every push after that auto-builds and deploys — no manual Worker setup required.
- Point it at your domain. Once it's deployed, go to the Worker's Settings → Domains & Routes → Add → Custom Domain and enter your domain (or subdomain). Cloudflare creates the DNS record for you.
-
Add your secret. Under Settings → Variables and Secrets → Add, set the type to
Secret, name it (I usedAPI_TOKEN), and paste in a token you generate yourself. This is the same value you'll paste into ElevenLabs later as a workspace secret — more on that in Wiring the tools, below.
The app itself
The whole thing is one Worker, built with Vite + vanilla TypeScript — no framework. index.html is the frontend: a static single-page site with the board grid and the embedded ElevenLabs widget. index.ts is the backend: a small Hono app defining the two API endpoints Surfie's tools actually call.
This is the part I like most about building on Cloudflare: one Worker serves both the marketing site and the API behind it. No separate backend deployment, no CORS dance between two origins for your own frontend — just one deploy for the whole thing.
wrangler.jsonc is where the bindings live: the D1 database that stores initiated refunds, plus the secret initiate-refund checks against (the secret itself never appears here — that's the point of a Secret binding, it only exists in the dashboard or via wrangler secret put):
{
"d1_databases": [
{
"binding": "DB_REFUNDS",
"database_name": "wau-refunds",
"database_id": "<your-d1-database-id>"
}
]
}
And the two endpoints themselves:
app.get('/orders/:orderId', (c) => {
const orderId = c.req.param('orderId').toUpperCase();
const order = ORDERS[orderId]; // a hardcoded lookup table for the demo
if (!order) {
return c.json({ order_id: orderId, status: 'not_found' });
}
return c.json({ order_id: orderId, ...order });
});
app.post(
'/initiate-refund',
bearerAuth({
verifyToken: (token, c) => token === c.env.API_TOKEN,
}),
async (c) => {
const body = await c.req.json().catch(() => null);
const orderId = body?.order_id;
if (typeof orderId !== 'string' || !orderId) {
return c.json({ error: 'order_id is required' }, 400);
}
const caseId = `REF-${crypto.randomUUID().slice(0, 8).toUpperCase()}`;
await c.env.DB_REFUNDS.prepare(
'INSERT INTO refunds (case_id, order_id, reason) VALUES (?, ?, ?)'
).bind(caseId, orderId, body?.reason ?? null).run();
return c.json({ status: 'refund_initiated', case_id: caseId, order_id: orderId });
},
);
Testing the endpoints directly
Before wiring either one into ElevenLabs, it's worth hitting them with curl. A real order comes back with shipping details:
curl https://site.demolabs.fyi/orders/WAU-77102
{
"order_id": "WAU-77102",
"status": "shipped",
"carrier": "DPD",
"tracking_number": "01778812340099887766",
"estimated_delivery": "2026-09-09",
"items": [
"Riverboard 5'6\" (B-Ware, Bav Boards)"
]
}
An order ID that doesn't exist doesn't error out, it just says so:
curl https://site.demolabs.fyi/orders/WAU-123456
{
"order_id": "WAU-123456",
"status": "not_found"
}
And a refund, with the right bearer token, opens a real case in D1:
curl -X POST https://site.demolabs.fyi/initiate-refund \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <token>" \
-d '{"order_id":"WAU-77102"}'
{
"status": "refund_initiated",
"case_id": "REF-B18C8D7F",
"order_id": "WAU-77102"
}
The storefront itself
No React, no Vue, nothing componentized — a widget embed and a couple of fetch() calls don't need a framework's state management. The board grid is generated from two arrays of real product names (river/wave-pool boards, ocean boards) mapped over a small color palette, since the point was authentic content without scraping real product photography:
const RIVER_BOARDS = ['GHOUL', 'THRIFTSHOP', 'THE C', 'WASP', 'TOMBSTONE', /* ... */];
const OCEAN_BOARDS = ['MAORI', 'LONGBOARD', 'PARTY FISH', 'SOUL GLIDER', /* ... */];
function renderBoardGrid(containerId: string, names: string[]) {
const container = document.getElementById(containerId);
if (!container) return;
container.innerHTML = names
.map((name, i) => `
<article class="board-card">
<div class="board-shape" style="--board-color: ${PALETTE[i % PALETTE.length]}"></div>
<h4>${name}</h4>
</article>`)
.join('');
}
Building the agent
- In the ElevenLabs dashboard, open Agents in the sidebar.
- Click + next to Agents to start a new one. This opens a "New agent" dialog — don't pick one of the sample cards (Personal Assistant, Business Agent) or Blank Agent; click Browse Templates at the top instead.
- Search for "Track" (or browse) and pick Order Status & Tracking — it handles order inquiries, tracking, delivery estimates, and basic returns out of the box, with two integrations already wired in.
- Under Knowledge Base in the sidebar, upload one markdown file per topic — about us, shipping & returns, products & pricing, ding repair, shop locations, and the "Test Me Baby" board-testing program — via Add Files. This is what the non-tool capabilities (company story, ding repair, shop locations, board testing, products and pricing) actually answer from.
I started from that template and then rewrote the system prompt from scratch, because the template locks you out of editing it directly — you have to fork it into a blank agent first if you want real control.
System Prompt can be change in Agent > System Prompt:
I followed ElevenLabs' recommended shape for the prompt — Personality, Environment, Tone, Goal, Flow, Guardrails, and a dedicated section for when to end the call — instead of one long block of instructions. The Goal section keeps Surfie to two things: order status lookups via a tool, and shipping/returns/B-Ware questions answered from the knowledge base. The two guardrail-shaped sections do a lot of the actual work:
# Guardrails
- Never invent an order status, tracking number, delivery date, or stock availability. Only state what the tool or knowledge base actually gives you.
- Never promise a refund, exchange, or exception to the return policy — that needs a human.
- Don't give board-size or skill-level recommendations as if they were firm advice; general orientation only ("that's listed as a beginner/intermediate board"), and point anyone who wants a real fitting to a human.
- If a board arrived damaged, or there's any freight claim or dispute, hand off to a person immediately rather than trying to resolve it.
- Keep it phone-call length, every turn.
# When to end the call
ALWAYS call the end_call tool (don't just say goodbye verbally) when:
- The caller says goodbye in any form ('thanks bye', 'I'm good', 'all set', 'no that's it')
- The caller explicitly asks to end the call
- Their question has been fully answered and they confirm they don't need anything else
Briefly acknowledge AND then call end_call. Verbal goodbye alone leaves the call open.
That second bullet is the one that matters most. It's not enough to build the refund tool correctly on the backend — the model also has to talk about it correctly, or a technically-safe system can still make a customer think a human doesn't need to look at anything.
The conversation is a graph, not just a prompt
The base prompt above isn't the whole story — underneath it, Surfie is wired as a small workflow. In the agent's Workflow tab I laid out six nodes, each able to layer its own instructions on top of the base prompt for just that step.
- Greeting — opens with the fixed first message, figures out whether this is an order question or a product question, and branches.
- Assess — looks up the order (or answers a product question straight from the knowledge base) and branches again: order status, warranty claim, or done.
-
Process Return / Process Warranty — the two branches. Process Warranty is the one that can actually call
initiate-refund, and only after re-confirming the order exists and was delivered. - Confirm — reads back what happened and any reference numbers, then moves to End.
- End — hangs up.
Each node's additional_prompt adds to the base prompt rather than replacing it. Greeting's, for example, is just:
Open warmly: 'Hi, this is Surfie... I can help you check on an order status or consult about our Surfboards, what is your order number or do you have a question about our Surfboards?' Capture identification. Identify intent: order number, or a question about products. Branch.
A node isn't limited to telling the model how to talk, though — it can also tell it which tool to reach for. Process Return, for instance, wires in order_status directly:
Capture: order number (format WAU-XXXXX). Never guess one.
- Call order_status with exactly that order number. Don't call it before you have a number.
IMPORTANT: If the customer signals they want to end the call, briefly acknowledge AND call end_call.
That's the part that makes this more than a flowchart drawn on top of one prompt — each step pulls in exactly the tool and instructions it needs, instead of the base prompt having to anticipate every tool call up front.
It's easy to assume a voice agent is "one prompt plus some tools." In practice, routing different parts of the conversation through different prompt slices means the warranty-specific caution (verify the order was actually delivered before offering a refund case) only shows up when it's relevant, instead of cluttering every turn of a plain order-status lookup. I ran every node on Gemini 3.5 Flash and it held up fine across all six — no need to reach for a bigger model just because the conversation now had more moving parts.
Wiring the tools: read vs. write get treated differently
Surfie has two webhook tools, and they are not configured the same way on purpose. To add either one, go to Tools in the sidebar and click Add tool:
order_status — GET /orders/{order_id}, no auth, open CORS. It's read-only and returns nothing sensitive beyond a shipping status, so there's no reason to gate it. Set Method to GET, the URL to https://site.demolabs.fyi/orders/{order_id}, and leave Authentication as None:
Because {order_id} is a path parameter, ElevenLabs asks how the LLM should fill it in. Set Value Type to LLM Prompt and describe the format in plain language — "this is order id in format WAU-XXXXX" — so the model extracts it from the conversation instead of guessing at the shape:
You can test the tool right from this screen before ever wiring it into the agent — see Setting up Cloudflare above for what a real and a not-found order actually return. This also matters for the prompt: its Flow section tells Surfie to handle a not_found status by asking the customer to double-check the number, not by panicking.
initiate-refund — POST /initiate-refund, requires Authorization: Bearer <token>. This is the one that opens a support case in a D1 table. Same Add tool flow, but Method is POST and, critically, Authentication is not None — it's set to a named auth connection (Token_API_WRITE) rather than a raw string, so the token itself never sits in a visible field on the tool config:
order_id here is a body parameter instead of a path parameter, but the same idea applies: mark it Required, set Value Type to LLM Prompt, and spell out the format so the model doesn't invent one:
A successful test creates a real case in D1 and hands back a case ID — see Setting up Cloudflare above for the exact response:
ElevenLabs lets you store that bearer token as a workspace secret and reference it by connection name in the tool config, so the raw value never sits in a visible field — the tool auth setting points at a named connection, not a string.
On the Worker side — shown in full back in Setting up Cloudflare — the check is Hono's bearerAuth middleware, gated on env.API_TOKEN. That's a Cloudflare Secret, not a plain variable — set once via the dashboard (or wrangler secret put), never checked into the repo, never shown again after you set it. The same string gets pasted into ElevenLabs' side as a workspace secret. Two independent places holding the same shared value, neither of which is source control.
A rejected call gets a 401 with a WWW-Authenticate header, which is a small thing but worth doing — it's the difference between a tool config that fails obviously during setup and one that fails silently in front of a customer.
The widget itself is two lines, dropped straight into the page body (change code in index.html of Cloudflare Worker to your agent-id):
<elevenlabs-convai id="surfie-widget" agent-id="<<insert-your-agent-id-here>>"></elevenlabs-convai>
<script src="https://unpkg.com/@elevenlabs/convai-widget-embed" async></script>
Note on Security
One gotcha worth flagging if you're trying this yourself: the widget needs a secure context for microphone access, so a plain file:// preview won't let voice input work. Test it via wrangler dev (localhost counts) or a real deploy — not by double-clicking the HTML file.
That embed snippet also exposes the agent's agent-id in plain text — anyone who copies those two lines onto their own page gets a fully working Surfie conversation, on my ElevenLabs usage. The fix is in the agent's Security tab: an Allowlist of hosts permitted to connect, plus a Fail when Origin header is missing toggle so a request with no Origin header at all doesn't sneak through. I added site.demolabs.fyi and turned that toggle on:
That's not the full list of widget-security settings worth knowing — this MindStudio writeup covers the rest in more depth than I will here.
What I'd add next
The natural next step is an MCP tool instead of a plain webhook, so Surfie could reach a broader set of backend actions through one connection rather than a hand-registered tool per endpoint — with the tradeoff that MCP tools need to be internet-reachable and currently don't mix with zero-retention/HIPAA workspace modes, which matters more once this stops being a demo. After that: a Twilio number so Surfie takes real phone calls instead of only the web widget, and a GET /refund-status/{case_id} tool so a customer can check on a case they already opened instead of only creating new ones.
Try it
Live at site.demolabs.fyi, clearly marked as an unofficial demo. Test order IDs if you want to see Surfie in action: WAU-10234, WAU-88213, WAU-55501, WAU-77102.
I built this evaluating ElevenLabs' agent platform properly rather than just reading the docs — the interesting decisions showed up in the tool config, the auth boundary, and how the conversation gets routed, not in the system prompt copy itself.
Top comments (0)