Building an AI receptionist for a small business: the call flow, the tools and the handoff

Most "AI receptionist" demos show a voice answering a phone and sounding human. That part is now the easy bit. What decides whether a plumber in Texas or a dental clinic in Ontario keeps using it after the first week is everything around the voice: what it is allowed to do, how it books, and how it hands a caller to a person.

This post walks through the architecture we use when we scope an AI receptionist for a small business, with the decisions you need to make before writing any code.

1. Start with the call log, not the model

Before choosing a speech or LLM provider, export a month of call data (most phone systems can give you call times, durations and outcomes) and listen to 20 to 30 recordings if the business already records calls lawfully.

Sort the calls into buckets:

• Answerable from a fixed source: opening hours, service area, prices that are published, parking, what to bring.
• Transactional: book, reschedule or cancel an appointment; take a quote request.
• Needs a human: complaints, emergencies, anything about money already owed, anything clinical or legal.

The first two buckets are what the AI receptionist handles. The third bucket defines your handoff rules. If the third bucket is more than half the calls, an AI receptionist is probably the wrong first project.

2. The pipeline

A typical inbound call goes through five stages:

Caller -> Telephony (SIP / phone number)
       -> Speech-to-text (streaming)
       -> Dialogue engine (LLM + tools + guard rails)
       -> Text-to-speech (streaming)
       -> Caller

Two practical notes:

• Latency budget. Callers notice a pause of more than about a second. Stream everything: partial transcripts into the model, partial model output into speech. Several providers now offer speech-to-speech models that collapse the middle stages; they are faster, but they make logging and guard rails harder, so check what you can inspect.
• Barge-in. People interrupt. Your stack has to stop speaking when the caller starts, and the dialogue state has to cope with half-finished sentences.

3. Tools, not free text

The model should never "promise" a booking in prose. Give it a small set of functions and make it call them:

[
  {"name": "find_slots", "parameters": {"service": "string", "date_from": "date", "date_to": "date"}},
  {"name": "book_slot", "parameters": {"slot_id": "string", "name": "string", "phone": "string"}},
  {"name": "lookup_faq", "parameters": {"question": "string"}},
  {"name": "transfer_to_human", "parameters": {"reason": "string", "urgency": "low|normal|urgent"}}
]

Rules that save you later:

• book_slot only accepts a slot_id that find_slots returned in the same call. No invented times.
• Read the booking back to the caller and get a "yes" before calling book_slot.
• lookup_faq answers only from the business's approved content. If there's no match, the model says it doesn't know and offers a callback.

4. The handoff is the product

Write the transfer rules as code, not as a hope in the system prompt:

• Keywords and intents that always transfer (emergency, complaint, "speak to a person").
• A maximum number of failed turns (two "I didn't catch that" in a row means transfer).
• Business hours: outside hours, take a message with a structured summary and send it to the right inbox or CRM.

When you transfer, pass a summary (caller name, number, reason, what was already tried) to the human. Nothing annoys a caller more than repeating everything.

5. Consent, disclosure and recording

This is where small-business projects get into trouble:

• Tell callers they are talking to an AI, at the start. It is the honest default, and it avoids confusion when the call is transferred.
• Recording and transcripts. Some US states, including California, require the consent of everyone on the call before recording. In Canada, PIPEDA expects you to tell callers a call is recorded and why. In the UK, recorded calls and transcripts are personal data under UK GDPR. Australia also has federal and state rules on recording calls. Play a short notice and keep only what you need.
• Outbound calls are a different matter. In February 2024 the US FCC ruled that AI-generated voices count as "artificial" voices under the TCPA, so outbound AI calls need prior express consent. Keep the receptionist inbound-only unless you have a lawyer involved.

6. Logging and review

Store, for each call: the transcript, tool calls with arguments and results, the transfer reason, and the outcome. Review a sample weekly with the business owner for the first month. Most improvements come from adding FAQ entries and tightening transfer rules, not from changing models.

7. A one-week pilot plan

1. Day 1: FAQ source agreed and loaded; tools wired to a test calendar.
2. Day 2–3: internal test calls with scripted edge cases (interruptions, accents, wrong dates, angry caller).
3. Day 4–5: live on overflow only (calls that would otherwise go to voicemail).
4. Day 6–7: review transcripts, fix FAQ gaps, decide whether to widen it to all calls.

Starting on overflow means the worst case is "slightly better than voicemail", which is a much easier conversation with the business owner.

FAQ

Can an AI receptionist replace a front-desk person?
For routine calls it can take a large share of the load, especially after hours. It should not replace the person who handles complaints, emergencies and anything that needs judgement.

Which calendar systems can it book into?
Anything with an API: Google Calendar, Microsoft 365, Calendly and most practice-management or field-service systems. If the system has no API, the receptionist takes a structured message instead.

What does it cost to run?
Running costs are per minute (telephony, speech and model usage) plus the build. Ask for an estimate based on your real call volume.

---

At Dzimark we build these for small businesses in the USA, UK, Canada and Australia. If you want to see how we scope them, our page on AI voice agents and AI receptionists has the details.