How to Build an AI Voice Agent That Handles Client Calls 24/7

Missed calls cost businesses money. A dental clinic missing 10 calls a day at $150 average ticket loses $1,500/day — $547K/year. That's not a staffing problem. That's a systems problem.

Here's exactly how to build an AI voice agent that answers, qualifies, and books appointments — no code needed for the basics, some JS for the advanced version.

What You're Building

An AI phone agent that:

• Answers calls in under 2 seconds, 24/7
• Speaks naturally (not robotic IVR)
• Qualifies callers (new vs. returning, urgency)
• Books appointments directly into your calendar
• Sends a WhatsApp confirmation after the call

Stack: Vapi (voice AI) + Cal.com (scheduling) + Twilio (phone number) + optionally a small Node.js webhook.

---

Step 1: Set Up Your Vapi Account

Vapi.ai handles the hard parts — real-time transcription, text-to-speech, interruption handling, and tool calling.

1. Create an account at vapi.ai
2. Go to Assistants → Create Assistant
3. Choose Phone Agent template
4. Set voice to Ava (ElevenLabs) or Nova (OpenAI) — these sound most human

{
  "name": "Riley",
  "voice": {
    "provider": "openai",
    "voiceId": "nova"
  },
  "model": {
    "provider": "openai",
    "model": "gpt-4o",
    "temperature": 0.7
  },
  "firstMessage": "Hi! Thanks for calling. This is Riley. How can I help you today?"
}

---

Step 2: Write the System Prompt

This is where most people go wrong. Generic prompts produce generic agents. Write it like a hiring brief for a new receptionist:

You are Riley, the virtual receptionist for [Business Name].

Your goals:
1. Greet warmly, identify if caller is new or existing client
2. Understand their need (appointment, question, emergency)
3. Book appointments using the bookAppointment tool
4. If emergency: give emergency line [phone number]
5. Always confirm name + callback number before ending call

Personality: warm, professional, efficient. Never robotic.
Do NOT: discuss pricing details, make promises outside your tools.

Business hours: Mon-Fri 9am-6pm, Sat 9am-2pm.
For after-hours: "We're closed right now but I can book you for tomorrow morning — does 9 AM work?"

---

Step 3: Connect Cal.com for Bookings

Vapi supports tool calling. Add a bookAppointment function that hits Cal.com's API:

// webhook.js — your Vapi tool endpoint
const express = require('express');
const app = express();

app.post('/book-appointment', async (req, res) => {
  const { name, phone, dateTime, serviceType } = req.body.message.toolCalls[0].function.arguments;

  // Cal.com API
  const booking = await fetch('https://api.cal.com/v2/bookings', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${process.env.CAL_API_KEY}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      eventTypeId: process.env.CAL_EVENT_TYPE_ID,
      start: dateTime,
      attendee: { name, email: `${phone}@sms.placeholder.com`, timeZone: 'America/Mexico_City' },
      metadata: { phone, serviceType }
    })
  }).then(r => r.json());

  // Send WhatsApp confirmation via Twilio
  await sendWhatsApp(phone, `✅ Appointment confirmed for ${dateTime}. See you then!`);

  res.json({
    results: [{ toolCallId: req.body.message.toolCalls[0].id, result: `Booked for ${dateTime}` }]
  });
});

Deploy this to Vercel in 2 minutes:

npx vercel --prod

---

Step 4: Get a Phone Number (Twilio)

1. Buy a local number in Twilio ($1/mo)
2. In Vapi: Phone Numbers → Import → Twilio
3. Paste your Twilio Account SID + Auth Token
4. Assign your assistant to the number

Done. That number now routes every call to your AI agent.

---

Step 5: Test Before You Go Live

Call your number. Run through these scenarios:

• [ ] New patient wanting an appointment → books correctly
• [ ] Caller asks about pricing → deflects gracefully
• [ ] After-hours call → offers next morning slot
• [ ] Caller says it's an emergency → gives emergency line
• [ ] Caller speaks over the agent (interruption handling) → recovers naturally

Fail any of these → tweak the system prompt before going live.

---

Real-World Numbers

I've deployed this for a dental clinic in Mexico City:

• Setup time: 4 hours (including Twilio onboarding)
• Monthly cost: ~$80/mo (Vapi credits + Twilio + Cal.com)
• Calls handled first week: 47
• Appointments booked autonomously: 31 (66%)
• Staff time saved: ~3 hours/day

At $150 average ticket value, 31 extra bookings/week = $4,650 additional revenue in week one.

---

The No-Code Version

If you don't want to write a single line of code:

1. Vapi — assistant setup (UI only)
2. Cal.com — built-in Vapi integration (no webhook needed for basic booking)
3. Twilio — phone number import
4. Zapier — WhatsApp confirmation via Twilio

Total setup: ~2 hours. No developers needed.

---

What This Unlocks

Once this is running, your business:

• Never misses a call again
• Books appointments outside business hours
• Handles call spikes without hold times
• Frees staff for high-value in-person work

The AI doesn't replace your team. It handles the 80% of calls that are routine, so your team can focus on the 20% that need a human.

---

Next Steps

If you want to implement this for your business (or build it for your clients as a service), rooxai.com has the full setup guide and a live demo you can call right now.

Or if you want us to build and manage it for you: rooxai.com/services/ai-receptionist

---

Questions about the Vapi setup or Cal.com integration? Drop them in the comments — I answer everything.