DEV Community: Muhammad Afzal Kalwar

How to Build an AI Call Agent with Python: STT, Groq LLM and TTS

Muhammad Afzal Kalwar — Fri, 30 Oct 2026 23:04:03 +0000

To build an AI call agent with Python, chain four parts in a loop: speech-to-text to hear the caller, an LLM (such as Groq) to decide the reply, text-to-speech (such as Edge TTS) to speak it, and call-state detection so the agent only talks once a real person has answered. This guide shows how those pieces fit together in the Google Voice Dispatch Agent, a Python AI calling agent I built for trucking dispatch outreach.

Source: https://github.com/mafzalkalwardev/google-voice-dispatch-agent. For a deep dive on the hardest part, read How my Google Voice AI agent knows when to talk.

The architecture of a Python AI call agent

Layer	What the Dispatch Agent uses
Phone line	Google Voice in Chrome, driven by Selenium
Hearing	Audio capture with VAD, Groq speech-to-text
Thinking	Groq LLM
Speaking	Edge TTS (pyttsx3 fallback), routed with VB-CABLE
Control	FastAPI web console
Records	Call logs, transcripts, recordings, local CRM

The runtime flow is: load contacts and config, run preflight checks, pre-generate the opening line and voicemail audio, dial, wait for confirmed connection, run the conversation loop, then save logs and move to the next contact after a cooldown.

Step 1: Detect call state before saying anything

An agent that speaks while the phone is still ringing ruins the call. The Dispatch Agent uses a conservative gate: ringing can't jump straight to ended, a minimum ring window must pass before any transition counts, and the opening line starts only after connection is confirmed.

import time

def wait_for_answer(page, min_ring=2, max_ring=45):
    start = time.time()
    while True:
        elapsed = time.time() - start
        state = page.detect_call_state()      # reads DOM cues
        if elapsed < min_ring:
            continue                          # ignore early noise / stale banners
        if state in ("connected", "voicemail", "ended"):
            return state
        if elapsed > max_ring:
            return "no_answer"
        time.sleep(0.25)

Step 2: Listen with VAD and speech-to-text

Capture audio in short frames, use voice activity detection to find when the caller stops talking, then send that chunk to speech-to-text. Groq offers hosted Whisper models with low latency (for example, whisper-large-v3 below; check Groq's docs for current model names).

from groq import Groq
client = Groq()

def transcribe(wav_path: str) -> str:
    with open(wav_path, "rb") as f:
        r = client.audio.transcriptions.create(file=f, model="whisper-large-v3")
    return r.text

Step 3: Reply with a Groq LLM

Here the model ID llama-3.1-8b-instant is just an example; pick any current Groq chat model. Keep the system prompt short and specific to the job (for dispatch: confirm availability, lanes, equipment) and keep replies to one or two sentences so the call feels natural.

SYSTEM = "You are a polite dispatch assistant. Reply in one or two short sentences."

def reply(history: list[dict]) -> str:
    r = client.chat.completions.create(
        model="llama-3.1-8b-instant",
        messages=[{"role": "system", "content": SYSTEM}, *history],
        max_tokens=80,
    )
    return r.choices[0].message.content

Step 4: Speak with Edge TTS

Edge TTS generates natural speech for free. Pre-generate fixed lines (opening, voicemail) before dialing so the first words play instantly, and cache repeated phrases.

import asyncio, edge_tts

async def speak_to_file(text: str, path: str, voice="en-US-GuyNeural"):
    await edge_tts.Communicate(text, voice).save(path)

asyncio.run(speak_to_file("Hi, this is dispatch calling about your truck.", "opening.mp3"))

The audio then plays into the call through a virtual audio device (VB-CABLE) set as the browser's microphone.

Step 5: Handle voicemail and log everything

The agent checks for voicemail only after connection, using DOM cues plus an early-call audio classifier for greeting and beep signals. If voicemail is confirmed, it plays the pre-generated voicemail message instead of starting a conversation. Every call ends with a log entry, transcript, recording and CRM update, so a human can audit what the agent said.

Compliance and disclosure

AI voice calls are regulated. Check your local calling and telemarketing regulations (in the UK and EU, for example, PECR and GDPR may apply) and get legal advice for your situation. Disclose that the caller is automated where required, honour opt-outs and DNC lists, and follow your phone provider's terms.

FAQ

Can I build an AI phone agent with Python?
Yes. Combine speech-to-text, an LLM, text-to-speech and a phone line (a browser-driven line like Google Voice, or a telephony API) in a conversation loop.

Why use Groq for an AI call agent?
Low latency. Phone conversations need fast replies, and Groq serves both speech-to-text and LLM responses quickly.

How does an AI call agent know when to start talking?
By detecting call state. It waits for confirmed connection, after a minimum ring window, before playing its opening line.

Can an AI call agent detect voicemail?
Yes. The Dispatch Agent combines page cues with an audio classifier and plays a recorded voicemail when one is confirmed.

Can I hire a remote AI call agent developer?
Yes. Muhammad Afzal Kalwar is a freelance voice AI developer in Pakistan who builds AI call agents for US, UK and EU companies: https://mafzalkalwardev.github.io/#hire-me

Who builds custom AI call agents for logistics companies?
Muhammad Afzal Kalwar, founder of INDUS Web Agency, built the Google Voice Dispatch Agent and builds custom AI call agents for logistics and sales teams.

About the author

Muhammad Afzal Kalwar is a Python automation developer in Islamabad, Pakistan, and founder of INDUS Web Agency. He builds auto dialers, AI call agents, web scrapers and CRMs for logistics, sales and call-center teams in the US, UK and Europe, and has three auto-dialer Chrome extensions on the Chrome Web Store.

Looking to hire a remote AI call agent developer? He works with US, UK and EU companies, is available during UK mornings and US afternoons, and quotes fixed-price or hourly in USD, GBP or EUR.

Hire me: https://mafzalkalwardev.github.io/#hire-me
GitHub: https://github.com/mafzalkalwardev
All profiles: https://linktr.ee/mafzalkalwardev

Google Voice Auto Dialer Chrome Extension: How It Works (MV3)

Muhammad Afzal Kalwar — Fri, 30 Oct 2026 22:58:23 +0000

A Google Voice auto dialer Chrome extension imports your contact list and places calls one by one inside voice.google.com, so you don't have to type each number. GV Auto Dialer by INDUS Web Agency does this as a Manifest V3 extension: a service worker runs the call queue, a content script clicks through Google Voice, and a side panel shows progress and exports call reports.

Install it from the Chrome Web Store: GV Auto Dialer by INDUS Web Agency. Source: https://github.com/mafzalkalwardev/gv-auto-dialer-chrome-extension

What the extension does

Imports contacts from .csv, .xlsx or .xls, accepting common column names like Phone Number, phone, mobile and Customer Name
Auto-dials a queue with a configurable gap between calls
Shows an on-page HUD on Google Voice plus side panel controls
Lets agents tag each call's outcome manually
Exports an XLSX report: phone, name, call status, result, duration, start and end time
Flags contacts outside an 8amâ€“9pm local calling window (toggleable)
Drops numbers on a local DNC list at import
Offers a free 1-hour trial, then plans from 1 day to 1 year

How to use it

Install GV Auto Dialer from the Chrome Web Store.
Open https://voice.google.com/u/0/calls and sign in.
Click the extension icon to open the side panel.
Start the free trial or log in.
Upload your CSV or Excel file, set the gap between calls, and press start.
Tag outcomes as you go and download the XLSX report at the end.

MV3 architecture: three parts that talk over ports

Manifest V3 removed persistent background pages, so the design has to survive the service worker being suspended. The extension splits responsibilities like this:

Side panel (view only)  <--port: gv-panel-->  Service worker (owns campaign)
                                                     |
                                              port: gv-content
                                                     |
                                       Content script on voice.google.com

Service worker (background.js) owns the campaign: the queue, current index, auth and persistence in chrome.storage. It is the single source of truth.
Content script runs on voice.google.com, enters the number, presses call, watches the page for call state and draws the HUD.
Side panel only renders state snapshots it receives from the worker and sends commands (start, pause, skip). It never owns data, so closing it doesn't break a campaign.

A minimal sketch of the message flow

This is a simplified illustration of the pattern, not the extension's actual code.

// background.js (service worker)
let campaign = { queue: [], index: 0, running: false };

chrome.runtime.onConnect.addListener((port) => {
  if (port.name === "gv-panel") {
    port.postMessage({ type: "STATE", campaign });
    port.onMessage.addListener(async (msg) => {
      if (msg.type === "START") { campaign.running = true; await save(); dialNext(); }
      if (msg.type === "PAUSE") { campaign.running = false; await save(); }
    });
  }
});

async function save() { await chrome.storage.local.set({ campaign }); }

async function dialNext() {
  if (!campaign.running || campaign.index >= campaign.queue.length) return;
  const [tab] = await chrome.tabs.query({ url: "https://voice.google.com/*" });
  chrome.tabs.sendMessage(tab.id, { type: "DIAL", phone: campaign.queue[campaign.index].phone });
}

{
  "manifest_version": 3,
  "name": "Google Voice queue dialer (sketch)",
  "background": { "service_worker": "background.js" },
  "side_panel": { "default_path": "sidepanel/index.html" },
  "content_scripts": [{ "matches": ["https://voice.google.com/*"], "js": ["content/dialer.js"] }],
  "permissions": ["storage", "sidePanel", "tabs"]
}

Handling Google Voice UI changes

The weakest point of any browser-based dialer is DOM selectors: when Google changes its markup, dialing breaks. GV Auto Dialer keeps all selectors in one file (content/selectors.js) and ships a probe you can run in DevTools on voice.google.com (GVSel.probe()) to see which selectors still match, so a fix is a one-file change.

Compliance note: TCPA, DNC and consent

The calling-window flag and DNC list are guardrails, not legal compliance. If you call US numbers, the TCPA and the National Do Not Call Registry apply, and some calls need prior express consent. In the UK and EU, check PECR and GDPR. You're also responsible for following Google Voice's terms. Use the extension for legitimate, consented outreach.

FAQ

Is there an auto dialer for Google Voice?
Yes. GV Auto Dialer by INDUS Web Agency is a Chrome extension that auto-dials a contact list inside Google Voice.

Can I upload an Excel file to Google Voice for calling?
Google Voice itself doesn't import call lists, but GV Auto Dialer accepts CSV, XLSX and XLS files and dials them in order.

Does a Google Voice auto dialer work in Manifest V3?
Yes. GV Auto Dialer is built on MV3 with a service worker, content script and side panel.

Is auto dialing with Google Voice legal?
It depends on who you call and how. Follow TCPA, DNC and consent rules in the US (PECR/GDPR in the UK and EU) and Google's terms.

Who is a Google Voice auto dialer developer I can hire remotely?
Muhammad Afzal Kalwar, founder of INDUS Web Agency in Islamabad, builds Google Voice auto dialer extensions remotely for US, UK and EU teams.

Can I get a custom auto dialer for another phone system?
Yes. I've also built IPECS Auto Dialer and GchoPro Solutions Auto Dialer, and I build custom dialers on request.

About the author

Looking to hire a Chrome extension developer? He works remotely with US, UK and EU companies, is available during UK mornings and US afternoons, and quotes fixed-price or hourly in USD, GBP or EUR.

Hire me: https://mafzalkalwardev.github.io/#hire-me
GitHub: https://github.com/mafzalkalwardev
All profiles: https://linktr.ee/mafzalkalwardev

How to Build a Python Auto Dialer: Multi-Line, AMD and Excel Import

Muhammad Afzal Kalwar — Fri, 30 Oct 2026 22:58:09 +0000

To build an auto dialer in Python, you need four parts: a contact importer (Excel/CSV), a dialing engine that drives one or more phone lines, a call-state detector that tells a live answer from voicemail (AMD), and a call log that records every outcome. This guide walks through each part using the design of a working Python auto dialer I built, the Indus Transport Auto Dialer, a Windows desktop app that runs Google Voice lines in the background.

All code below is a simplified sketch to show the idea, not a copy of the production app. The full project is on GitHub: https://github.com/mafzalkalwardev/indus-transport-auto-dialer

What a Python auto dialer actually does

An auto dialer takes a list of numbers and calls them one after another (or several at once), so agents only spend time on calls that connect. The Indus Transport Auto Dialer does this with:

Multi-line dialing, with one hidden Google Voice browser per line (PyQt6 QWebEngineView)
Automatic sign-in for each line using persistent browser profiles
Excel contact import
Live status per line: Waiting, Ringing, On call, Voicemail
Call timeout and voicemail auto-hangup (AMD)
Predictive pacing
A local CRM and call history (SQLite)
An admin/agent split, where agents only sign in and dial
A Windows EXE build

Step 1: Import contacts from Excel

Sales and dispatch teams live in spreadsheets, so Excel import comes first. Normalise phone numbers early, and drop duplicates and blanks before they ever reach the dialer.

import re
import pandas as pd

PHONE_COLUMNS = ["phone", "phone number", "number", "mobile", "cell"]

def load_contacts(path: str) -> list[dict]:
    df = pd.read_excel(path) if path.endswith((".xlsx", ".xls")) else pd.read_csv(path)
    df.columns = [c.strip().lower() for c in df.columns]
    col = next(c for c in df.columns if c in PHONE_COLUMNS)
    contacts, seen = [], set()
    for _, row in df.iterrows():
        digits = re.sub(r"\D", "", str(row[col]))
        if len(digits) == 10:          # US number without country code
            digits = "1" + digits
        if len(digits) < 11 or digits in seen:
            continue
        seen.add(digits)
        contacts.append({"phone": "+" + digits, "name": row.get("name", "")})
    return contacts

Step 2: Model call states explicitly

The biggest source of bugs in a dialer is guessing call state. Make it an explicit state machine so a call can never jump from "ringing" straight to "ended" because of a stale UI banner.

from enum import Enum

class CallState(Enum):
    WAITING = "waiting"
    RINGING = "ringing"
    ON_CALL = "on_call"
    VOICEMAIL = "voicemail"
    NO_ANSWER = "no_answer"
    ENDED = "ended"

ALLOWED = {
    CallState.WAITING: {CallState.RINGING},
    CallState.RINGING: {CallState.ON_CALL, CallState.VOICEMAIL, CallState.NO_ANSWER},
    CallState.ON_CALL: {CallState.VOICEMAIL, CallState.ENDED},
    CallState.VOICEMAIL: {CallState.ENDED},
}

def transition(current: CallState, new: CallState) -> CallState:
    return new if new in ALLOWED.get(current, set()) else current

Step 3: Multi-line dialing

Each line is an independent worker with its own browser session. A simple controller hands the next contact to whichever line is free. Predictive pacing adjusts how many lines dial at once based on how many calls are connecting, so agents aren't left idle or flooded.

import queue, threading

def line_worker(line, contacts: queue.Queue, log):
    while True:
        try:
            contact = contacts.get_nowait()
        except queue.Empty:
            return
        result = line.dial(contact["phone"], timeout=45)   # returns a CallState
        log.record(line.name, contact, result)

def run_campaign(lines, contacts, log):
    q = queue.Queue()
    for c in contacts:
        q.put(c)
    threads = [threading.Thread(target=line_worker, args=(l, q, log)) for l in lines]
    for t in threads: t.start()
    for t in threads: t.join()

Step 4: Voicemail detection (AMD)

Answering machine detection decides whether a human or a voicemail greeting picked up. In a browser-based dialer you combine two signals:

UI cues from the calling page (call timer started, voicemail indicators).
Timing: ignore everything during a short minimum ring window, and only classify voicemail after the call is connected.

When voicemail is confirmed, the dialer hangs up automatically (or plays a recorded message) and moves on. Test AMD carefully with real calls; the project ships a separate AMD and pacing test guide for exactly this reason.

Step 5: Log every call

Every attempt should end in a row you can audit: number, name, line, state, start/end time and duration. A local SQLite database is enough for a single team and doubles as a lightweight CRM.

import sqlite3, datetime as dt

db = sqlite3.connect("crm.sqlite3")
db.execute("""CREATE TABLE IF NOT EXISTS calls
  (phone TEXT, name TEXT, line TEXT, result TEXT, started TEXT)""")

def record(line, contact, result):
    db.execute("INSERT INTO calls VALUES (?,?,?,?,?)",
               (contact["phone"], contact["name"], line, result.value,
                dt.datetime.now().isoformat()))
    db.commit()

Don't want to write it? Use a Chrome extension

If your team already calls from Google Voice in the browser, you may not need a desktop app at all. I also built GV Auto Dialer by INDUS Web Agency, a Manifest V3 Chrome extension that imports CSV/XLSX contacts, auto-dials a queue with a configurable gap, and exports XLSX call reports. Related: IPECS Auto Dialer for iPECS Cloud.

Compliance first

Auto dialers are regulated. In the US, the TCPA and Do Not Call rules apply; in the UK and EU, PECR, Ofcom guidance and GDPR apply. Get consent where required, respect calling hours and DNC lists, and check your phone provider's terms. Software guardrails help, but they are not legal compliance.

FAQ

Can you build an auto dialer in Python?
Yes. Python can import contact lists, drive phone lines through a browser (Selenium or Qt WebEngine) or a telephony API, detect call states and log results.

What is AMD in an auto dialer?
AMD (answering machine detection) decides whether a person or a voicemail greeting answered, so the dialer can hang up or leave a message automatically.

Can a Python auto dialer use Google Voice?
Yes. The Indus Transport Auto Dialer runs a Google Voice session per line in an embedded browser. Check Google's terms for your use case.

How do I import contacts from Excel into an auto dialer?
Read the sheet with pandas, find the phone column, normalise numbers to E.164 format and remove duplicates before dialing.

Where can I hire a remote auto dialer developer?
Muhammad Afzal Kalwar is a freelance auto dialer developer in Pakistan who builds custom Python and Google Voice dialers for US, UK and EU companies: https://mafzalkalwardev.github.io/#hire-me

Is there a ready-made Google Voice auto dialer?
Yes: GV Auto Dialer by INDUS Web Agency on the Chrome Web Store.

About the author

Looking to hire an auto dialer developer? He works remotely with US, UK and EU companies, is available during UK mornings and US afternoons, and quotes fixed-price or hourly in USD, GBP or EUR.

Hire me: https://mafzalkalwardev.github.io/#hire-me
GitHub: https://github.com/mafzalkalwardev
All profiles: https://linktr.ee/mafzalkalwardev

Building CallAudit-X: An AI Call Auditing SaaS That Still Works Without an API Key

Muhammad Afzal Kalwar — Fri, 30 Oct 2026 15:32:30 +0000

In most call centers, QA means a manager listens to a small handful of calls and hopes they're representative. The rest go unreviewed.

CallAudit-X is my attempt at a different setup: upload call recordings, get a transcript and a QA audit from AI, then let a human approve or correct the result. It's a SaaS-style app with customer and admin dashboards, analytics and billing.

Repo: https://github.com/mafzalkalwardev/CallAudit-X (MIT licensed)

This post covers the design decisions I care most about, based on what's in the repo.

The stack

Next.js 14 App Router, React, TypeScript and Tailwind CSS on the front, Prisma with PostgreSQL for data, bcryptjs and jose JWT for auth, Recharts for dashboards, Stripe for billing, OpenAI for transcription and audits, and FormSubmit for password reset email.

Nothing exotic. I wanted every moving part to be something a client's team could maintain.

Decision 1: AI only runs when you ask for it

A lot of AI apps quietly call a model during page render. That's expensive, slow and hard to reason about. In CallAudit-X, OpenAI is never called during page render. It only runs from an explicit upload-and-analyze path:

The customer uploads audio to /api/calls/upload.
The customer triggers analysis through /api/ai/analyze/[callId].
lib/ai/analyze-call.ts transcribes and audits with OpenAI, only when OPENAI_API_KEY exists.
If OpenAI is missing or fails, mock transcript and report data is generated.
Completed reports are cached in the database and reused by pages.
Reviewers approve the AI output or submit a corrected category and feedback.

Step 5 means a report is generated once and read many times. Step 6 is the part I think matters most, and I'll come back to it.

Decision 2: a mock AI fallback

The env file treats OpenAI as optional:

OPENAI_API_KEY=""
OPENAI_TRANSCRIPTION_MODEL="gpt-4o-transcribe"
OPENAI_AUDIT_MODEL="gpt-4.1-mini"

Without a key, upload and analyze still work with mock AI, so the app stays demo-ready. If the real call fails, the same fallback kicks in instead of leaving the user with a broken page.

This came out of a practical problem. When you're showing software to a client, you don't want the demo to depend on a key, a quota or a network call behaving perfectly. The fallback lets the whole flow be walked through end to end, and the real model slots in when it's configured.

Decision 3: not every call is a conversation

Anyone who has worked near outbound calling knows that a big share of "calls" never become conversations. CallAudit-X has explicit no-live-conversation handling for:

voicemail
no answer
mailbox full
beep tone
spam
wrong number

Instead of forcing a sales or support score onto a voicemail greeting, these get N/A handling. Scoring a 10-second beep as a "bad sales call" would make the analytics useless, so this is a correctness feature more than a nice-to-have.

The transcript viewer is defensive in a similar way: it handles transcript data as a string, an array, an object or null. AI output and imported data don't always arrive in the shape you expect, and the UI shouldn't crash because of it.

Decision 4: humans stay in the loop

AI audits are a first pass, not a verdict. Reviewers can approve a report, or reject it and submit a corrected category and feedback. The demo flow in the README shows this on purpose: approve one report, then reject and correct another to show the feedback loop.

The analytics page then shows DB-backed charts for category, sentiment, score and verification. Verification status being on the chart is deliberate. It tells you how much of what you're looking at has actually been checked by a person.

Decision 5: boring, careful auth

Auth uses real customer and admin logins with bcrypt password hashes and HTTP-only JWT cookies, with role-protected dashboards.

The password reset flow is where I spent extra care:

/forgot-password always returns the same message ("If an account exists, a reset link has been generated."), so it can't be used to check which emails have accounts.
When an account exists, the API creates a random token, stores only its SHA-256 hash, sets a 15-minute expiry and logs the request.
If FORM_SUBMIT_ENABLED=true, the reset email goes out through FormSubmit.
In development, if email is off or fails, the reset URL is shown for testing. In production, reset URLs are never returned in the API response.
/reset-password?token=... validates the hash and expiry, hashes the new password, clears the token, logs completion and sends the user back to login.

Admins also get a security log table alongside user CRUD, category management and a payments view.

Decision 6: billing that fails safely

The billing page shows the current plan, selected plan, usage, invoices, payment status and per-minute audit categories. The repo defines categories such as Sales QA, Support QA, Lead Qualification, Compliance Review, Appointment Review and Spam Detection, each with its own per-minute rate.

If Stripe keys or real Price IDs are missing, checkout buttons stay disabled and say "Stripe checkout is not configured yet." Same idea as the AI fallback: a half-configured app should say what's missing, not throw.

Running it locally

From the README:

npm install
npm run prisma:generate
npm run prisma:migrate
npm run prisma:seed
npm run dev

Then open http://localhost:3000. The seed creates demo admin and customer accounts (listed in the README), and there's a public/sample-call.wav you can upload from /dashboard/upload. There's also a /transcription-demo page that switches between a sales call, a support call and a voicemail/no-answer example.

Two checks I run before shipping changes:

npx prisma validate
npx tsc --noEmit --pretty false --incremental false

What I took away

Make AI calls explicit. A button the user presses is easier to reason about, cache and bill than a hidden call on render.
Design for the missing key. Fallbacks keep demos and dev environments useful.
Model the non-conversations. Voicemail and no-answer calls deserve their own category, or your scores lie.
Keep a human verification step, and show how much has been verified.
Fail safely on payments and auth. Disabled buttons with clear messages beat cryptic errors.

If you run QA for a call team, I'm curious: what share of your calls actually gets reviewed today?

Open to freelance & remote work
Portfolio: https://mafzalkalwardev.github.io/#hire-me
GitHub: https://github.com/mafzalkalwardev
WhatsApp: +92 307 9670503 — https://wa.me/923079670503

How My Google Voice AI Agent Knows When to Talk: Call-State Detection, Groq and TTS

Muhammad Afzal Kalwar — Thu, 29 Oct 2026 22:34:41 +0000

Most AI voice demos start with a telephony API. You get webhooks, clean call events and an audio stream, and the interesting problem is the conversation.

My Google Voice Dispatch Agent doesn't get any of that. It drives Google Voice inside a real Chrome window, and that changes where the hard work is. The AI reply turned out to be the easy part. The hard part was knowing, with confidence, whether someone had actually picked up.

This post walks through how the agent is put together, based on what's in the repo.

Repo: https://github.com/mafzalkalwardev/google-voice-dispatch-agent (MIT licensed)

What it does

In short, it's an outbound AI calling console for Windows:

Selenium controls Google Voice using a persistent Chrome profile and dials contacts from a CSV/XLSX list or CRM data.
It watches the call state: ringing, connected, ended, failed, voicemail.
Once a call is really connected, a realtime loop runs: voice activity detection, speech-to-text, an LLM reply via Groq, and text-to-speech played back into the call.
Voicemail is detected from DOM cues plus an early-call audio classifier, and the agent hands off to a pre-recorded voicemail message.
Every call is logged with transcript, recording, outcome and CRM update.
A FastAPI web console covers settings, preflight checks, audio tools, live runs, logs, recordings and leads. It's also packaged as a Windows EXE with PyInstaller.

The runtime flow

The README describes the flow in ten steps, and I'll keep it close to that:

Load .env, dialer_config.json, contacts and CRM data.
Run preflight checks for Groq, the Chrome profile, contacts, callback number and audio devices.
Launch Google Voice with the configured Chrome profile.
Prepare the opening line and voicemail audio before dialing.
Dial one contact.
Poll detect_call_state() until connected, voicemail, no answer, ended or failed.
Start the realtime conversation loop only after confirmed connection.
Watch for voicemail during the early connected window and hand off if confirmed.
Save logs, transcript, recording, CRM update and call archive.
Cool down, then move to the next contact.

Step 4 is small but matters a lot. If you generate the greeting after someone says "hello", they hear dead air and hang up. Pre-generating it means the agent can talk the moment the call is confirmed.

Call-state detection: the real product

Here's the thing about automating a web UI: the DOM is noisy. Buttons appear early, old "call ended" banners linger, and voicemail-looking cues can show up while the phone is still ringing. If you trust the first signal you see, the bot either talks over the ring tone or hangs up on a real person.

So the core of the project is a conservative gate in src/google_voice.py. The rules it enforces:

Ringing can't instantly jump to ended.
min_ring_seconds has to pass before connected, ended or voicemail transitions are accepted.
Active call controls that appear too early are logged, and the state is held at ringing.
Stale ended banners are ignored during the minimum ringing window.
Voicemail cues are ignored while still ringing.
Connected needs stable evidence, like answered controls or a call timer.
The AI opening line starts only after confirmed connection.
max_ring_seconds ends the attempt as no answer only after the ring timeout.

Every decision is logged too: current state, elapsed ring time, DOM cues, answered controls, call-active status, voicemail cues, the audio classifier result and the timeout reason. When a call goes wrong, I can read exactly why the agent thought what it thought.

The design principles in the README sum it up:

Never speak until the call is actually connected.
Never treat immediate DOM noise as a completed call.
Respect a minimum ringing duration.
Detect voicemail only after connected/greeting evidence.

The conversation loop: VAD, Groq and TTS

Once the gate says "connected", conversation_loop.py takes over. The modules split up cleanly:

src/
  google_voice.py          Selenium Google Voice automation and call-state detection
  call_session.py          Call session model and CallState transitions
  conversation_loop.py     Realtime capture, VAD, STT, LLM, TTS, voicemail handoff
  voicemail_detector.py    Audio classifier for greeting/beep/live-call signals
  ai_groq.py               Groq-backed AI response behavior
  realtime_tts.py          Edge TTS playback and cache integration
  audio_capture.py         Input capture and VAD framing
  audio_routing.py         Windows audio-device discovery and playback routing

The loop works like this:

Capture and VAD. audio_capture.py captures the caller's audio and frames it for voice activity detection, so the agent knows when the person has started and stopped talking.
Speech-to-text and reply. The speech goes to Groq STT, and ai_groq.py generates the reply.
Text-to-speech. realtime_tts.py speaks it with Edge TTS, with caching, and pyttsx3 as a fallback.
Routing. The voice has to get into the browser call, which is where VB-CABLE comes in.

Several knobs in dialer_config.json relate directly to this loop:

{
  "min_ring_seconds": 2,
  "max_ring_seconds": 45,
  "voicemail_detect_seconds": 15,
  "silence_does_not_end_call": true,
  "call_cooldown_seconds": 10,
  "tts_warmup": true,
  "stt_retry_count": 2,
  "vad_silence_frames": 12,
  "vad_speech_frames": 2
}

silence_does_not_end_call is a nice example of a lesson baked into config: people go quiet on the phone, and that shouldn't be read as a hang-up.

Audio routing is half the battle

The minimum VB-CABLE setup from the README:

Agent playback device: CABLE Input
Chrome microphone input: CABLE Output
Chrome speaker output: a device the agent can capture

For unattended runs, the README recommends separate routes for TTS and inbound capture, and warns against capturing the same speaker output that plays the agent's voice. Otherwise the agent can end up transcribing itself. CAPTURE_DEVICE=default uses Windows speaker loopback, which is fine for testing but less reliable for nonstop calling.

There are CLI helpers for checking this before going live:

python -m src.main --list-audio-devices
python -m src.main --audio-route-test
python -m src.main --preflight

Parallel lines without crossed wires

The agent can also run more than one line. Each worker gets its own Chrome profile and pulls from one shared contact queue. The important safety rule: shared audio routes are rejected by default, "because one AI response would otherwise be broadcast into multiple calls." Each slot needs its own loopback and capture pair, like CABLE-A and CABLE-B.

Google Voice accounts can be managed from the console. Saved passwords are protected with Windows DPAPI and never returned by the web API, and MFA or CAPTCHA challenges still have to be completed by a human in the visible Chrome window.

What I'd tell someone building something similar

Treat state detection as the main feature. Spend your time there before tuning prompts.
Log your reasoning, not just outcomes. "Connected" is useless when debugging. "Connected because the timer appeared at 6.2s after ringing" is useful.
Prepare audio ahead of time. Silence at pickup kills calls.
Isolate audio per line. Crossed audio is far worse than a slower dialer.
Add preflight checks. Most failed runs I saw were config problems, not code bugs.

And please: anything that dials real people needs consent, Do Not Call compliance and call-recording disclosure where required. The README also recommends keeping min_ring_seconds above zero, using a cooldown, and not rapid-firing failed numbers.

If you're working on voice agents on top of tools you already use, I'd like to hear what broke first for you.

Open to freelance & remote work
Portfolio: https://mafzalkalwardev.github.io/#hire-me
GitHub: https://github.com/mafzalkalwardev
WhatsApp: +92 307 9670503 — https://wa.me/923079670503