DEV Community: Hariom Yadav

TLDR: Dograh is an open-source voice AI platform- an OSS alternative to Vapi. Self-hostable, no per-minute fees, visual workflow builder, full call traces per turn. Choose any LLM, STT, and TTS provider. one docker command to run.
Your voice agent made 2000 calls last night. 180 failed. 110 hit answering machines and kept talking anyway. 40 transferred to the wrong department. 30 said something your prompt definitely didn't tell it to say.
You have a call recording. You have a transcript. But you have no idea which turn went wrong, what the LLM actually decided, whether the STT heard something different from what was said, or why latency spiked on call 47 but not call 48.
Voice agents are getting deployed everywhere right now. We haven't spent nearly enough time giving builders the visibility to know what their agents are actually doing.

duct tape as voice AI infra

If you're building a voice agent today, your stack probably looks something like this: Twilio for telephony, Vapi or Retell as the orchestration layer, Deepgram for speech-to-text, ElevenLabs for text-to-speech, OpenAI for the LLM, your own logic for answering machine detection, and some observability to debug when something breaks.
Six vendors. It works. Kind of.
Until Vapi's per-minute fee eats 70% of your margin. Until a call fails silently and you have no turn-level trace to show why. Until your enterprise client says "we can't send call recordings to a third-party cloud." Until you need to change one prompt and you're back to redeploying the whole stack.
The real problem isn't any single vendor. There's no single layer connecting all of them. Every component is a black box. When something goes wrong between them, you're guessing.

tracing is the layer your voice agent is missing

Traditional voice AI platforms are built around per-minute billing. You sign up, connect a Twilio number, write a prompt, hope it works, and get a bill. That's fine for demos. It's wrong for production agents.
Dograh gives your voice agent a complete, observable, self-hostable runtime. Every call generates a full trace - not just a transcript. For every turn you get: what the STT heard, what the LLM decided, which tool was called, what the latency was, what the TTS said, and whether the human interrupted. The call trace is the unit of debugging, not the recording.
The mental model is a debugger, not a phone bill. When a call goes wrong, you open the trace, find the turn, see exactly what happened, fix the prompt, and re-run. No support tickets to a vendor who can't show you the internals.
Dograh is BSD-2 licensed. Self-hosted via docker-compose. There is no per-minute platform fee because you own the platform.
GitHub: https://github.com/dograh-hq/dograh

what dograh gives you out of the box

Dograh does a lot because voice agents need a lot. The important thing is that it's modular - every layer is swappable without touching the rest of the system.

Telephony- works via Twilio, Vonage, and Cloudonix for both inbound and outbound calling. You bring your own numbers. If you're on a PBX, Cloudonix connects directly to your existing SIP trunk. You own the telephony layer with no vendor lock-in.

STT- supports Deepgram, Speechmatics, Sarvam, and OpenAI Whisper. You pick the model per-agent or per-call depending on language, speed, or accuracy needs. Indian English works better on Sarvam. Low-latency real-time transcription works better on Deepgram Nova-3. High accuracy on noisy calls works better on Whisper. You swap without rewriting any logic.

LLM -support covers OpenAI, Google Gemini, Groq, OpenRouter, Azure, AWS Bedrock, and fully self-hosted models. The agent doesn't care which model responds - the interface is the same across all of them.

TTS- runs on ElevenLabs, Cartesia, Deepgram, and OpenAI TTS. There's also a hybrid voice mode that's worth calling out separately. Instead of generating every response with TTS, the LLM picks from a library of pre-recorded human voice clips for common responses and only falls back to TTS when it needs to improvise. This cuts latency because there's no generation delay, cuts cost because you're using less TTS, and sounds more human because it literally is a human recording for the predictable parts. And when you have a dynamic text, fallback to the voice clones TTS.
Here’s a quick tutorial on this hybrid approach
https://www.youtube.com/watch?v=1uZqhG0_cIo

Call traces -are the thing that actually changes how you debug. Every turn is logged with the STT input, LLM output, tool calls, TTS output, and timing at each step. These aren't logs dumped to a file - they're structured and queryable. This is what debugging production voice agents actually requires, and it's the thing that's missing everywhere else.

what people are actually building with dograh

Lead screening and follow-ups. The agent calls a list, detects answering machines using a custom detection prompt, disconnects on voicemail, and books when it reaches a human. The trace shows every call - what speech was detected, what the agent said, where drop-off happened.

Outbound sales. The agent pulls CRM data before each call and injects it into the prompt. It qualifies, handles objections, and transfers to a human rep when intent is high. It updates the CRM automatically so the rep already knows what was said before they pick up.

Inbound support. The agent handles tier-1 support - order status, appointment changes, basic troubleshooting. When it can't resolve, it transfers with a conversation summary already written to the CRM. The human agent picks up with full context, not a blank screen.

Multi-language outbound. One agent, multiple languages. Sarvam for Hindi, Deepgram for English and Spanish. The agent detects language on the first turn and switches STT and TTS provider automatically. No separate agent per language, no separate infrastructure per market.

other things built into dograh

Beyond the call itself, Dograh has a few other things built in worth knowing about.

Automatic QA and post-call analysis.
After every call, Dograh checks what happened automatically. It looks at sentiment, whether the user got confused, whether the agent followed its instructions, and what actions actually fired. You don't need to listen to 200 recordings to find problems. It surfaces where things went wrong - whether the agent sounded off, missed a question, or skipped a step in the flow.

Dedicated telephony with integrated dialer and Asterisk ARI. Telephony is built in, not bolted on. You get low-latency calling across regions and a dialer that works out of the box for both inbound and outbound. No separate systems to stitch together. For teams that need deeper control, Dograh supports Asterisk ARI - you can plug into existing telephony infrastructure, customize call logic at a low level, and build more complex flows. Flexible enough for serious production deployments.

API key rotation.
Add multiple API keys for any LLM, STT, or TTS provider and Dograh rotates between them automatically. No custom hacks needed to stay under rate limits or handle concurrency at scale.

Looptalk (coming soon). AI-to-AI call testing. You spin up a test caller with a persona - "skeptical prospect", "fast talker", "non-native English speaker" - and run it against your production agent at scale. Every simulated call leaves a full trace. You find edge cases before real customers do.

why Open Source Matters Here

Closed-source voice AI platforms have a structural problem. When your agent breaks, you file a ticket. The platform tells you what they can see. You don't get the internals, you don't get the turn-level trace, and you can't fix it yourself.
Better support doesn't fix that. It's a fundamental property of closed infrastructure.
With Dograh you run the platform. The trace is yours. The data never leaves your VPC unless you want it to. When something breaks at 3am, you look at the trace. You don't wait for a vendor to respond.
This is also why BSD-2 matters. Not AGPL, not SSPL, not "open core with enterprise features behind a paywall." BSD-2 means you can fork it, modify it, white-label it, embed it in a commercial product, and deploy it for clients without owing anyone a license fee. The code is yours.
The per-minute fee model in closed platforms creates a genuinely adversarial relationship - the platform makes more money when your calls are longer or more frequent. Dograh's business model is managed hosting on app.dograh.com for teams that don't want to self-host. The self-hosted version is completely free and always will be.

get started

Github link - https://github.com/dograh-hq/dograh
Runs on any Linux host or Apple Silicon Mac. The default config works for local dev. Drop your LLM, STT, and TTS API keys in the .env file and swap providers without touching code.
dograh - open-source voice AI runtime, full call traces, self-hostable
docs.dograh.com - setup, provider config, AMD, call transfer, knowledge base
app.dograh.com - managed hosting if you don't want to run the infra
Every provider is pluggable. Every call leaves a trace. The platform is yours.

The Open-Source Voice AI Stack Every Developer Should Know in 2026

Hariom Yadav — Sat, 21 Mar 2026 09:35:05 +0000

"Voice AI just had its "ChatGPT moment."
A year ago, building a voice agent meant stitching together five different APIs and paying multiple vendors per minute of conversation. Today the open-source ecosystem has genuinely caught up - and it's moving fast.
I've been deep in this rabbit hole building Dograh, an open-source voice agent platform like n8n. This post is basically the research I wish existed when I started. Here's the full OSS stack - from raw audio all the way to a deployed phone agent.

The Stack at a Glance

A production voice agent has five layers:
Telephony / Transport  ->  Twilio, Vonage, WebRTC
STT (Speech-to-Text)   ->  Parakeet, Canary Qwen, Silero VAD
LLM                    ->  GPT-4o, Claude, Llama 3
TTS (Text-to-Speech)   ->  Chatterbox, Kokoro, XTTS-v2
Orchestration          ->  Dograh, Pipecat, LiveKit Agents

Every single layer now has solid open-source options. Let's go through them one by one.

Speech-to-Text

If you're building anything real-time, you need something built for streaming from the ground up. Whisper was a great model in 2022 and has kept up well with the voice agent use case. You may try whisper turbo first before other alternatives.
The best option right now for English real-time transcription is NVIDIA's Parakeet TDT 0.6B V2. It sits at #3 on the Hugging Face Open ASR leaderboard with a 6.05% WER, but the number that actually matters for voice agents is its RTFx score of 3386 - meaning it can process audio roughly 3000x faster than real-time. On a T4 GPU it's extremely affordable to run. It handles punctuation, capitalization, and word-level timestamps out of the box.

Python
import nemo.collections.asr as nemo_asr

model = nemo_asr.models.ASRModel.from_pretrained("nvidia/parakeet-tdt-0.6b-v2")
transcript = model.transcribe(["audio.wav"])
print(transcript[0])

If accuracy matters more than raw speed - say you're transcribing medical calls or anything where a wrong word is costly - NVIDIA's Canary Qwen 2.5B is the current accuracy leader on the Open ASR leaderboard at 5.63% WER. It combines ASR with LLM capabilities under the hood, which helps a lot with context and unusual vocabulary. The tradeoff is it's heavier to run and not as snappy for real-time use.
Either way, pair your STT model with Silero VAD. It's a small Voice Activity Detection model that tells your agent when someone is actually speaking. Without it you're either cutting people off mid-sentence or waiting awkwardly for them to finish. Every real-time voice pipeline needs this.

Text-to-Speech

Chatterbox from Resemble AI is the most exciting TTS release of the past year. It hits commercial-grade quality in blind tests, supports voice cloning, and has built-in audio watermarking for responsible use. If you're building anything customer-facing, this is probably your best open-source option right now.

Python
import torchaudio
from chatterbox.tts import ChatterboxTTS

model = ChatterboxTTS.from_pretrained(device="cuda")
wav = model.generate("Hello! This is Chatterbox speaking.")
torchaudio.save("output.wav", wav, model.sr)

For multilingual voice cloning, XTTS-v2 from Coqui is the go-to. It supports zero-shot cloning across 20+ languages with just a 6-second reference clip. Works well for audiobook tools, multilingual assistants, and anything where you need a consistent voice across languages.
If latency is your main constraint, look at Kokoro. It's only 82M parameters, runs on CPU, and can hit under 100ms on consumer hardware. The quality isn't Chatterbox-level but for edge deployments or high-throughput scenarios it's hard to beat.

Orchestration

This is the layer most developers underestimate. Orchestration ties STT, LLM, and TTS together and handles all the hard real-time stuff - barge-in when the user interrupts, turn detection, audio streaming, silence handling. Getting this wrong is what makes voice agents feel robotic even when the individual models are great.

Dograh is what I've been building. Think of it as the n8n of voice AI - a visual workflow builder where you can wire up your entire agent flow without touching code. It's the most direct open-source alternative to Vapi and Retell, and unlike those it's fully self-hostable with no per-minute markup.
It's pretty mature at this point. You get telephony out of the box via Twilio, Vonage, and Cloudonix, inbound and outbound calling, a fully built workflow builder, and one-command setup. All the plumbing is already there - knowledge base, dictionary, KYC, voicemail detection, variable extraction, QA you calls, multilingual, transfer call to human. It can be an inbound and outbound call as well as be deployed as a widget on your website. And you just bring your own LLM, STT, and TTS keys and plug them in.
curl -fsSL https://raw.githubusercontent.com/dograh-hq/dograh/main/install.sh | bash

The visual workflow builder is the big differentiator from raw frameworks like Pipecat or LiveKit Agents. Changing agent behavior means dragging a node, not editing Python and redeploying. For teams that want to iterate fast on agent logic without a full engineering cycle every time, that's a pretty big deal.

Pipecat is a Python framework built by Daily.co. It treats audio as a stream of typed frames and lets you build a pipeline of processors in sequence. It's transport-agnostic and gives you fine-grained control over every step. That control comes at a cost though - every time you want to change agent behavior, you're editing Python code, redeploying, and hoping nothing broke in the pipeline. For a team without dedicated voice engineering experience, the iteration loop gets slow fast.

Python
pipeline = Pipeline([
    transport.input(),
    silero_vad,
    deepgram_stt,
    openai_llm,
    cartesia_tts,
    transport.output(),
])

LiveKit/Agents has a cleaner API and abstracts away the WebRTC infrastructure, which makes the initial setup quicker. But the same problem applies - your agent logic lives in code. Any prompt change, flow tweak, or new use case means a code change and a redeploy. It's genuinely a good framework if you have engineers who live in this stuff full-time, but it's not something a small team can move fast with.

python
agent = VoicePipelineAgent(
    vad=silero.VAD.load(),
    stt=deepgram.STT(),
    llm=openai.LLM(),
    tts=cartesia.TTS(),
)

Both Pipecat and LiveKit Agents are solid if you want maximum control and have the engineering bandwidth to match. If you don't, you'll spend more time maintaining infrastructure than actually improving your agent.

Quick Comparison

Feature	Dograh	Pipecat	LiveKit Agents
Type	Platform	Framework	Framework
Visual Workflow Builder	Yes	No	No
Frontend UI	Yes	No	No
Telephony	Twilio, Vonage, Cloudonix	Twilio	SIP
Self-hostable	Yes	Yes	Yes
Setup Time	Minutes	Hours	Hours
Bring Your Own LLM	Yes	Yes	Yes
Open-source	Yes	Yes	Yes

The Mistake Most People Make

The biggest trap I see developers fall into when building voice agents: treating voice like chat with a microphone attached.
It's a completely different problem. The hard parts aren't the models - they're the real-time engineering. When do you cut off the STT and start the LLM? What happens when the user interrupts the agent mid-sentence? How do you handle answering machines on outbound calls? What about codec mismatches - PSTN phone lines use 8kHz u-law, but most STT models expect 16kHz PCM? These are the things that will actually bite you in production.

If you're just starting out, use either Dograh or Vapi or Retell to prototype. They're all fast and handle a lot of edge cases well. But once you hit serious volume or need custom logic, the open-source stack should be your default. Choose Dograh - and the cost difference is real. Running your own stack costs under $0.02 per minute. Managed platforms charge $0.10 to $0.15.

A Starter Stack That's Completely Free

If you want to get something running this weekend with zero API bills:
VAD - Silero VAD
STT - Parakeet TDT 0.6B V2 running locally
LLM - Llama 3.1 via Groq's free tier
TTS - Kokoro running locally
Orchestration - Dograh
Total infra cost - basically zero. Real latency - very achievable under 500ms end-to-end with a mid-range GPU.

What Are You Building?

Curious what stacks people are actually running in production. Is anyone using Kokoro for real-time agents? The latency numbers look great on paper but I haven't seen many production writeups.
Drop your stack in the comments.

I'm building Dograh - an open-source alternative to Vapi and Retell. If you're tired of vendor lock-in, check it out and star it if it's useful.