Open-source speech-to-text got good. That part isn't marketing spin, and pretending otherwise is how you lose an engineer's trust in the first paragraph.
Whisper Large-v3 transcribes clean English about as well as anything you can buy. Qwen3-ASR and NVIDIA's Parakeet and Canary models are genuinely strong. Mistral shipped Voxtral. And the checkpoints are free — pip install, download the weights, point it at an audio file, done. Spin one up on Baseten, Modal, or Fireworks and you've got a working transcription endpoint in an afternoon. For a growing number of teams, that afternoon ends with a reasonable question: why are we paying an API for this?
It's a fair question. Here's the more useful one: what does it actually cost to run that endpoint in production for the next two years?
Because the sticker price of an open model is zero, and the total cost of ownership is not. The gap between those two numbers is where most self-hosting decisions quietly go wrong.
What "free" actually buys you
A model checkpoint is one component of a transcription product. It is not the product.
When you call a managed speech-to-text API, the price covers the model plus the parts you'd otherwise have to build, operate, and keep alive yourself: the inference infrastructure, the accuracy work on messy real-world audio, speaker diarization, entity formatting, streaming, language handling, and the on-call rotation for when any of it breaks at 2am. Download a raw checkpoint and you own every one of those. The weights are free. The system around them is the job.
So the honest comparison isn't "$0 vs. $0.21 an hour." It's "your engineering org vs. $0.21 an hour." Let's break down what lands on your side of that trade.
| Factor | Self-hosted open source (Whisper, Parakeet, Voxtral, Qwen3-ASR) | Managed API (AssemblyAI) |
|---|---|---|
| Upfront model cost | $0 — open weights, no license fee | $0 to start — free credits and playground, then pay-as-you-go |
| What you actually pay for | GPU hours (billed even when idle), plus engineering and DevOps time | Only the audio you process, billed per second |
| Typical run cost | ~$2–4+/hr per on-demand A100/H100-class GPU, regardless of utilization | $0.15–$0.21/hr async; $0.15–$0.45/hr streaming |
| Idle / low utilization | You pay full price for idle GPUs — 15% utilized still bills 100% | No idle cost — you only pay for audio transcribed |
| Accuracy on real-world audio | Varies; higher hallucination and WER on noisy, accented, code-switched speech (e.g., Voxtral Mini ~18% WER code-switched) | Tuned for production audio; Universal-3.5 Pro <8% WER on code-switched speech, and Universal-3 Pro runs ~30% fewer hallucinations than Whisper |
| Speaker diarization | Build or integrate a separate system | Built in — transcript and speaker turns produced jointly |
| Real-time streaming | Build your own low-latency serving stack | Native streaming API with ~300 ms partial transcripts |
| Entity accuracy & PII redaction | Not included — build and maintain separately | Included: entity detection, PII redaction, and formatting |
| Scaling & concurrency | You manage autoscaling, queuing, and retries | Unlimited concurrency, no rate limits |
| Reliability & maintenance | Your team owns uptime, monitoring, model upgrades, and on-call | Managed uptime with 24/7 support |
| Data control & compliance | Full control; runs entirely in your own VPC or on-prem | Voice AI Cloud, EU data residency, self-hosted deployment, and a BAA available |
| Time to production | Weeks to months of setup and tuning | Minutes — sign up and get an API key |
| Best fit | High-utilization offline batch, research/prototyping, strict data isolation | Real-time and customer-facing apps needing accuracy, features, and scale without ops burden |
Hidden cost #1: the GPU bill nobody quotes you
Open weights don't come with a price tag, but GPUs do. To serve transcription at production quality and latency you're renting accelerators — and you pay for them whether or not audio is flowing through them.
This is the detail that surprises teams. Utilization is rarely high. Traffic is spiky, models sit warm waiting for requests, and a GPU idling at 15% still bills at 100%. Serverless platforms like Modal soften the idle problem but hand you a cold-start problem instead: scale to zero and your next request waits for a container and a multi-gigabyte model to load. Keep instances warm to avoid that and you're back to paying for idle. Autoscaling a GPU fleet against bursty voice traffic is a real, ongoing engineering project, not a checkbox.
A managed API bills per second of audio — AssemblyAI's pricing runs $0.15–$0.21/hr for pre-recorded transcription with no minimums and no idle time — because absorbing that utilization math is the vendor's problem, not yours.
Hidden cost #2: everything the checkpoint doesn't do
Here's where the "it's basically the same model" argument falls apart. Raw ASR checkpoints give you a transcript. Production voice products need a lot more than a transcript.
Speaker diarization. Who said what. Most open checkpoints don't do it at all, and the ones that gesture at it don't hold up on the audio that matters — short back-and-forth turns, interruptions, overlapping speech. Getting diarization right is its own research problem; AssemblyAI's latest models produce the transcript and the speaker boundaries jointly, optimized for concatenated word error rate rather than the looser metrics that make demos look fine and production look broken.
Entity accuracy. In the real world people say credit card numbers, email addresses, medication names, and account IDs — the exact tokens where a single wrong character makes the transcript worthless. A base model transcribes the words; it doesn't reliably format an alphanumeric confirmation code spelled out over a phone line.
Streaming. Most open checkpoints are async-only. If you're building anything live — a voice agent, real-time captions, in-call intelligence — you get to build the streaming layer yourself: chunking, partial hypotheses, endpointing, the works. Real-time transcription is a different engineering problem than batch, not a config flag.
Code-switching, redaction, formatting. Multilingual speakers switch languages mid-sentence. Recordings need PII redacted. Numbers, dates, and punctuation need to render like a human wrote them. Every one of these is a feature you either buy once or build and maintain forever.
You can build all of it. The question is whether reimplementing diarization and a streaming stack is the best use of your team versus shipping your actual product.
Hidden cost #3: accuracy on audio that isn't clean
Benchmarks are run on clean audio. Your users are not clean audio.
They're on a phone in a moving car, in a call center with three conversations bleeding together, using product names and industry jargon no model saw in training, in accents the leaderboard underweighted. This is exactly where open checkpoints tend to slip — and where one specific failure mode gets expensive: hallucination. Whisper is known to invent fluent, confident text during silence or noise. It's not a rounding error; in AssemblyAI's own testing, Universal-3 Pro's hallucination rate runs about 30% lower than Whisper's, and on published head-to-head benchmarks, open models trail badly on the audio that resembles production — Voxtral Mini, for instance, posts nearly 18% word error rate on code-switched speech versus under 8% for a managed flagship. A transcript that's wrong in a plausible way is more dangerous than one that's obviously garbled, because nothing downstream flags it.
Closing that gap yourself means fine-tuning, building an eval set from your own audio, and re-running it every time you touch the model. That's a standing capability you'd be committing to own. And word error rate alone won't even tell you if you're winning — you need to measure the errors that actually break your product.
Hidden cost #4: reliability is now your pager
The afternoon demo runs on one GPU serving one request. Production is a different animal.
Concurrency at real volume, retries and graceful degradation, latency under load, model-version upgrades without downtime, region failover, monitoring, uptime — this is the operational surface of a speech platform, and self-hosting means it's yours. Someone carries the pager. A managed platform runs hundreds of millions of inference calls a month with unlimited concurrency and no rate limits, because reliability at scale is the product it's selling. When you self-host, reliability at scale is a headcount question.
When self-hosting is the right call
None of this means "never self-host." There are real cases where owning the stack is the correct decision:
You're doing research or a prototype where model control matters more than production polish. You're running massive offline batch jobs where latency is irrelevant and you can keep GPUs pinned at high utilization, which fixes the economics. You have a hard data-isolation requirement that rules out any external call — though note that a managed platform can meet strict needs too, through self-hosted deployment in your own VPC, EU data residency, and a signable Business Associate Addendum for teams handling protected health information. Or you have an ML platform team that wants to own this and has the cycles to.
If one of those is you, self-hosting is defensible. If none of them is you, the "free" model is the most expensive line item on the roadmap — it just doesn't show up on the invoice.
The number that actually matters
The mistake isn't choosing open source. The mistake is comparing the wrong two numbers.
The comparison that matters isn't the model's price against the API's price — it's the fully loaded cost of running your own speech stack, GPUs and engineers and eval pipelines and on-call included, against a per-second bill where someone else owns all of it. Run that math honestly and the answer flips more often than the afternoon-demo instinct suggests. Cheap models are real. Free ones don't exist.
If you want to see where a managed model lands on your actual audio before you commit engineering time to either path, run it against a batch of your worst recordings — the noisy ones, the accented ones, the jargon-heavy ones. That's the benchmark that decides it.
Frequently asked questions
Is open-source speech-to-text really free?
Open-source speech-to-text models are free to download but not free to run in production. Model weights like Whisper, NVIDIA Parakeet, Qwen3-ASR, and Mistral's Voxtral carry no license fee, but you still pay for the GPUs that serve them, the engineering time to add production features, and the on-call burden to keep them running. The real comparison isn't "$0 vs. an API bill" — it's your fully loaded infrastructure and engineering cost versus a managed per-second price.
How much does it cost to self-host an open-source speech-to-text model?
Self-hosting cost is driven by GPU time, not the model. A single on-demand cloud GPU in the A100/H100 class typically runs $2–4+ per hour and bills whether or not it's actively transcribing, so a model idling at 15% utilization still costs you 100%. On top of that you add engineering time to build diarization, streaming, and formatting, plus ongoing monitoring and model upgrades. By contrast, a managed API like AssemblyAI charges $0.15–$0.21 per hour of audio for pre-recorded transcription, billed per second with no idle cost.
Is self-hosted Whisper cheaper than a speech-to-text API?
It depends almost entirely on GPU utilization. Self-hosted Whisper can be cost-effective for large offline batch jobs that keep a GPU near 100% busy, but for spiky or low-volume workloads the idle GPU time usually makes a per-second managed API cheaper. Whisper also lacks production features and shows higher hallucination rates on real-world audio — AssemblyAI's Universal-3 Pro runs about 30% fewer hallucinations than Whisper. Benchmark both on your own audio and include engineering and infrastructure time, not just the sticker price.
What features do open-source speech-to-text models lack out of the box?
Raw open-source checkpoints transcribe audio but omit most of what production apps need. Commonly missing pieces include speaker diarization (who said what), real-time streaming, PII redaction, entity accuracy for data like credit cards and medication names, code-switching across languages, and consistent formatting. Each one has to be built, integrated, and maintained yourself, whereas a managed API ships them as part of the model or a single parameter.
When does self-hosting open-source speech-to-text make sense?
Self-hosting is a good fit when you have high, steady GPU utilization or strict control requirements. The clearest cases are large offline batch jobs that keep GPUs busy, research and prototyping where you need to modify the model, strict data-isolation rules, and teams that already run a dedicated ML platform. For most real-time or customer-facing products that need accuracy, features, and scale without an ops burden, a managed API is usually the lower-cost and faster path to production.
How much does a managed speech-to-text API cost?
AssemblyAI's pre-recorded speech-to-text starts at $0.15–$0.21 per hour of audio, billed per second with no minimums, and streaming transcription ranges from $0.15 to $0.45 per hour. There are no idle-GPU charges, no rate limits, and unlimited concurrency, so you pay only for the audio you actually process.
Top comments (0)