We recently built Soniox Compare, a tool that lets you test real-time voice AI systems side by side.
You can:
- Speak into your mic in any language
- Upload an audio file
The same audio is sent to multiple providers (Soniox, Google, OpenAI, etc) and their outputs appear live, side by side.
We built this because evaluating speech APIs is surprisingly tedious. Static benchmarks often don’t reflect real-time performance, and API docs rarely cover the messy edge cases: noisy input, overlapping speech, mid-sentence language shifts, or audio from the wild.
We wanted a quick, transparent way to test systems using the same audio under the same conditions. And to see what actually works in practice.
So we made something simple, open, and reproducible.
Want to go deeper?
We also open-sourced the full benchmarking framework here: https://github.com/soniox/soniox-compare
You can:
- Run batch tests on your own data
- Add new providers
- Modify the workflow to fit your use case
Try it yourself
- Live tool: https://soniox.com/compare
- GitHub repo: https://github.com/soniox/soniox-compare
Would love any feedback or ideas. What kinds of audio would you throw at this?
Top comments (0)