Voice Arena: I made two AI voice agents fight live on Agora and let the internet judge

#ai #programming #agora #dailybuild2026

TL;DR: Voice Arena puts two real Agora Conversational AI Engine agents in one RTC channel, gives them the same prompt, and lets you vote the winner. Red (VAD + barge-in) vs Blue (semantic end-of-speech + keyword-only interruption). Real get_turns() latency receipts, ELO leaderboard, shareable knockout cards. Built with agora-agents 2.8.1, FastAPI, and a neon zero-build frontend.

Why I built it

Text LMArenas are solved. Voice is where the pain lives: Who feels faster? Does VAD or semantic end-of-speech win a roast battle? Does generated filler actually hide LLM lag? I wanted a toy that answers those questions by showing them — same prompt, two tunings, live audio, crowd vote. If you've ever argued about interruption settings in a PR, this is your boxing ring.

How it works

Prompt ─┬─► Red session ──► RTC channel ─┐
        │     (VAD EoS, SoS interrupt)    ├─► your ears + your vote
        └─► Blue session ─► RTC channel ─┘
              (semantic EoS, keyword-only)

Server (app.py, FastAPI):

POST /api/battle — text undercard. Two OpenAI-Compatible completions (MODEL_A/MODEL_B), blind-shuffled left/right, latency timed server-side:

text, ms, usage = await _chat(base, api_key, model, style["system"], req.prompt, temp, max_tokens)

POST /api/arena/start — the main event. Real SDK sessions via arena_agent.py:

client = AgentClient(area="GLOBAL", app_id=..., app_certificate=...,
                     customer_id=..., customer_secret=...)
red = build_fighter(client, "red").create_session(
    channel=channel, agent_uid="100", remote_uids=["*"])
red.start()
red.think(prompt, on_speaking_action="append")  # inject without killing the greeting

GET /api/arena/turns — post-fight receipts: session.get_turns() returns real per-turn ASR/LLM/TTS timings. No vibes, just milliseconds.
POST /api/vote — ELO (k=24) persisted to data/leaderboard.json.

Fighters (arena_agent.build_fighter): the tuning is the content.

OpenAI(api_key=..., base_url=..., model=..., temperature=...,
       system_messages=[{"role": "system", "content": SYSTEM_RED}])
.with_stt(AresSTT())
.with_tts(MiniMaxTTS(model="speech-2.8-turbo", voice_id=...))  # managed creds, no keys
.with_turn_detection({"mode": "default",
                      "config": {"end_of_speech": {"mode": "vad"}}})  # red: twitchy
.with_interruption({"mode": "start_of_speech"})
.with_filler_words({"enabled": True, "content": {
    "mode": "generated",
    "generated_config": {"fallback_strategy": "static"},
    "static_config": {"phrases": ["Give me a sec.", ...]}}})
.with_labels({"arena": "voice-arena", "fighter": fighter})

Swap one block and the whole fight changes character. That's the demo.

Frontend (static, zero build step): prompt roulette, live typing with EQ bars,
latency badges (⚡ms), SAY-IT-LIVE stage speech via session.say(), voice voting via Web Speech API
("left wins"), confetti, canvas-rendered 1080×600 knockout card, 𝕏 intent link,
and ?battle=id deep links so every fight is shareable.

Three things I learned

think(append) is the showrunner. Injecting the prompt with on_speaking_action="append" lets greetings land instead of getting clobbered. interrupt feels chaotic on stage; append feels produced.
Generated filler words earn their keep. With filler_words.mode=generated, the 1–2s LLM gap stops reading as "dead bot" and starts reading as stage presence. A/B it in the arena and you'll hear it instantly.
EoS mode is a personality slider. VAD Red interrupts, jokes, steps on lines. Semantic Blue waits, lands cleaner closers. Same models, different souls — and the crowd splits exactly the way you'd predict. That's the viral argument baked in.

Run it yourself

cd voice-arena
pip install -r requirements.txt   # agora-agents==2.8.1 pinned
cp .env.example .env              # add Agora + LLM creds
uvicorn app:app --port 8000       # open http://localhost:8000
python arena_agent.py --channel arena-demo --prompt "Dump me like a pirate."

Try: same model both sides, different temps → then different EoS modes → then different voices. Each change is one env var. Each fight is one tweet.

What's next

Heckle button (session.interrupt() live), clip exporter from turn timestamps, third fighter round-robin, avatar video knockouts. PRs welcome — keep SDK pinned, keep .env out of git, keep the fights blind.