DEV Community

杨继成
杨继成

Posted on

Tried `livekit/agents` Today: A Fast Path to Realtime Voice AI

#ai #deepseek #machinelearning #benchmark

Tried livekit/agents Today: A Fast Path to Realtime Voice AI

livekit/agents is a framework for building low-latency voice, video, and multimodal AI agents on LiveKit’s realtime infrastructure. It picked up +256 GitHub stars today, which makes sense: it removes much of the glue code between WebRTC media streams, STT, LLM reasoning, TTS, turn detection, and tool calls.

The useful design choice is that the agent runtime is provider-flexible. You can keep realtime orchestration in LiveKit while pointing the reasoning layer at an OpenAI-compatible gateway.

import os
from livekit.agents import Agent, AgentSession
from livekit.plugins import openai

llm = openai.LLM(
    model="claude-fable-5",
    api_key=os.environ["B_LOST_API_KEY"],
    base_url="https://b-lost.com/v1",
)

assistant = Agent(
    instructions=(
        "You are a concise voice assistant. "
        "Confirm intent before executing sensitive actions."
    ),
    llm=llm,
)

session = AgentSession()
# Connect `session` to your LiveKit room, then run `assistant`.
Enter fullscreen mode Exit fullscreen mode

A practical deployment shape:

Layer Responsibility Swap cost
LiveKit WebRTC rooms, audio/video transport Low
livekit/agents Turn-taking, tools, agent lifecycle Low
STT/TTS provider Speech recognition and synthesis Medium
claude-fable-5 gateway Reasoning and tool planning Low

For cost planning, B-Lost’s relay advertises 0.8× official list pricing (20% off). The exact effective cost still depends on input/output token mix, audio duration, retries, and tool-call verbosity—so I would measure per completed conversation rather than only cost per 1M text tokens.

Metric to benchmark Why it matters
TTFT Determines whether a voice agent feels interruptible
End-of-speech → first audio Best user-perceived latency metric
Tokens / completed task Captures prompt and tool overhead
Tool-call success rate More useful than generic code benchmarks

If your workflow can use native Anthropic /v1/messages, Prompt Caching is especially relevant: cache hits can receive up to a 90% discount, useful for repeated system prompts, policy blocks, and long tool schemas. For OpenAI-compatible routing, verify cache behavior per endpoint rather than assuming it is automatically applied.

The main attraction: LiveKit Agents lets teams benchmark providers independently without rebuilding the realtime stack each time.

Top comments (0)