While Learning LangChain, I Ended Up With This Realtime Voice Pattern

Jingyi — Fri, 26 Jun 2026 11:46:09 +0000

I've recently started learning LangChain while exploring different ways to build voice agents.

As someone who's still new to LangChain, I tried a few different approaches before landing on one that felt surprisingly clean.

I don't know whether this is already a common pattern in the LangChain ecosystem, but it worked well enough that I thought it was worth sharing. Hopefully it can also spark some discussion with people who've been building voice agents for longer.

The idea

The basic idea is to separate responsibilities instead of letting one framework handle everything.

In this pattern:

LangChain stays responsible for tool selection, tool execution, and response composition.
A separate realtime runtime handles RTC / RTM, speech input/output, and session lifecycle.
The two communicate through an OpenAI-compatible endpoint.

What surprised me

When I first started experimenting, I assumed adding realtime voice would require restructuring most of the application.

Instead, it felt more like adding another interaction layer.

LangChain continues doing what it already does well—tool orchestration and workflow management—while the realtime runtime focuses on voice-specific concerns.

That separation made the overall architecture feel much simpler.

The pattern

To better understand the idea, I put together a small recipe that demonstrates this integration pattern.

The recipe consists of three pieces:

Python exposes an OpenAI-compatible endpoint and manages the agent lifecycle.
Next.js handles the client-side realtime interaction.
LangChain remains server-side as the orchestration and tool layer.

It's not intended to be a complete application or a production-ready project.

It's simply a pattern that helped me understand how these pieces can fit together.

A quick note

For transparency, I work at Agora, so I naturally used Agora as the realtime runtime while putting this recipe together.

That said, I'm much more interested in learning how other people are building voice agents with LangChain.

If you're using a different stack or have taken a completely different approach, I'd love to hear about it.

The recipe

If you're curious, here's the recipe:

https://github.com/bluemotional/recipe-agent-langchain

Since I'm still learning LangChain myself, I'd really appreciate any feedback or suggestions.

What's next?

If people find this pattern useful, I'd like to keep expanding it with more recipes, for example:

RAG
Internal tools
Docs Copilot
MCP integrations

I'm also curious to see what other patterns the community has found useful.

Project Spotlight: An Open-Source AI Sports Commentator That Watches the Match Live

Jingyi — Fri, 26 Jun 2026 01:35:29 +0000

I recently came across an open-source project called WorldCupVoice and thought it was worth sharing.

Quick note first: I’m not the creator.

The project is by zicojiao.

The demo makes the idea click immediately: this is not a post-game recap system. It watches a live match feed and speaks over the action in real time.

Project links:

What makes this project interesting is that it is not a “generate a match summary later” system.

It is a live pipeline:

a match feed comes in through RTMP
Agora converts it into a real-time RTC stream
the backend samples frames from the live video
OpenAI generates grounded play-by-play commentary
TTS turns that into speech
the generated audio is published back into the same RTC room

What I found especially interesting from reading the README and code is that the project is clearly trying to solve a harder problem than image captioning.

The prompts appear to push the model toward a live football commentator role by constraining it to:

comment on visible action rather than narrate loosely
avoid inventing off-screen goals or fouls
use player names only when visual evidence is strong enough
return NO_CALL when the frame is not readable enough

Another thing I liked was that the project seems to take operational reality seriously.

It includes explicit Start AI / Stop AI controls, viewer heartbeat, and a hard session TTL, which makes a lot of sense for a live AI system where cost can keep growing even when nobody is watching.

The World Cup framing is fun, but the accessibility angle might be even more compelling.

A system like this could potentially add more field-level spatial detail for blind and low-vision fans than traditional commentary often does.

Again, I’m not the author, so if I’m misreading any implementation detail, happy to be corrected. But as an open-source example of a real-time multimodal media pipeline, this was one of the more interesting projects I’ve come across recently.

Full credit to zicojiao for the project itself. I’m only sharing it because I found it worth discussing.

DEV Community: Jingyi