Translation apps interpret between two people speaking different languages. Code-switching is different: one caller switches languages mid-conversation and the agent follows them. No interpreter, no restart, no separate vendors — one Telnyx AI Assistant that detects the spoken language on every turn and replies in kind.
The Telnyx code example is:
It is a Python Flask app with a simple browser page showing a phone number. No database, no Cloud Storage. Call the number, speak in English, switch to Spanish mid-conversation, and the agent follows you.
The Flow
A customer calls a support line. They start in English, switch to Spanish for a sensitive question, then switch back. The agent detects the language from their speech on every turn and replies in that language. No language picker menu, no "press 1 for English." The caller just speaks.
One phone number, one AI Assistant, five languages — English, Spanish, Portuguese, Hindi, and Mandarin.
How Code-Switching Works
The entire behavior lives in the assistant's instructions. There is no application-layer language detection, no routing, no manual STT/LLM/TTS pipeline. The LLM follows plain English instructions:
listen carefully to the caller. detect the language they are speaking on every turn.
reply in the same language the caller is using right now.
if the caller switches language mid-conversation or mid-sentence, switch with them.
The STT (Deepgram nova-3 with language: "auto") transcribes in whatever language the caller speaks. The LLM follows the instructions. The TTS (voice ultra katie) renders the reply in that language. One platform, one API key.
Why This Is Different From Translation
The existing ai-real-time-translation-bridge-python example connects two callers who speak different languages and translates between them. That is interpretation — two people, two languages, one bridge.
This example is code-switching — one caller, multiple languages, one agent. The agent is not translating. It is the agent. It replies in whatever language you speak, switches when you switch, and never says "I did not understand" or asks you to pick a language.
Key Technical Decisions
deepgram/nova-3 with language: "auto"
Per the AI Assistants docs: "To enable a multilingual agent, set the transcription model to deepgram/nova-3." The language: "auto" setting lets nova-3 auto-detect the spoken language on every turn. Note: language: "multi" is not valid and returns a 400 error. Use "auto".
voice ultra katie in the instructions
The existing phone assistant examples use voice ultra katie in the instructions. Ultra supports 36+ languages, so one voice handles all five demo languages without switching voices per language.
moonshotai/Kimi-K2.6
A native Telnyx model — no external API key required.
The Flask App Is Minimal
Since the Voice AI Assistant handles the entire conversation, the Flask app only needs to:
- Create/reuse the assistant via
provision_assistant.py - Serve a page showing the inbound phone number
- Log webhook events for observability
There is no call-control logic, no application-layer STT, no manual TTS. The conversation runs entirely on Telnyx.
Run It
git clone https://github.com/team-telnyx/telnyx-code-examples.git
cd telnyx-code-examples/ai-multilingual-code-switching-agent-python
cp .env.example .env
python provision_assistant.py
python app.py
Open the browser, call the number, speak in any language. Switch mid-conversation and the agent follows.
Related Examples
-
ai-real-time-translation-bridge-python— two-caller interpreter -
ai-language-learning-phone-tutor-python— phone-based language tutor -
language-learning-flashcards-python— browser-based pronunciation scoring
Top comments (0)