Language learning apps work fine on a screen, but the hardest part of picking up a new language is the thing screens avoid: actually speaking out loud, in real time, to someone who responds. This walkthrough builds a Flask app that turns a phone call into a language practice session — the caller dials a number, picks a language, and talks to an AI tutor that responds in the target language with English support, adjusts difficulty as the conversation grows, and corrects mistakes gently.
The whole loop runs on Telnyx Call Control for the voice path and AI Inference for the conversation, with text-to-speech rendering the tutor's responses back to the caller.
The canonical code example lives in the Telnyx code examples repo:
What This Example Builds
A single-file Flask app with three endpoints, driven by a webhook state machine:
Method Path Purpose
POST /webhooks/voice Telnyx Call Control webhook handler — the call flow state machine
GET /sessions List recent completed call sessions (caller, language, exchanges)
GET /health Service health check (active calls, total sessions)
The call flow runs in four stages:
An inbound call arrives and is answered
Text-to-speech greets the caller and asks them to pick a language by pressing a digit
The caller speaks (or dials a number), and their speech is transcribed via Call Control's gather action
The transcript goes to AI Inference with a system prompt tuned for language tutoring — the model responds in the target language with English support, corrects mistakes, and gradually raises difficulty. The response is spoken back to the caller via TTS, and the loop repeats until hangup.
Why This Matters
The phone tutor exercises two Telnyx AI products in a real-time, full-duplex loop — Voice AI for the call path and AI Inference for the conversation — without any third-party telephony or LLM plumbing. That makes it a useful reference for anyone building conversational voice applications where the AI is not just reacting to a single prompt but holding a multi-turn conversation with state.
Developers use this pattern as the foundation for:
Language practice hotlines — Give learners a number they can call anytime for low-stakes speaking practice
Customer support voice agents — Replace rigid IVR menus with an AI that asks clarifying questions and responds naturally
Interactive voice surveys — Ask open-ended questions and let the AI probe for richer answers than DTMF menus allow
Accessibility services — Offer spoken-language interfaces for users who can't or won't use a screen
Educational tutoring — Extend the tutor prompt to any subject and let callers practice on any phone
Products Used
telnyx_products: [Voice AI, AI Inference]
Telnyx Call Control drives the call — answer, gather speech or DTMF, play TTS, and hangup — all through a webhook-driven state machine so the app never holds a socket open for the call's duration. Telnyx AI Inference exposes an OpenAI-compatible chat completions endpoint, so the conversation context is just a list of {role, content} messages sent to POST /v2/ai/chat/completions.
Architecture
Inbound Phone Call
│
▼
┌──────────────────┐
│ call.initiated │ app answers the call
└────────┬─────────┘
│
▼
┌──────────────────┐
│ call.answered │ TTS greeting: "Press 1 for Spanish, 2 for French…"
└────────┬─────────┘
│
▼
┌──────────────────┐
│ call.speak.ended │ gather DTMF (1 digit) or speech
└────────┬─────────┘
│
▼
┌──────────────────┐
│ call.gather.ended│ parse selection → seed system prompt
│ │ → AI Inference → TTS response
└────────┬─────────┘
│
▼
┌──────────────────┐
│ call.speak.ended │ gather speech (caller's turn)
│ │ ◄── conversation loop
└────────┬─────────┘
│
▼
call.hangup
The Code
The full app is 108 lines in a single app.py. The key pieces:
Configuration and call state
The Telnyx API key and public key are loaded from the environment; the public key is used to verify incoming webhook signatures. Two in-memory stores track active calls and completed sessions, with a background TTL thread that prunes stale entries after an hour so memory does not grow without bound:
client = telnyx.Telnyx(api_key=os.getenv("TELNYX_API_KEY"), public_key=os.getenv("TELNYX_PUBLIC_KEY"))
active_calls = {}
session_history = []
def _start_ttl_cleanup(*stores, ttl_seconds=3600, interval=300):
def _cleanup():
while True:
_ttl_time.sleep(interval)
cutoff = _ttl_time.time() - ttl_seconds
for store in stores:
expired = [k for k, v in store.items()
if isinstance(v, dict) and v.get("_ts", _ttl_time.time()) < cutoff]
for k in expired:
store.pop(k, None)
threading.Thread(target=_cleanup, daemon=True).start()
Webhook signature verification
Every incoming webhook is verified against the Telnyx Ed25519 signature before the app trusts the event. Unverified requests get a 401 and the handler returns immediately:
@app.route("/webhooks/voice", methods=["POST"])
def handle_voice():
try:
client.webhooks.unwrap(request.get_data(as_text=True), headers=dict(request.headers))
except Exception:
return jsonify({"error": "invalid signature"}), 401
payload = request.get_json()
...
AI Inference call
A thin wrapper over the OpenAI-compatible chat completions endpoint. The conversation is passed as a list of messages — the first being the system prompt that defines the tutor's behavior, followed by alternating user/assistant turns:
INFERENCE_URL = "https://api.telnyx.com/v2/ai/chat/completions"
def call_inference(messages, max_tokens=200):
resp = requests.post(INFERENCE_URL, headers={
"Authorization": f"Bearer {TELNYX_API_KEY}",
"Content-Type": "application/json",
}, json={
"model": AI_MODEL, "messages": messages,
"max_tokens": max_tokens, "temperature": 0.7,
}, timeout=15)
resp.raise_for_status()
return resp.json()["choices"][0]["message"]["content"]
The call flow state machine
Each Telnyx event triggers the next action. The caller picks a language on the first gather, which seeds the system prompt with the target language and tutor behavior. From then on, every gather's transcribed speech becomes the next user message, the model's reply becomes the next TTS payload, and the loop continues until hangup:
if event_type == "call.initiated" and p.get("direction") == "incoming":
active_calls[ccid] = {"caller": p.get("from"), "state": "language_select", "conversation": []}
client.calls.actions.answer(ccid)
elif event_type == "call.answered":
client.calls.actions.speak(ccid, payload="Welcome to Language Tutor! Press 1 for Spanish, 2 for French, 3 for Japanese, 4 for Mandarin.", voice="female", language_code="en-US")
elif event_type == "call.speak.ended" and call:
if call["state"] == "language_select":
client.calls.actions.gather(ccid, input_type="dtmf speech", timeout_secs=10, min_digits=1, max_digits=1)
else:
client.calls.actions.gather(ccid, input_type="speech", end_silence_timeout_secs=3, timeout_secs=20, language_code="en-US")
elif event_type == "call.gather.ended" and call:
if call["state"] == "language_select":
lang = LANGUAGES.get(digits or speech[:1], LANGUAGES["1"])
call["language"] = lang
call["state"] = "tutoring"
call["conversation"] = [{"role": "system", "content": f"You are a {lang['name']} language tutor. Start with a simple greeting in {lang['name']}, then English translation. Gradually increase difficulty. Correct mistakes gently. Mix {lang['name']} and English. Keep each response short for phone conversation."}]
intro = call_inference(call["conversation"] + [{"role": "user", "content": "Start the lesson."}])
call["conversation"].append({"role": "assistant", "content": intro})
client.calls.actions.speak(ccid, payload=intro, voice="female", language_code="en-US")
elif call["state"] == "tutoring" and speech:
call["conversation"].append({"role": "user", "content": speech})
response = call_inference(call["conversation"])
call["conversation"].append({"role": "assistant", "content": response})
client.calls.actions.speak(ccid, payload=response, voice="female", language_code="en-US")
elif event_type == "call.hangup":
call = active_calls.pop(ccid, None)
if call and call.get("conversation"):
session_history.append({"caller": call["caller"], "language": call.get("language", {}).get("name"), "exchanges": len(call["conversation"]) // 2})
The system prompt is the only thing that needs to change to repurpose this app for a different domain — swap "language tutor" for "support agent" or "intake assistant" and the same state machine drives a completely different conversation.
Run It
git clone https://github.com/team-telnyx/telnyx-code-examples.git
cd telnyx-code-examples/ai-language-learning-phone-tutor-python
cp .env.example .env # add TELNYX_API_KEY, TELNYX_PUBLIC_KEY, TUTOR_NUMBER
pip install -r requirements.txt
python app.py
The server starts on http://localhost:5000. Expose it for webhooks with a tunnel:
ngrok http 5000
Copy the HTTPS URL and configure it in the Telnyx Portal:
Call Control Application → Webhook URL → https://.ngrok.io/webhooks/voice
Then call your Telnyx number from any phone. The app answers, greets you, and asks you to pick a language. Speak or dial a digit, and the tutor starts the lesson.
Check the health and recent sessions:
curl http://localhost:5000/health
{"status":"ok","active":1,"sessions":0}
curl http://localhost:5000/sessions
{"sessions":[{"caller":"+12125551234","language":"Spanish","exchanges":6}]}
Choosing a Model
The sample defaults to AI_MODEL=moonshotai/Kimi-K2.6 with max_tokens=200. Kimi-K2.6 is a reasoning model — it spends tokens on internal chain-of-thought before producing a final answer. On a phone call where every second of silence counts, that reasoning budget can consume the entire token window and leave the caller with no spoken response.
For a phone tutor, two practical configurations:
Keep Kimi-K2.6, raise max_tokens — Set max_tokens=1000 or higher so the model has room for both reasoning and a usable response. Latency goes up, but the tutor still speaks.
Switch to a non-reasoning model — Set AI_MODEL=meta-llama/Llama-3.3-70B-Instruct (or any other chat model on Telnyx AI Inference) for snappier responses with no reasoning overhead. With max_tokens=200, the caller hears a reply in well under a second of generation.
Set the model and token ceiling in .env:
AI_MODEL=meta-llama/Llama-3.3-70B-Instruct
Optionally override max_tokens in app.py's call_inference() default
See the available models list for the full catalog.
Available Languages
The app ships with four languages out of the box:
Digit Language Code
1 Spanish es
2 French fr
3 Japanese ja
4 Mandarin zh
Add more by extending the LANGUAGES dict in app.py and updating the greeting prompt:
LANGUAGES = {
"1": {"name": "Spanish", "code": "es"},
"2": {"name": "French", "code": "fr"},
"3": {"name": "Japanese", "code": "ja"},
"4": {"name": "Mandarin", "code": "zh"},
"5": {"name": "German", "code": "de"},
"6": {"name": "Korean", "code": "ko"},
}
The system prompt is generated from the language name, so any language with a working TTS voice on Telnyx works without further changes.
Going to Production
This example uses in-memory dicts for call state and session history, which is fine for local testing but loses state on restart. For production:
Persistent state — Replace active_calls and session_history with Redis or PostgreSQL so call state survives restarts and scales across multiple Flask workers
Model selection — Pick a non-reasoning chat model for sub-second phone responses, or tune max_tokens carefully if you keep a reasoning model
TTS language matching — The sample speaks the tutor's responses with language_code="en-US". For authentic pronunciation in the target language, map the language_code to the selected language (e.g. es-ES for Spanish, ja-JP for Japanese) and pick an appropriate voice
Conversation length limits — Cap call["conversation"] to the last N turns before sending to inference, so long calls do not blow past the model's context window
Authentication — Add API key validation on /sessions and /health if the app is publicly reachable
Webhook signature verification — Already implemented via client.webhooks.unwrap; ensure TELNYX_PUBLIC_KEY is set in .env or the app will reject every event
Error recovery — If call_inference raises (timeout, non-200), speak a graceful "let me try that again" message and re-gather rather than dropping the call
Monitoring — Add structured logging and alert on inference failures, gather timeouts, and call hangup rates so you catch upstream issues early
Frequently Asked Questions
Do I need a real phone number to test this? Yes. You need a Telnyx number with voice enabled and a Call Control Application configured with your webhook URL. You can use a free Telnyx trial balance to receive an inbound call for testing.
Why does the caller sometimes hear silence after speaking? If AI_MODEL is a reasoning model like Kimi-K2.6 and max_tokens is too low (200 or less), the model can spend the entire token budget on internal reasoning and return an empty content field. The TTS then has nothing to say. Fix it by raising max_tokens to 1000+ or switching to a non-reasoning chat model. See the "Choosing a Model" section above.
How does the app know what the caller said? Telnyx Call Control's gather action transcribes the caller's speech and returns the transcript in the call.gather.ended webhook payload under payload.speech.result. The app sends that transcript to AI Inference as the next user message.
Can I use this for a language that Telnyx TTS does not support? The conversation still works — AI Inference can generate text in any language the model supports. But TTS playback needs a matching voice and language_code. If Telnyx does not offer a voice for your target language, the app falls back to speaking the model's text with an English voice, which works for comprehension practice but not pronunciation.
Is the conversation context sent to the model on every turn? Yes. The full call["conversation"] list is sent to AI Inference on each turn, so the model has the full context. For very long calls, cap the list to the last N turns to stay within the model's context window and keep latency bounded.
How does the app handle concurrent calls? Each call is tracked in active_calls keyed by call_control_id, so multiple calls can run in parallel. Flask's default development server handles one request at a time; for production load, run under a WSGI server like gunicorn with multiple workers and move active_calls to Redis so all workers share state.
What happens if the caller hangs up mid-sentence? Telnyx sends a call.hangup event. The app removes the call from active_calls, logs the session to session_history (caller, language, exchange count), and returns a 200. Any in-flight call_inference request completes but its response is discarded because the call is no longer active.
Resources
Call Control docs
Call Control API reference
AI Inference docs
AI Inference API reference — chat completions
Available AI Inference models
Webhook signing and verification
Telnyx Portal — API keys
Telnyx Portal — Call Control Applications
Full code example
If you build something interesting with this pattern — a support agent, a voice survey, an intake bot — let me know in the comments. The state machine is more general than the example.
Top comments (0)