DEV Community

Dialog Brain
Dialog Brain

Posted on Originally published at dialogbrain.com

Telegram voice via MTProto: what the Bot API cannot do

Bots cannot place, answer or join a Telegram call. This is the layer that can, the constraints it puts on you, and the three audio rules we learned from calls that went out sounding wrong.

The Bot API has no voice calls at all

This is usually discovered late, after the bot is built and someone asks it to ring a customer. The Bot API can send a voice message and a video note, recordings, files with a play button. It has no method to place a call, no method to answer one, and no way to join a group voice chat.

No token scope changes that. Calls are not a permission a bot can be granted; they live on a different protocol layer entirely.

# There is no such method. Not "restricted", not "beta": absent.
curl "https://api.telegram.org/bot$TOKEN/getMe"           # works
curl "https://api.telegram.org/bot$TOKEN/sendVoice"       # a voice *file*
curl "https://api.telegram.org/bot$TOKEN/startCall"       # 404: no such method

# The Bot API surface has sendVoice and sendVideoNote: recordings.
# Placing, answering or joining a live call is not part of it.
Enter fullscreen mode Exit fullscreen mode

MTProto is a user account, not a bot

Telegram calls run over MTProto, the protocol the real clients speak. To take part, software has to sign in as a user account, the same way the desktop app does: phone number, login code, session. That has consequences worth deciding on before you build.

The caller sees a person, not a bot: your account name is what rings. The account is subject to the ordinary rules of a Telegram account, including the ones about automation. And an account is a single physical line, which brings the first constraint.

One call at a time, and no ringing callback

MTProto allows exactly one 1:1 call per account at a time. A second outbound call while the first is up is refused by Telegram itself, so a system that dials on demand needs either a pool of accounts or a queue, and it needs to know which account is busy before it tries.

The subtler constraint is that there is no pre-pickup event. Nothing tells you "it is ringing" and then "they answered". You learn the call connected because the playback call returns normally instead of raising, and every failure mode arrives as a different exception: declined, busy, rang out, cancelled mid-ring. If your product needs an "answered" timestamp, for billing, for a follow-up, for an answering-machine rule, that return is the only edge you get.

# MTProto gives no "ringing → answered" callback. You learn the callee
# picked up because the play call returns instead of raising.
try:
    await pytgcalls.play(chat_id, audio_stream)     # returns == answered
    await post("/internal/voice/call-answered", {"call_id": call_id})
except CallDeclined:                                # they pressed decline
    outcome = "declined"
except CallBusy:                                    # already on a call
    outcome = "busy"
except TimedOutAnswer:                              # rang out
    outcome = "no_answer"
except CallDiscarded:                               # cancelled mid-ring
    outcome = "cancelled"
Enter fullscreen mode Exit fullscreen mode

Group voice chats are a different animal

A group voice chat is not a 1:1 call with more people. The agent joins as a named, visible participant, hears each speaker separately, and can speak into the room. That is what makes an assistant in a team call useful: it can attribute what was said to who said it, answer when addressed, and translate for one participant without translating for everyone.

It is also why an account already sitting in a group call will busy-decline personal calls: to Telegram it is on a line.

Bridging the audio is where it actually gets hard

Getting into the call is a week. Sounding right is the rest. Our agent lives in a media room; the call lives on Telegram; something has to carry frames between them. We wrote that bridge four times, Telegram, WhatsApp, Twilio, the browser, and fixed the same three defects in each, once per channel, before we made the rules shared.

Route by participant identity, never by "everything that is not me is the agent". A bridge that assumes the rest of the room is one speaker will happily interleave a human operator with the agent the moment somebody takes over a call. We shipped exactly that: the operator sounded rasping for four months.

Let exactly one browser publisher through, latest wins. Two tabs overlap for a second on every take-over; consuming both corrupts shared resampler state and produces audio nobody can debug from a log.

Mix several sources sample-wise with int16 saturation, never by concatenating into one queue. This is the one that passes every unit test.

# Rule 3, the one that sounds fine in tests and wrong on a call.
# Two sources are summed sample-wise with int16 saturation, never
# concatenated into one queue: a 10 ms interleave is audible as crackle
# even when one of the two sources is pure silence.

import audioop

def mix(tts_chunk: bytes, other: bytes, gain: float) -> bytes:
    if audioop.rms(other, 2) == 0:        # silence: return the chunk as is
        return tts_chunk
    if len(other) != len(tts_chunk):      # never mix mismatched lengths
        return tts_chunk
    return audioop.add(tts_chunk, audioop.mul(other, 2, gain), 2)
Enter fullscreen mode Exit fullscreen mode

And the rates will get you

Telegram audio is 48 kHz; agent rooms commonly run at 16 kHz. Every boundary needs a resampler, and, the part that is easy to miss, the amount you push per tick has to equal the amount of time that tick represents.

We once fed a bridge 10 ms of audio every 20 ms. Nothing errored. Every log line was healthy, the room state was healthy, the transcript was fine. The voice trembled, and only a person listening could say so. Feed rate is not something you can assert in a unit test; put a real ear on the call before you ship one.

# pytgcalls speaks 48 kHz. The agent room runs at 16 kHz.
# Resample at the boundary, and push exactly one tick's worth per tick:
#   48000 samples/s * 0.020 s * 2 bytes = 1920 bytes per 20 ms
#
# Push 960 bytes on a 20 ms tick and the caller hears a trembling,
# warbling voice: you are feeding at half real time. We shipped that
# bug once and only an ear caught it: every log line looked healthy.
Enter fullscreen mode Exit fullscreen mode

Or do none of this

All of the above is why we sell it. Connect a Telegram account once, then a call is one request: the agent dials, greets, holds the conversation, and the transcript lands in the same conversation as your text history with that contact.

The same endpoint reaches a group voice chat with an @name, a WhatsApp number, a phone number in E.164 or a Google Meet link, the target string decides the channel.

curl -X POST https://api.dialogbrain.com/v1/calls \
  -H "X-API-Key: $DIALOGBRAIN_KEY" \
  -H "Idempotency-Key: call-9f21c7" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": 118,
    "target": "telegram:123456789",
    "greeting": "Hi, this is the assistant calling about your order."
  }'

# 202 Accepted
# {"call_id":"7a6d…","status":"dialing","channel":"telegram",
#  "target":"telegram:123456789","thread_id":6808472}
Enter fullscreen mode Exit fullscreen mode

Questions and answers

Can a Telegram bot make or answer voice calls?

No. The Bot API has no call methods at all, it can send voice messages and video notes, which are recordings. Live calls require an account speaking MTProto, the protocol the regular Telegram clients use.

What is MTProto and why is it needed for calls?

MTProto is Telegram’s own client protocol. Voice calls, group voice chats and their media negotiation live there, not in the HTTP Bot API, so participating means signing in as a user account rather than as a bot.

How many Telegram calls can one account handle at once?

One 1:1 call at a time. Telegram refuses a second concurrent call on the same account, so concurrency needs several accounts or a queue, and the dispatcher has to know which account is busy before dialling.

How do you detect that the callee picked up?

There is no pre-pickup callback. The playback call returning normally is the answer signal; declined, busy, rang out and cancelled each arrive as a distinct exception instead.

Can an AI agent join a Telegram group voice chat?

Yes. It joins as a named participant, hears speakers individually, can answer when addressed and can translate the conversation live. An account in a group call will busy-decline personal calls while it is there.

Do I have to build the MTProto side myself?

No. DialogBrain connects your Telegram account and exposes calls as one API request: POST /v1/calls with an agent id and a target such as telegram:123456789 for a person or @group for a group voice chat.


Originally published at dialogbrain.com.

Top comments (0)