Building a free Urdu AI voice with Gemini TTS: three bugs we hit

We make Sitey, a small app for people in Pakistan who write in Urdu. In October we added a speaker button: every AI answer can be read aloud in Urdu, and you can send it to WhatsApp as a voice note. We also put the same voice on the web, free, at app.sitey.pk/urdu-ai-voice.

Getting a voice to talk was quick. Making it work properly took longer. These are the problems we hit, in case you are building text-to-speech for a language that isn't English.

Picking the voice: listen, don't read benchmarks

We first tried a different Urdu TTS service. Our founder listened and didn't like it. So we ran a small test: the same four lines through two Gemini TTS models and the tablet's own Android Urdu voice.

1. A line in Urdu script
2. The same line in Roman Urdu (Urdu typed in English letters, which is how most people here text)
3. An English line
4. A mixed Urdu and English line, because that's how people actually talk

Several Gemini voices each, saved as .wav files in one folder, and a human listened to all of them. The whole test cost well under ten cents. The winner was Aoede on gemini-3.8-flash-lite-tts. It handled all four lines, and it says "Sitey" right.

The mixed line was the useful one. Plenty of voices sound fine on clean Urdu and fall apart when an English brand name shows up in the middle of a sentence.

Bug 1: the voice read our instructions out loud

In the voice test, a style instruction placed in the text (something like "Read this warmly") was read aloud by some voices as part of the message.

The fix: send only the words.

body: JSON.stringify({
  contents: [{ parts: [{ text }] }],
  generationConfig: {
    responseModalities: ['AUDIO'],
    speechConfig: { voiceConfig: { prebuiltVoiceConfig: { voiceName: 'Aoede' } } },
  },
}),

The same thing happened in the app with labels and emojis. A wishes card titled "Short and sweet" got its title read before the wish. An emoji was read as a random syllable (the cake 🎂 came out as "chha"). Now the speaker only gets the message text, and emojis are stripped first. There is a unit test for that.

Bug 2: a loud noise instead of a voice

Older examples return raw 16-bit PCM, and you wrap it in a 44-byte WAV header yourself. We did that, and the phones played a loud noise.

Gemini now sends a finished WAV file, with a C2PA content-credentials chunk after the audio. We were putting a second WAV header around a file that already had one, so phones played the inner header and that chunk as sound.

The fix is to check the first four bytes and pass a real WAV through untouched:

const bytes = fromBase64(part.inlineData.data)
const isWav = bytes.length > 12 && String.fromCharCode(...bytes.subarray(0, 4)) === 'RIFF'
if (isWav) return { ok: true, audio: part.inlineData.data, usage }
// Otherwise it's raw PCM: read the rate from the mime type and add our own header.

Bug 3: WhatsApp cut off the end of voice notes

People sent the WAV to WhatsApp as a voice note, and the last part of the message was missing.

We didn't fully work out whether WhatsApp or the trailing C2PA chunk was to blame, and we didn't need to. The app now re-encodes the audio on the phone to AAC .m4a with Android's own encoder, drains the encoder to the end of the stream, and shares that. A device test checks that 3-second and 11.7-second notes keep their full length. On the tablet a real 4.60 s note came out as 4.61 s.

If you share generated audio into a chat app, test the received file's length, not just whether it plays.

Keeping it free without going broke

The voice runs on a Cloudflare Worker that holds the Gemini key; the app and the web page never see it. A few limits keep the cost predictable:

• The website gives 5 voices a day, up to 500 characters each, with a Cloudflare Turnstile check. The app gives 40 a day.
• There is a daily dollar cap on all AI calls, and a separate monthly cap for Gemini.
• If Gemini is busy, over its limit or offline, the app falls back to the phone's own Urdu voice. It sounds worse, but it still talks.
• We store the count and the cost of each call, never the text or the audio.

Lite TTS pricing at the time of writing is $0.50 per million input tokens and $6.00 per million audio tokens. A short Urdu announcement costs a fraction of a cent.

What I'd tell someone starting this

• Test with real mixed-language sentences, the way your users speak, and have a person listen.
• Send the model only the words you want spoken.
• Look at the first bytes of the audio before you wrap or convert it.
• Check the length of the audio after it goes through WhatsApp, not before.

You can try the voice at app.sitey.pk/urdu-ai-voice without signing up. Questions about Urdu TTS are welcome in the comments.