DEV Community: Servin Osmanov

My TTS read a whole novel. The model was the most reliable part of the pipeline

Servin Osmanov — Fri, 09 Oct 2026 18:01:17 +0000

In the first post of this series I mentioned, in a single table, that a synthesized voice had read an entire novel in Crimean Tatar: Merdiven ("Staircase") by Şamil Alâdin, sixteen chapters, 1 hour 58 minutes of audio. In the second one I promised the story behind that table. Here it is.

The short version, if you build anything like this: the model held up. The problems weren't in the model but in the text and audio preparation around it, which I'd written myself and could fix without retraining: splitting text into sentences, gluing short ones together, trimming breaths, pulling the book out of its file, checking the result. And a good share of those bugs simply cannot appear in a demo. They need a whole chapter, or a whole book, before they show up.

Thirty seconds of context

Crimean Tatar has a sound, къ (a /q/), that ready-made models from neighbouring languages flatten into a plain k, so qara ("black") comes out as kara. I picked a multilingual base that had heard a lot of Kazakh, Uyghur and Bashkir, and I write our къ, гъ, нъ as the Bashkir letters ҡ, ғ, ң, which the model already knows. From there it reads our text as if it were Bashkir. That whole story is in the first post.

The voice is a separate input. The model I fine-tuned learns the language, not the timbre. The timbre comes from a short reference clip, a few seconds of someone's recording with its exact transcript. Merdiven is read in the voice of Sevil, and every sentence of the book was conditioned on one and the same seven-second clip. Keep that in mind, it comes back later.

Fourteen sentences can't tell you how a voice behaves for ten minutes

Before the book, I compared model versions on fourteen probe sentences, stuffed on purpose with the hardest sounds we have: къ, гъ, нъ, дж. A fine regression set. It tells you nothing about whether the timbre drifts by the end of a chapter, what the model does with dialogue, names, numbers and one-word replies, or how it breathes. In a fifteen-second clip you don't hear any of that. In a book you hear all of it.

So the next step was one full chapter: chapter one of Merdiven, 1,608 words, about twelve minutes. The point was not to admire it. The point was to find what breaks.

First I made sure the text wasn't in the training material: zero matching runs of six consecutive words against nearly 1.4 million words. Otherwise the model wouldn't be reading, it would be remembering. Later the whole book went through the same check, also with zero matches.

Why this book? I love Şamil Alâdin's writing, and I wanted the first book a machine read in our language to be one of his. It also suited the model: even narrative prose. Humorous stories or children's books, where the voice has to play, were beyond it back then.

The first number was already a lesson. On the probe sentences, about 14% of characters came out wrong when I read the synthesis back with our own speech recogniser and compared it to the text. On the chapter's ordinary prose, under 2%. The model hadn't improved overnight. The probe set is hostile by design and ordinary text isn't. I no longer quote the probe number as if it described how the model actually reads. It's a tripwire, not an estimate.

Fourteen sentences couldn't tell two fixes apart

In the first post I told how I heard ниже (Russian for "lower") where the book said нидже ("how many"): my letter-mapping rule was also collapsing дж into ж, so the model was literally handed a Russian word. This is what happened next.

I compared five ways of spelling that sound on the input. On the fourteen probe sentences the two best were indistinguishable. Both fixed the word. So I re-rendered the whole chapter, 195 sentences, with each of them. One, the Tatar letter җ, fixed нидже and quietly damaged everything else: character errors across the chapter went up by 27%. Plain дж fixed the word and broke nothing.

Fourteen sentences said the two candidates were equally good. A chapter showed that one of them hurts. A fix that has only been checked on the example that was failing hasn't really been checked. Run it over enough ordinary text to see what else it touches.

What broke in chapter one

The verdict on chapter one surprised me. The voice held: the timbre barely changed from the first minute to the last, and the pace stayed even. The defects weren't in the model but in how text and audio were prepared around it. I'd written that part myself, which made it good news: none of it needed retraining.

Gluing short sentences dropped whole clauses

The model learned from clips no shorter than about three seconds, so I reasoned that a two-word line shouldn't go to it alone and glued short sentences to their neighbours. The logic felt airtight. I never tested it. That was a mistake.

On glued chunks the model started throwing out entire sentences:

asked to read	what came out
Men yañılğanım. O, oca degil eken.	O, oca degil eken.
– Yoq. Siz yañlışasıñız. Meni bir daa iç bir yerde köralmaycaqsıñız.	Meni bir daa iç bir yerde köralmaycaqsıñız.

"I was wrong. He isn't a teacher after all" turned into just "He isn't a teacher after all". The glued chunks were a quarter of the chapter's text and produced 43% of its errors.

Only then did I test the assumption itself and render the chapter with no gluing at all. Short sentences really are worse, roughly two and a half times the error rate, although in absolute terms that's about two percentage points. Errors keep falling until around seventy characters and barely move after that. And that penalty turned out smaller than the price of gluing, which was losing whole pieces of sentences. Gluing went off.

Chapter one rendered without gluing. Everything under 43 characters used to be glued to a neighbour, but errors keep falling until about seventy.

Before you build a workaround for something the model "can't handle", measure how badly it actually handles it. Here the workaround cost more than the problem it was meant to avoid.

The breath trimmer ate first words

Sometimes the synthesis opens a sentence with an inhale, like a human narrator would. I wrote a trimmer: everything before the first voiced sound counts as breath and gets cut. But plenty of first words start with a voiceless consonant. Çañ qaqıldı turned into Qaqıldı, Tışta, yeşergen… into yeşergen…. In 111 of 147 clips the cut landed inside the first word. The chapter scored better with no trimming at all than with my "fix".

The new trimmer looks at loudness instead of voicing. A consonant is quiet, but it's still far louder than the silence in front of it. With the new one, the chapter scores better than with no trimming.

Benchmark every post-processing step against doing nothing. Mine lost, and I only found out because I ran that comparison.

The 89% that was the trimmer grading itself

The first check reported that 89% of sentences opened with an inhale. Scary number. Then it turned out the trimmer was reporting on itself: it counted as breath everything before the first voiced sound, which included those same voiceless consonants. An honest measurement found a breath at the start of one sentence in five.

Most of the real breaths were somewhere else entirely: between words, exactly where the trimmer never looked. 76 of 195 sentences had them. I learned to turn those down to the level of the background noise without moving a single timestamp, and 68 of the 76 sentences transcribe exactly as they did before.

A component that defines what it detects will always agree with itself. And it only finds things where it looks.

Pace and pauses

The model reads faster than a human narrator: 17.4 characters per second against 14.5. A chapter I expected to run fifteen minutes came out at twelve. For an audiobook that sounds a little hurried.

The pauses between sentences I didn't make up. I measured them on ten minutes of a human narrator: the median pause between sentences was 288 milliseconds, and only three pauses in those ten minutes were longer than 0.6 seconds. So my pipeline puts 300 ms between sentences, a bit more when the speaker changes, and a bit more again between paragraphs.

After all of this, chapter one went from 2.56% character errors to 1.84%. Same model, same weights. What changed was what I fed it and what I did to the audio afterwards.

Sixteen chapters: bugs that only exist at book length

Then came the other fifteen chapters. I didn't re-render chapter one. It came out of the same pipeline, and after every fix below its extracted text was byte-for-byte identical to before. Which also means chapter one contained none of what came next, so a one-chapter test had no way to find it.

The three asterisks that separate scenes, * * *, went to the model as text, and it dutifully read them out.
The last paragraph of the novel is the author's dateline, "1940 s.". My extraction took it for ordinary text, so the novel ended with a year read aloud.
Five sentences were torn in half wherever a paragraph broke off on an ellipsis.

None of that is the model. It's extraction, the least glamorous code in the whole pipeline. Run it over the entire text before you trust it on a sample.

A guard refused to build chapter seven, and it was right

The packaging step has guards built in. In chapter seven one of them refused to build the chapter: the words of one sentence, joined back together, did not reproduce the original sentence. The cause was a double space in the book file.

You can't hear a double space, and you can't see it on screen. But a reader that renders the text word by word would have lost it without a sound. It was the second time on this project that a program refusing to run caught something no amount of listening ever would.

That incident also changed how the run behaves. If a guard fires in one chapter, the remaining chapters keep going. The failed chapter still doesn't get built, the rest just don't wait for it. A book that stops at chapter seven at night is a lost night.

The checker is a model too

I checked every chapter separately, and one looked bad: chapter four, 8.3% character errors, four times the book's median. I went to listen, and the chapter was fine. The whole error came from one two-word line, "– dep tüşündi.", on which the checking recogniser got stuck in a loop and wrote it out twenty-five times in a row. Without that line the chapter is at about 2%, like the rest. The same line tripped the check in chapter two as well, it just stood out less there. The machine that read the book hadn't failed. The machine that checked it had.

That's also why one average for the whole book would have been the wrong number. A single looping line wrecks the mean and says nothing about the reading. Per chapter the picture is flat: median 2.15%, best chapter 1.5%, worst 2.7%. Quality doesn't drift towards the end of the book or on long chapters. The longest, chapter two, runs nineteen and a half minutes and reads at 2.5%.

Hatched: what checker glitches on short lines added, not the reading itself. Nearly all of it is one line, "– dep tüşündi.", which the recogniser looped on in chapter four and again in chapter two.

If your evaluation runs on a model, it has failure modes of its own. Report per unit, keep the checker's failures visibly apart from the system's, and listen to an outlier before you believe it.

On cost, as I wrote in the first post: two hours of audio take about three hours of my home computer. Synthesis itself is less than half of that. More goes into tying every word to a timestamp and checking everything. It's worth it.

A reader that highlights the current word

I didn't want just an audio file. I wanted a reader: the text on screen, the voice reading it, the current word highlighted, and a click on any word jumps playback there.

For that you need the time of every word. The trick the whole project started from helped again: align audio to text you already know. Only now the known text is exactly what the model was given as input. Each sentence is aligned on its own, against its own text, and then shifted into the chapter's timeline. In all sixteen chapters 100% of the words have a timestamp. The only things left without one are dashes and symbols that aren't pronounced.

The doubtful timestamps aren't scattered at random. Most of them sit on the first word after a dialogue dash, which is exactly where the recognition check stumbles too. Two independent instruments point at the same spots. So a low word score is now my cue to listen to that sentence, not to delete the timestamp.

Two more guards come from earlier mistakes. The timeline that had drifted 44 seconds by the end of a chapter is in the first post. Since then the build compares the end of the last sentence with the length of the audio file and refuses to run if they disagree by more than half a second. The other one is from the нидже story. I made the first build of the reader on the old narration, from before the дж fix, and the only symptom was the voice saying ниже. Now every chapter stores a fingerprint of its audio file, so a stale narration shows up in the data and not only to someone listening carefully.

Put together, the packaging step looks roughly like this:

# a chapter is written only if these hold
assert " ".join(sentence.words) == sentence.text    # words rebuild the sentence exactly
assert abs(last_sentence_end - audio_seconds) <= 0.5  # the timeline matches the audio

# and the chapter records which audio it was built from
chapter["audio_fingerprint"] = content_hash(audio_file)  # stale narration is visible in data

None of these is clever. Two of them are only there because something had already got past me once.

The reader is live on ana-yurt.com, one page for both reading and listening. You can open Merdiven and listen to the whole thing: ana-yurt.com/kutuphane/3444/read.

The reference clip is an input too

Since the timbre comes from the reference, picking a voice means picking a few seconds of audio. That clip turned out to matter more than I expected.

I first tried my own voice on single sentences and heard long, out-of-place pauses in the middle of them. My first thought was the model. It was the reference: 0.77 seconds of silence before the words and 1.77 seconds after. The model takes more than timbre from the reference. It also takes the tempo, how much time to spend on each letter, and it was spending that extra time as pauses inside phrases. I trimmed the silence at the edges, and the median total pause inside a sentence fell from 930 to 85 milliseconds. In a blind comparison the voice with the trimmed reference couldn't be told apart from the old one: 7 to 6. The defect went away and the voice stayed.

Then the voice started to sound dull to me. This time it was the microphone: a lavalier clipped to the chest sits off to the side of the mouth and loses exactly the frequencies that were missing, and a flat setting didn't help. No filter can paint that back in. What the mic didn't record isn't in the recording. I recorded the references again.

I also tried to help the male voice through training, by adding more male speech to the training material. In a blind comparison the new version lost to the previous one, 3 to 11, and one of the female voices slid lower in pitch, from 208 to 199 Hz. I only added the pitch measurement after my ear had spoken, and both pointed at the same voice. That version didn't stay.

If you build voice cloning into a pipeline, treat the reference clip like any other input data and check it. A bit of silence at its edges turned into pauses in the middle of every sentence.

Once the pauses and the microphone were sorted out, I decided to try my own voice on a whole book. So the second book in our library, Yunus Qandım's Küreş meydanını ot basmaz, 28 chapters, 6 hours 29 minutes of audio, was read by the machine in my voice. You can listen to it as well: ana-yurt.com/kutuphane/3516/read. It was read by the current version of the model. Merdiven was read by an earlier one.

This book wasn't a random pick either. Its author, Yunus Qandım, is my uncle, and I really wanted his book to be heard in my voice.

Usually, when you hear your own voice in a recording, it sounds foreign and a bit strange. This time it didn't. When I heard the first sentence, I was over the moon: the quality was very good.

What still doesn't work

Stress. This one went in a circle, and I care about it more than the rest. I noticed the model sometimes reads the name Riza as Rıza, with a back vowel. A check confirmed it happens in about half the sentences, and the same goes for Aziz. There was a fix: spell the vowel on the input as the Cyrillic «ий» instead of «и». The back vowel disappeared completely and the errors in the name went down. The instruments were happy.

I listened blind and rejected it. My note on the blind page said: "if I hear И, the stress is wrong." The fix brought back the right vowel and dragged the stress to the first syllable, while ours sits on the last. Both versions were wrong, each in its own way.

I wasn't too upset. These are first models, and I accept mistakes like this from them. I decided to come back to it later.

Then I started measuring the stress itself. In our language the last syllable of a word is usually stretched a little, and human narrators stretch some words a lot and others hardly at all. On average the model stretches the last syllable as much as people do. But it reproduces only about six tenths of that range: it under-stretches exactly the words narrators stretch most and over-stretches the flat ones.

So I went through the suspects one at a time. The reference clip changes only the overall tempo. The training recordings have the stretch where it belongs, so there was something to learn from. The inference settings change nothing. And the base model, before my fine-tuning, has the same flatness. The next suspect is how the model decides the length of the whole phrase up front. If the total duration is fixed in advance, the last syllable can only grow at its neighbours' expense. For now that's a hypothesis, and testing it means working on the model itself, not on the text that goes into it.

Breaths. Breath cleaning never made anything worse, but short breaths, around a tenth of a second, partly remain. My voice has the most: in one measurement breaths took up 15% of the audio, against about 1% for Sevil. Trimming the silence in the reference cut that by more than half, but not to zero.

Pauses on emotional lines. I had the impression that the model pauses longer where the text is more emotional. I tested that on sentences picked so that calm and emotional ones didn't differ in length or number of commas, and found no link. Pauses follow sentence length and commas, and emotional sentences in prose just tend to be longer. But I could only measure the total length of the pauses, not where they fall. A pause in the wrong place in an exclamation grates more than the same pause at a comma, so for me the question is still open.

Dialogue. The model sees the dash before a reply, but it reads a change of speaker as a new sentence, not a new voice.

Numbers. I turn them into words beforehand, but recognition can't check how they were read: it writes down what it hears as digits, and "1924" comes back as "19124". For now only an ear checks the numbers.

The whole list, in one place

what broke	what caught it	what's in the pipeline now
gluing short sentences dropped clauses	reading the audio back and comparing it with the text	no gluing; short lines cost less than lost ones
the breath trimmer cut into first words	the same readback, and a no-trim baseline that won	trimming on loudness, not voicing
"89% of sentences open with a breath"	measuring apart from the trimmer	breaths between words turned down, timestamps untouched
`* * *`, a dateline read aloud, five torn sentences	the whole-book run	fixed in extraction
a double space in the source file	a guard that refused to build	words must rebuild the sentence exactly
a timeline 44 seconds off by the end of a chapter	an ear, playing single sentences	timeline end vs audio length, half a second
a stale narration in the reader	an ear (ниже)	a fingerprint of every chapter's audio
chapter four at 8.3%	listening to the "bad" chapter	per-chapter numbers, checker loops kept apart
Riza stressed on the wrong syllable	an ear, blind	still open

The voice turned out to be the most reliable part of the whole chain. Nearly everything that broke was in the handling of text and audio around it: gluing, trimming, letters, timestamps, a double space. Some of it only instruments found, and no ear ever would have. Some of it only an ear found, and the instruments couldn't even score it: нидже, the stress in Riza, breaths between words. And the whole book found things that neither fourteen sentences nor one chapter would ever have shown.

When I finished listening to the book, I felt proud. The work hadn't been for nothing, and the result was excellent. But that's no reason to relax. You can still hear places in the book that need work, which means there's plenty of it ahead.

Third post in a series. Next I'll go deeper into how I chose a base for the voice that can already say our къ, and why neighbouring Turkish doesn't work for that. After that, as promised, cloud GPUs.

Audio samples and project page: https://ai.ana-yurt.dev/.

Want to help? What the project needs most is native speakers. Listen to a chapter and note where the stress lands in the wrong place, where a name is misread, where a pause tears a phrase apart. You don't need to program, just a good ear for your own language. In this book the most important finds were made by ear. Write to support@ana-yurt.com.

If you'd rather support the work financially, you can do it here: ko-fi.com/anayurt. That support turns into new recordings of living speech, and those need to be made now, while there are still people to learn from.

Fine-tuning was the easy part: what shipping ASR for a low-resource language taught me

Servin Osmanov — Wed, 23 Sep 2026 19:49:45 +0000

The fine-tune took an hour and a half on a single consumer GPU. Everything that made that hour and a half mean anything took months.

This is the second post in a series about building speech tech for Crimean Tatar, a language with no datasets to speak of. The first one covered synthesis and how the project started; this one is the recogniser — much less a story about training than about measurement.

The headline, so you can decide whether to keep reading:

	WER	CER
the model I started from	34.6%	11.9%
+ fine-tuning (≈1.5 h, one GPU)	20.1%	9.4%
+ tuning how it decodes	17.0%	7.0%

Left to right: starting model, after fine-tuning, after decode-time tuning. The third bar cost zero training.

Now the parts worth your time.

I gave the model an exam before I tried to teach it

I already had a "working" Crimean Tatar recogniser: a large Whisper, fine-tuned on our language, which I'd been using on clean studio audio where it looked fine. I had read its transcripts and nodded. I had never scored it.

So the first thing I did here was not improve anything. I measured. 34.6% WER — every third word wrong. That's not "rough": it can't back archive search, can't produce training pairs for synthesis, and an assistant built on it mishears you constantly. I only knew that because I finally scored the thing I'd been eyeballing for a year.

One footnote on that baseline: 34.6% slightly understates it, because the model writes 1954 where the book spells out Biñ doquz yüz elli dört and the scorer counts four substitutions. Small, but if I don't say it, the number is dressed up.

Hold out whole recordings, not random clips

Standard practice is a random 90/10 split. Here that would have been meaningless. Adjacent clips come from the same recording, same reader, same microphone, same afternoon — and the segmenter routinely cuts one sentence across two neighbouring clips, so a random split drops half a sentence in train and the other half in eval. What you measure then isn't language ability. It's whether the model memorised a voice and a room.

So I held out two entire books, two readers the model had never heard: 893 clips, 1 h 52 min. One move removes the voice, the mic, the recording era and the vocabulary of a particular text at once. If your data has structure — speaker, session, document, device — split on it. Random splitting is only safe when clips are genuinely independent, and in low-resource audio they essentially never are.

That feels like enough caution. It wasn't.

Audit your eval set by content, not by filename

Before launching anything I checked the obvious thing: is any of my held-out eval material also sitting in training?

By filename: zero overlap. Not one collision. The two name sets looked like they came from different universes.

Then I compared the text instead: for every eval clip, does any training clip contain the same run of six consecutive words? Word 6-grams — cheap to build, cheap to hash, long enough that natural coincidence is rare at this corpus size.

Four audiobooks were in my material twice. Same recordings, segmented by two different tools at two different times, under completely unrelated names.

book	share of its clips with a twin in training
book I'd picked for eval	96.9% (651 / 672)
second eval book	58.8% (130 / 221)
third book	82.1% (769 / 937)
fourth book	69.8% (549 / 787)

Four books stored twice. Zero overlap by filename; the collision only showed up in the text.

Top row is the book I was about to grade the model on.

I wasn't panicked. I just understood two things at once: every measurement I'd ever taken on this data was void, and thank God it surfaced before training rather than after.

The counterfactual is the point. If I hadn't checked, nothing would have gone wrong. No crash, no warning, no weird loss curve. The model trains on recordings it will later be examined on — different cuts, same sentences, same voices — posts a great number, and I publish it. The number means nothing: the exam questions were handed out in advance. That's the nasty property of this bug class. It has no symptoms. Everything works, the result is better than expected — which is exactly what should make you suspicious.

And it generalises: in a low-resource language, re-cut copies of the same recordings are the norm, not the exception. There is little material, so you keep returning to it with new tooling, and every pass produces fresh filenames wrapping old content. Deduplicate by content. Filenames are not identity.

After cleaning I re-verified rather than trusting myself: zero eval clips in training, zero 6-gram matches. Measured and written down, not "I'm pretty sure I got them all".

Don't judge a fine-tune by its first checkpoint

Now the part everyone thinks is the project, which was the fastest and dullest bit of it.

I didn't retrain the model. I froze it and trained a small adapter alongside — 2% of parameters, 31M trainable against ~1.5B frozen. 15.5 hours of speech survived the cleanup, 3 epochs, 705 steps, 1 h 30 min on one home GPU, 10.4 GB peak VRAM, and a finished adapter of 126 MB. Result: 20.1% WER, 9.4% CER — a 42% relative cut in word errors.

I felt nothing at the time; a number moved as designed. The pride arrived later, when I ran the model over material it had no business handling — films, podcasts, noisy field recordings — and the transcripts came back near-perfect. That's when it stopped being a table row and became working ears.

Two things from the run are more useful than the headline number.

Checkpoint 100 was worse than the base model. I saved every 100 steps and scored all of them. The first checkpoint came out noticeably worse than what I started from: the adapter has half-learned the new behaviour, it's already interfering with the old one, and it isn't giving anything back yet. If I'd glanced at it and concluded the approach was a dud, I'd have killed a run that ended up cutting error by 42%. A dip at the first checkpoint is what partial adaptation looks like, not what failure looks like.

All the gains landed in epoch 1 of 3. Past roughly step 200 the numbers stopped moving; epochs two and three contributed nothing measurable. So "1.5 hours" is itself padded — half an hour would have done. Checkpoint often and score often; runs converge long before the schedule says so.

One detail that reassured me more than the average: both eval books improved, and the smaller, later, different-reader one improved more. That's the signal you want — the model didn't just learn one narrator's delivery.

Decode-time wins are real — decompose them anyway

Then error dropped again with no training at all.

A trained model has knowledge, but it also has a decoding policy. The default is greedy — take the locally most probable token, never reconsider. Fast, occasionally catastrophic: take one wrong turn and it can't back out, so it goes in circles. That's exactly what the worst eval clips were. Not mishearing — looping:

Qıznıñ qıznıñ qıznıñ qıznıñ …
– dedi Akimoviç, – dedi Akimoviç, – dedi Akimoviç, …

The fix is beam search: hold several candidate sequences and score whole hypotheses rather than committing token by token. Costs decode time, costs zero training. I swept 24 configurations on a separate selection set, not on the exam (more below), then ran the winner on the test set once. 17.0% WER, 7.0% CER. Not one weight changed. Decoding got 3.1× slower.

Here it would have been easy to write "beam search: −3 WER points" and move on. True, and half the truth. So I broke the win down.

409 word errors were recovered in total. 160 of them came from 3 clips out of 893. 0.34% of the material delivered 39% of the gain. Those are the loopers: on 41 reference words the greedy model emitted 175 words of repeated garbage. The remaining 890 clips improved by a modest, even 10%.

Both halves are real, and neither alone justifies a 3.1× slowdown. But they mean different things. The broad 10% is "the model got more careful". The three clips are "it stopped occasionally losing its mind". For archive transcription the second matters far more, because a clip containing 500 bytes of one repeated token isn't merely bad — it poisons anything you train on it later. After the change: zero looping clips, 0 of 893.

Decompose your aggregate wins. If most of your improvement lives in a handful of examples, you didn't improve accuracy; you fixed a failure mode. Those call for different follow-up work, and averaging hides the difference.

The obvious anti-loop knob broke real language

There's a more direct tool against repetition: forbid the model from emitting the same n-gram twice. On paper, exactly the right hammer.

I tested it. It was net harmful: across 255 selection clips it broke 17 and fixed 1, and 12 of the 17 it broke had been essentially perfect. The transcripts explain themselves. The constraint doesn't stop the model repeating — it forces it to repeat incorrectly:

book        – Qartbaba, qartbaba, – tez-tez çapıp kelgen kiçkene Aziz
unrestricted– Qartbaba! Qartbaba! – tez-tez çapıp kelgen kiçkene Aziz
restricted  – Qartbaba! Qartbabaa! – tez-tez çapıp kelgen kiçkene Aziz
harder      – Qartbaba! Qartbava! – tez-tez çapıp kelgen kiçkene Aziz

A child runs up calling "Qartbaba, qartbaba!" — grandpa, grandpa. The model heard it perfectly. The ban wouldn't let it write down what it heard, so it corrupted the second word to avoid matching the first. Elsewhere bağışla… bağışla… ("forgive me… forgive me…") came back the second time as the non-word Bağuşla; in a third case the model routed around the constraint by inventing a suffix, parçalarnıñ parça instead of parçalar. Doubled address and doubled pleading are ordinary Crimean Tatar. People talk that way.

This is my favourite finding of the stage, and it isn't really about decoding. You cannot fix a language with prohibitions. Any rule of the form "this never occurs" eventually collides with how people actually speak, and it breaks precisely that spot while leaving everything else intact — so you won't notice. A decoding default borrowed from an English-language tutorial encodes assumptions about English. Test it against your own language's rhetoric first.

Three sets, and the test set gets decoded once per decision

I've mentioned a "selection set" twice. Every number above rests on it. There are three sets, not two.

Train — the model learns here.
Dev / selection — 255 clips, half an hour, two short whole works pulled out of the training material. This is where I choose: which checkpoint, which of the 24 decode configs. Dozens of evaluations, as many as I liked.
Test — the two held-out books. Decoded exactly twice all stage: once on the winning checkpoint, once on the winning decode config. The selection rule was written down before I looked at any result.

Why so strict? Because if you run the test set repeatedly and take the best result each time, you're no longer selecting the best model — you're selecting the luckiest one. The number comes out pretty and never reproduces.

The dev set got the same cleanliness audit: zero shared clips, zero 6-gram matches. After the duplicate story I don't file that under paranoia.

Your dev set measures ranking, not absolute quality

The dev set was much easier than the test set. Same model: 17% WER on dev, 34.6% on test. Both honestly held out, neither seen in training. But the dev material comes from the same family of recordings as training, while the test books are two entirely foreign readers.

So dev is good for "config A beats config B" and useless for "the model errs at X%". Report the dev number as your result and you're not lying exactly — you're reporting the easier exam. Keep both in their lanes: dev ranks, test reports.

The tool I couldn't validate gets no number

One consequence of that discipline. Whisper ships a temperature-fallback mechanism against repetition: it detects circular output and retries. Precisely the tool this problem wants, and nearly free at decode time.

But my dev set contained zero looping clips. Nothing for it to act on — it looked like a config that does nothing at all. By my own selection rule I couldn't pick it, and I refused to sneak it onto the test set afterwards to see whether it would have helped. So I have no number for it, and I'm not inventing one. If you couldn't validate a tool on the set you're allowed to validate on, write down that it's unvalidated and keep it out of the results table.

17% in the studio. What about the world?

Everything above was measured on one kind of audio: one person, one mic, quiet room, reading a book. Honest for its own world, and it says little about the world a voice assistant lives in. So I took the same 893 clips and corrupted them 17 ways — noise, music, competing speech, reverb, telephone-band filtering, time-stretching. The reference text is known, so error is measurable in each.

Telephone band and aggressive compression are free. Through a telephone-band filter the model does a hair better than in the studio. Speech rate barely matters either: ±15% tempo costs at most 6%.
Competing speech is the enemy, by a wide margin. Babble costs roughly 2× what steady noise at the same loudness costs, ~4× what music costs. At equal loudness with the target, WER goes from 17% to 69% — the model stops working.
Reverb is second. A hallway-sized room multiplies error by 2.5×.

That points somewhere specific: the highest-leverage investment in robustness is not more data. It's source separation and microphone placement. More training hours will not fix a model whose input holds two people at equal volume.

One last check that made me happy: on an open Crimean Tatar audiobook set the model had never trained on, clean chapters landed at essentially the same number as my own exam. It didn't memorise my readers. It hears the language.

What I'd actually tell you

Training was 90 minutes. The months went into auditing the eval set by content, splitting by recording, checkpointing and scoring instead of trusting the schedule, decomposing wins instead of quoting averages, refusing to let an English-shaped default constrain our grammar, and keeping a test set clean enough that its number means something.

Fine-tuning is the part with a progress bar. It is not the part where the work is.

Second post in a series. Coming up: a synthesised voice reading an entire book — sixteen chapters, nearly two hours of audio — and what had to be verified chapter by chapter. Then the one where I rented somebody else's GPUs and set money on fire, computing almost nothing.

Audio samples and project page: https://ai.ana-yurt.dev/.

Two ways to help. If you speak Crimean Tatar, the highest-value contribution is fifteen minutes with headphones — listen to a clip, say whether the machine wrote it down correctly, fix a transcript. No programming needed, and ears like that caught things here that no automated check did; write to support@ana-yurt.com. If you'd rather fuel the work directly, there's ko-fi.com/anayurt.

What I learned building speech tech for a language with zero datasets

Servin Osmanov — Thu, 10 Sep 2026 00:44:51 +0000

The word was qara, "black". I fed it to an off-the-shelf TTS model, hit play, and got back a soft, polite, very Turkish kara.

In Crimean Tatar къ and к are two separate sounds, and къ sits inside words we say every day: yoq, qız, vaqıt, qadar. Getting it wrong is not an accent. It is the model quietly speaking a neighbouring language and hoping nobody notices.

I noticed. That one syllable is why I spent the next several months building speech models for Crimean Tatar on a single consumer GPU, in the evenings, with no dataset to start from. What follows is the engineering residue: what I would tell a solo developer about to do the same for their own low-resource language. Most of it cost me a week per lesson.

The chicken and the egg were both endangered

The standard way to build a speech dataset is to take audio and run ASR over it to get text. Except we had no usable ASR for Crimean Tatar, and you cannot build good ASR without a corpus. Classic chicken and egg, except the chicken is also endangered.

I stared at that loop longer than I want to admit before noticing I was holding the problem the wrong way round. Audiobooks come with the text. It is printed. It exists. There is nothing to recognise.

So I stopped trying to transcribe and started trying to align: known text, audio, find where each word lands, cut on those boundaries. Forced alignment instead of recognition. That flip is the reason the project exists at all, because it sidesteps the exact thing the language did not have.

If you take one thing from this post, take that one. Before training a recogniser, check whether the ground truth is already sitting on a shelf. If it is, you have an alignment problem, and alignment is enormously cheaper.

The run that lied to me for three and a half hours

The first big alignment run covered seven and a half hours of audio. It ran, it did not error, and it produced clean, confident, well-formatted output.

The first two hours were correct. The remaining three and a half hours were garbage. Not obviously-broken garbage either: plausible timestamps, plausible segments, looking exactly like the good ones. Out of seven hours I got a bit under two of usable material, and I only found out because I went listening.

I have come to think of a crash as a gift. A crash points at the line. A process that returns confident nonsense takes a week off your life, slowly.

What fixed it was not a code change but a granularity change. Instead of feeding the aligner three enormous blocks, I split the material into forty chapters and located each independently. Same audio, same text, same tooling, and over five hours of clean speech instead of under two.

That is the rule I kept: if a stage can fail silently, never run it on big units. Small units fail visibly, and they let you bin the bad ones instead of the whole batch.

Words do not line up. Letters do.

That chapter-matching step had its own trap waiting. Locating a chapter inside a book sounds like a string search problem, and my first instinct was word-level similarity. It found nothing. I tested it on a pair I knew matched, checked by hand, and the similarity came back under one percent.

The reason will bite anyone working outside English. Crimean Tatar is agglutinative, so the "same" word carries different suffixes in the two sources. Add two alphabets in circulation (Cyrillic and Latin), and word tokens simply do not line up.

Switching the comparison to short character sequences fixed it in one go: 38 of 40 chapters located. The characters survive the suffixes. The words do not. For anything morphologically rich or multi-script, character-level matching is the default and word tokens are an English-shaped assumption.

The base model had already decided

Back to qara. My first assumption was the obvious one: the model had never heard the sound. Easy to test, so I checked. In my data the letter q showed up in 88% of recordings, more often than k. The model had been shown the correct pronunciation thousands of times and had cheerfully ignored every one.

The problem sat underneath the fine-tuning. In the base model, before it ever saw a word of my language, "q" was already wired to /k/ by English, Spanish and French. Fine-tuning was not overwriting that prior, it was decorating it. Which leads somewhere unpleasant: more data would not have helped. Not ten more hours, not a hundred. The fix had to happen at the foundation.

So I went base-model shopping with a different question. Not "which base speaks a language close to mine" but "which base already produces the sound I keep losing".

Everybody's first suggestion is Turkish, the big well-resourced relative. But Turkish has no /q/ at all, only one k, which is exactly how you get kara. I verified that twice, because I did not trust my own ears the first time. Azerbaijani looks promising until you listen: its q leans towards /g/, then softens further into something breathy. The languages that keep the sound, with a dedicated letter of its own, are Kazakh, Uyghur and Bashkir.

The multilingual base I ended up choosing had roughly 1500 hours of Kazakh, 400 of Uyghur and 250 of Bashkir in its training mix, against 125 hours of Turkish. Exactly upside down from what "pick the closest big language" would tell you.

It still had no Crimean Tatar in it. I got around that at the writing-system level: our къ, гъ and нъ each map onto a single letter the model already knows (ҡ, ғ, ң), so you hand it Crimean Tatar text and it reads something familiar. One trick, two problems.

The nice part is that this is verifiable before you spend a single GPU-hour on training. Out of the box, with no fine-tuning at all, the new base pronounced our къ correctly 88–100% of the time. I was no longer teaching the model a new sound. I had stopped fighting it.

I took the Kazakh and Bashkir variants all the way to the end, because I wanted the comparison rather than the argument, and they tied to the third decimal place. I shipped Bashkir: at a certain point in training the Kazakh one returned a few empty phrases and let the /q/ slide back towards k. Given equal numbers, I take the one that breaks less often.

For a language with no data of its own, picking the base is a phonetics decision, not a taxonomy one. And you can smoke-test the prior in an afternoon.

Three times the data, and the number would not move

My favourite negative result of the project. The corpus kept growing: 5.9 hours, then 11.5, then 15.3. I tracked character error rate by synthesising speech and running it back through a recogniser, averaged over three runs each:

corpus	CER (mean of 3)
base, no Crimean Tatar at all	0.173–0.198
5.9 h	0.136
11.5 h	0.138
15.3 h	0.141

Two and a half times the data and the number goes slightly worse. Meanwhile the run-to-run spread on one unchanged voice was wide enough to swallow that whole difference, so the honest reading is that all three runs are indistinguishable by this metric. And yet in blind listening, not knowing which sample was which, I picked the newer voice every time.

The corpus grew two and a half times over. The automatic metric flatlined. Blind listening kept preferring the newer voice anyway.

The explanation matters more than the numbers. Intelligibility saturates early, around six hours. Prosody, breathing, the liveliness of a sentence keep improving well past the point where CER stops moving. The metric goes quiet exactly where the remaining quality lives, so anyone reading that table alone would have stopped collecting data, reporting the truth while making the wrong call.

The cheaper form of the same lesson: one measurement is not a measurement. The same voice gave one number on the first run and a noticeably different one averaged over three. I have deleted several "wins" that turned out to be variance wearing a hat. Measure three times, and never give a single automatic metric the only vote.

The bug you can only hear if you click one sentence

I was listening to a narrated chapter, clicked on sentence 195 to check something, and heard the wrong sentence. Not slightly wrong, a different sentence. The drift was beautifully linear:

sentence	timestamp offset
1	0 s
49	+10.7 s
98	+21.8 s
147	+33.0 s
195	+43.9 s

My first theory was the fashionable one: the model falls apart over long text. Wrong, and worth checking rather than assuming. Speaker similarity to the opening held at 0.96–0.98 throughout, and the tempo never shifted. The audio was fine.

The actual cause: two parts of my pipeline computed time independently, and one counted the pauses between phrases twice. Forty-three seconds is exactly the sum of those double-counted pauses across the chapter. Correct audio, wrong labels.

I did not debug this by reading code. I debugged it with a knife: cut the audio at the old timestamps and at the new ones, push both through recognition, compare. New marks came back at CER 0.008, so the text matches. Old marks came back at 0.712, so those are other sentences entirely. On the final line the old timestamp had run past the end of the file, and the recogniser dutifully "heard" its own boilerplate captions in the silence.

The part that should worry you: played straight through, the file sounds perfect. Every consumer of those timestamps, a word-highlighting reader or a training pipeline, would have inherited garbage from a file that passes any listening test you can run. Whole-file playback validates whole-file playback and nothing else.

The exam the model had already seen

The ASR side of the project produced numbers I liked:

	WER	CER
the model I started from	34.6%	11.9%
+ fine-tuning (about 1.5 h on one GPU)	20.1%	9.4%
+ tuning how the model decodes its answer	17.0%	7.0%

Left to right: starting model, after fine-tuning, after decoding changes.

The third row involved no training at all, not a single weight touched, only a more careful decoding setup, and it moved WER by three points. The entire adapter I added to that model is 126 MB.

Now the part that nearly ruined it. The same four audiobooks existed twice in my material. Cut differently, segmented differently, stored under unrelated filenames, zero overlap by name. For one book I had picked as an eval set, 96.9% of the segments had a twin sitting in the training data.

If I had missed it, I would have been examining the model on questions it had already been handed the answers to. A lovely number, published, believed, with the next six months built on top.

Filenames are not identity. I check for near-duplicates by content now, always, and I treat a suspiciously good eval score as a bug report. Same reflex later, when a synthesised voice read a whole book: I proved by measurement, not by reasoning, that the book's text never entered training. Zero word-sequence matches against a corpus of nearly one and a half million words. Otherwise it is not reading, it is reciting.

What it costs now

A synthesised voice read an entire book, Merdiven by Şamil Alâdin. Not a demo paragraph. All sixteen chapters, 15,444 words, one hour fifty-eight minutes of audio, word timings verified across 100% of the words, per-chapter CER median 0.022 (worst 0.027, best 0.015).

The timings got a control that was required to fail: cut 30 words per chapter at the recorded marks and recognise them, 447 of 480 matched; shift the same fragments by 0.4 seconds and 4 of 480 match. That gap is what makes the check worth running.

And the number I care about most: roughly three hours of machine time per two hours of finished audio, everything included. An audiobook in Crimean Tatar now costs one night of one home computer. Not a studio, not a year of waiting.

Earlier work is public if you want a starting point: the dataset and the model.

Where the months actually went

Training was hours. An hour and a half here, a few hours there. It is not the line item anybody thinks it is.

The months went into collecting and aligning data, repairing the parts that broke without saying anything, proving the evaluation was honest, re-measuring, and listening with headphones, blind, one sentence at a time.

That last one produced the finding I did not expect. Across the project the bugs split roughly half and half between instruments and a human ear. Instruments caught the drifting timestamps, which no amount of listening would ever have revealed. The ear caught breaths in wrong places, bad annotations, and this one:

I heard ниже where the book said нидже. The cause took a minute to find. My letter-mapping rule, the same trick that gave me ҡ and ғ, was also collapsing дж into ж. The model had been handed the string "ниже" and pronounced it flawlessly. Perfect execution of a wrong input.

The rule had erased a distinction the language genuinely has: дж in our own words like джан, оджа, нидже, versus ж in loanwords. After that collapse nothing downstream could have said them differently. That chapter held fifty-five such words. I caught exactly one, the case that happened to land on another real word. The other fifty-four sounded "a bit off", and the ear slides right over "a bit off".

Fixing it barely moved any metric. I fixed it anyway, because it is our language, not a leaderboard.

Which is the most useful thing I can hand anyone doing this for their own language: your cheapest and strongest tool is one native speaker, fifteen minutes and headphones. It beats the automation, and it means you do not have to be a programmer to contribute. You need to hear your language well.

First post in a series. Coming up: teaching a recogniser to hear Crimean Tatar; narrating a book chapter by chapter; running blind A/B tests so you stop fooling yourself; and the one where I set money on fire renting somebody else's GPUs and computed almost nothing.

Voice samples and project page: https://ai.ana-yurt.dev/blog/2026-09-kak-rodilas-ideya/.

Two ways to help. If you're a Crimean Tatar speaker, the highest-value contribution is fifteen minutes with headphones — blind listening and proofreading beat any tool in my stack; write to support@ana-yurt.com. If you'd rather fuel the work directly, there's ko-fi.com/anayurt.

My Journey Improving a TTS Model for the Crimean Tatar Language

Servin Osmanov — Fri, 07 Nov 2025 19:42:08 +0000

When you work with machine learning, success often hides behind hours of frustration, countless errors, and broken pipelines. This project — improving the Crimean Tatar TTS (Text-to-Speech) model — was exactly that kind of journey. What started as a small experiment to fine-tune an existing model turned into a full-scale debugging adventure that taught me more about data integrity, audio processing, and patience than any tutorial could.

In my previous article, Why Language Tech Matters: Developing AI Tools for Small Languages, I explored how AI can empower low-resource languages. This piece continues that journey with a hands-on look at improving TTS models.

The Starting Point: A Model That Worked — but Only Partially

My goal was simple: improve the voice model “Sevil” for the Crimean Tatar language. I had already worked with similar voices — “Arslan” and “Abibullah” — using Hugging Face datasets like speech-uk/tts-crh-arslan and speech-uk/tts-crh-abibullah.

The first training attempts went well — loss around 0.283, results acceptable. But something didn’t add up. The dataset had 1,566 audio files, yet the training logs showed only 415 were being used — about 26.5% of the total.

That meant almost three-quarters of my data was silently ignored.

At first, I thought it was a fluke. Then I realized it was a systemic problem in the Hugging Face datasets API when loading compressed audio from Parquet files.

Diagnosing the Problem

When I loaded the dataset through datasets.load_dataset(), most files threw errors:

Error: "Error while decoding audio"
Error: "Audio file appears to be empty"

That didn’t make sense — the audio bytes were clearly present in the Parquet files.

After checking manually with pandas.read_parquet(), I confirmed the data was there. The problem wasn’t in the files — it was in how the decoder handled them.

That’s when I realized: the datasets API couldn’t decode raw audio bytes correctly. The data was fine, but the pipeline was broken.

Turning Bytes into Sound

At this point, I tried everything.

I extracted the bytes manually, saved them as .raw files, and tried to load them with librosa and soundfile. Nothing worked.

Without proper metadata (sample rate, channels, encoding), the files were unreadable.

Eventually, I discovered the solution: FFmpeg.

By using known dataset parameters —

sample rate: 16000 Hz, channels: mono, format: PCM 16-bit little-endian —

I could convert all .raw files into clean .wav audio.

ffmpeg -f s16le -ar 16000 -ac 1 \
       -i sevil_0000.raw \
       sevil_0000.wav

And just like that — 1,566 files successfully converted.

No corruption. No decoding errors. 100% validation success.

It was the moment of breakthrough — the kind that makes you sit back, smile, and realize you’ve just solved a problem that haunted you for two days straight.

Training the Model — Again

With clean audio finally ready, I retrained the Sevil model from scratch, this time using all 1,566 recordings.

Training setup (based on my previous configs for “Arslan”):

num_train_epochs = 500
batch_size = 4
learning_rate = 1e-4
fp16 = True
warmup_steps = 2000
save_steps = 2000

The progress was promising:

Loss dropped from 1.14 → 0.80 → 0.50 → 0.27
The voice quality improved with every epoch
And then… it crashed at 78% completion.

The culprit? A familiar one for SpeechT5 users —

RuntimeError: torch.cat(): expected a non-empty list of Tensors

It turned out to be a bug in guided_attention_loss, a component that sometimes fails with uneven sequence lengths.

Fixing the Crash

Instead of starting over, I resumed training from the last checkpoint (step 16,000) and simply disabled guided attention:

model.config.use_guided_attention_loss = False

That one line saved the project.

Training resumed, completed 98% of the full cycle, and stabilized with a final loss of 0.267 — a small numerical improvement, but a big qualitative one.

The model became more consistent and robust across new data.

Comparing Results

Model	Data Used	Loss	Success	Notes
Sevil v1	415 files (26.5%)	0.276	47% test	Trained on partial data
Sevil v2	1,566 files (100%)	0.267	100% test	Fully trained, stable

The difference wasn’t just in metrics — it was in confidence.

Sevil v2 generalized better, produced smoother intonation, and maintained pronunciation consistency even on unseen words.

Lessons Learned

Never trust good metrics without checking data coverage.

My “good” baseline was trained on just 26% of the data.
FFmpeg is a lifesaver.

It solved what specialized libraries couldn’t.
Validate every single file.

Automation saved hours of manual checking.
Guided attention loss is optional.

Sometimes stability matters more than theoretical accuracy.
Document everything.

By keeping track of every attempt — successful or not — I could understand the full story, not just the happy ending.

Why This Project Matters

For me, this wasn’t just about fixing one dataset. It was about enabling a low-resource language — Crimean Tatar — to have a better voice in the digital world.

Improving the Sevil model means clearer pronunciation, smoother prosody, and better accessibility for learners and native speakers alike.

And for anyone working with custom TTS datasets:

Check your files, validate your data, and don’t give up when your model crashes at 78%.

That crash might be the best teacher you’ll ever have.

Author: Servin Osmanov

Lead Fullstack Python / ReactJS Engineer

AI researcher and TTS developer for low-resource languages

Project: servinosmanov/tts-crh-sevil-fixed on Hugging Face

My Journey as a Judge at CBIT Hacktoberfest 2025 — Lessons from the Other Side of the Table

Servin Osmanov — Thu, 30 Oct 2025 18:29:37 +0000

When I first started attending hackathons, I was always the one coding, building, and pitching. This year, for the first time, I got to experience the other side — as a judge at the CBIT Hacktoberfest Hackathon 2025.

It was an incredible 24-hour online event organized by the CBIT Open Source Community, celebrating open-source culture and collaboration as part of the global Hacktoberfest. The event gathered hundreds of students from universities across the world — developers, designers, and dreamers ready to learn, code, and share.

What the Event Was About

Hacktoberfest is all about celebrating open source, and this hackathon perfectly captured that spirit. The CBIT edition — now in its 8th year — encouraged participants to collaborate on innovative solutions using modern technologies while contributing to the open-source ecosystem.

Teams of 3–5 members worked virtually through Discord, tackling real-world challenges in just 24 hours. Despite the distance and time zones, the sense of connection and creativity was palpable.

The Judging Framework

Every project was evaluated based on a clear, balanced set of criteria (total: 50 points):

Criterion	Points	What It Measured
Innovation & Creativity	10	Original ideas and new approaches
Collaboration	5	Teamwork, communication, and shared contribution
Implementation	20	Technical soundness, scalability, and efficiency
Design	5	Usability and user experience
Presentation	10	Clarity, storytelling, and delivery

As a judge, I was looking for that spark — projects that combined solid implementation with a clear purpose.

What I Saw and Learned

Some projects were technically ambitious, pushing the boundaries of what’s possible in 24 hours. Others were beautifully simple, focusing on accessibility or social impact. A few used AI and automation in creative ways that genuinely surprised me.

But beyond the technology, what stood out was the teamwork. Many participants were strangers before the event — yet they managed to code, design, and present together like long-time collaborators. That’s the magic of hackathons.

The Judging Experience

Our panel included engineers and leaders from Electronic Arts, JP Morgan Chase, Oracle, Deliveroo, ServiceNow, EY, and more. Being part of such a diverse group gave every discussion depth. Each judge viewed “innovation” slightly differently — and that variety made our evaluations richer.

I appreciated the effort teams put into their presentations. Even short demos told full stories — from the problem statement to the final prototype. That’s where creativity met clarity.

Key Takeaways

Innovation isn’t always about technology. Sometimes it’s about empathy — understanding who you’re helping and why.
Good presentation matters. The best projects told their stories with confidence and focus.
Community is everything. Open-source hackathons like this show how technology connects us beyond borders.

As a developer, I’ve always believed in continuous learning. Judging this event reaffirmed that growth happens in many forms — whether you’re writing code or evaluating it.

Why Hackathons Matter More Than Ever

In a world where remote collaboration is the new normal, hackathons remain one of the most human ways to innovate. They combine competition, creativity, and community into one shared experience. For students and professionals alike, they’re the perfect place to experiment and grow.

I’m grateful to have played a small role in this journey — to witness ideas take shape, to learn from participants, and to see firsthand how open source continues to inspire a new generation of builders.

Author:
Servin Osmanov

Lead Software Engineer @ Anvaya Solutions Inc.

Judge at CBIT Hacktoberfest Hackathon 2025

Why Language Tech Matters: Developing AI Tools for Small Languages

Servin Osmanov — Sat, 25 Oct 2025 15:45:27 +0000

In a world where artificial intelligence is transforming how we communicate, the survival of small languages depends not just on cultural passion — but on technology.

While English, Chinese, or Spanish dominate the digital space, thousands of smaller languages remain digitally invisible. Without online presence, data, or digital tools, these languages risk extinction in the 21st century.

As a software engineer and cultural advocate, I’ve spent years developing digital tools for the Crimean Tatar language — a Turkic minority language spoken by less than 300,000 people worldwide. Through this work, I’ve learned that the intersection of AI and linguistics isn’t just a research topic; it’s a lifeline for cultural identity.

The Challenge: The Technology Gap for Small Languages

Mainstream AI models—from GPT-based chatbots to translation engines—are trained primarily on high-resource languages. This creates a serious imbalance: while global communication becomes easier for some, others are left behind.

For small linguistic communities, the lack of:

digital corpora,
standardized spelling systems,
and high-quality training datasets

makes it nearly impossible to integrate their languages into modern tools like voice assistants, translation services, or educational apps.

When a language isn’t “machine-readable,” it risks becoming irrelevant in the digital world — even if it’s still spoken at home.

Our Journey: Building Tools for the Crimean Tatar Language

In 2018, our team launched a series of educational and linguistic tools under the Qırımtatar Lugatı (Crimean Tatar Dictionary) project:

Our mission was to give people an accessible and modern way to learn, use, and preserve the Crimean Tatar language.

The apps quickly grew into more than just dictionaries — they became living platforms connecting speakers, learners, and educators. Yet, users wanted more: context-based search, on-the-fly translation, and natural voice pronunciation.

This inspired us to integrate artificial intelligence to make our tools more flexible and intelligent.

The Next Step: Bringing AI to Minority Languages

We are now integrating AI and machine learning into the Crimean Tatar dictionary project to expand its capabilities:

🧠 AI-assisted translation: dynamic, real-time translation between Crimean Tatar, Turkish, English, and Ukrainian.
🔊 Speech recognition and synthesis: allowing users to hear natural pronunciation and practice correct intonation.
📖 Adaptive learning: using AI to personalize vocabulary lessons based on user behavior and progress.
🪶 Data-driven NLP foundation: building scalable open datasets that will support future chatbots, voice assistants, and translation systems.

To achieve this, we’ve adopted and fine-tuned open-source speech synthesis models such as:

facebook/mms-tts-crh — part of Meta’s Massively Multilingual Speech project, providing a solid baseline for Crimean Tatar TTS.
robinhad/qirimtatar-tts — a community-driven model that helps us generate high-quality speech for educational and cultural content.

By leveraging these models, we’re building an AI-powered TTS system that brings Crimean Tatar audio resources to learners, teachers, and content creators for the first time.

Why This Matters

Language is more than communication — it’s collective memory.

When a language disappears, so does a unique worldview and cultural identity.

AI gives us a chance to reverse that process. With the right tools, small languages can become visible online, connect their communities, and survive in the digital era.

Projects like Qırımtatar Lugatı show that you don’t need a giant corporation to make an impact.

A small, passionate team with the right technical vision can give a digital voice — literally — to those who’ve been silent for too long.

How Developers and Researchers Can Help

If you’re a developer, linguist, or AI researcher, here are a few ways to make a difference:

Contribute to open-source datasets for small or low-resource languages.
Support multilingual NLP and TTS initiatives on Hugging Face or GitHub.
Collaborate with communities and educators who are digitizing endangered languages.
Help localize educational and cultural apps into underrepresented languages.

Every dataset, every model, and every contribution brings us closer to a more linguistically inclusive AI ecosystem.

Looking Ahead

Our roadmap for 2025 includes:

Full integration of AI-powered translation within the Qırımtatar Lugatı apps.
Implementation of voice features using our fine-tuned TTS models.
Launching an open API for developers who want to build tools for Crimean Tatar or similar minority languages.

Technology alone won’t preserve culture — but it can amplify it.

For small languages like Crimean Tatar, AI isn’t just a tool — it’s hope.

About the Author

Servin Osmanov is a Senior Full-Stack Engineer and founder of the Qırım.Online and Ana-Yurt.Com project — an initiative focused on preserving Crimean Tatar culture and language through modern technology.

LinkedIn: linkedin.com/in/servin-osmanov

GitHub: github.com/MrSerWin