DEV Community

Cover image for Why Android's Speech Recognizer Struggles with Tagalog and What I Did About It
Mark Anthony Catabona
Mark Anthony Catabona

Posted on

Why Android's Speech Recognizer Struggles with Tagalog and What I Did About It

#flutter #speechrecognition #android #nlp

ViVeL — Vida Verdadera De Dios, "True Life of God." I wanted a name shaped like Israel: not just a label, but a word that says something about who the people are. ViVeL is an offline Bible app for Android with voice search over Ang Dating Biblia (Tagalog, 1905) and the KJV.

In English, voice search works fine. In Tagalog, it fights back — and the ways it fails turned out to be interesting enough to write up.

The problem

Here's a real one from the app. I said nagkakalakip na gumagawa. The recognizer produced:

magkakalapit na gumagawa

The recognizer heard a completely different word, not a typo, not a misspelling, a different word entirely. And here's why that matters more for search than for dictation: with dictation, one wrong word in a sentence is survivable. With verse search, one wrong word means zero results. It's all-or-nothing. The verse either surfaces or it doesn't.

I'm using the speech_to_text Flutter plugin, which wraps the platform's on-device recognizer, with the locale set to fil-PH. Two things go wrong with Tagalog specifically:

  1. Tagalog is low-resource. There's far less Tagalog in the recognizer's training data than English, so its confidence on Tagalog words is shakier and it biases toward more common word sequences.
  2. Tagalog loves reduplication and long affixed words,naglalakip, magkakalapit, nangagsisiibig. These long, morphologically dense words are exactly the ones a shaky recognizer mangles, collapsing one into a nearby-sounding other.

What I could fix, spelling, not sound

There's a second mismatch that I could actually solve: the modern speech recognizer outputs modern spelling, but my bundled text is the 1905 ADB, which spells things the old way. Diyos vs Dios. sapagkat vs Sapagka't. kanya vs kaniya. ngunit vs Nguni't.

Even if the recognizer hears the word perfectly, the FTS5 search misses because the spellings don't match. So I fold both forms into one canonical form before searching:

test('normalizeForSearch folds archaic and modern spellings together', () {
  const n = DatabaseHelper.normalizeForSearch;
  expect(n('Diyos'), n('Dios'));
  expect(n('sapagkat'), n("Sapagka't"));
  expect(n('Hesukristo'), n('Jesucristo'));
  expect(n('kanya'), n('kaniya'));
  expect(n('ngunit'), n("Nguni't"));
  expect(n('gayon'), 'gayon');          // unaffected words stay stable
  expect(n('nangagsisiibig'), 'nangagsisyibig'); // folds only whole words
});
Enter fullscreen mode Exit fullscreen mode

The tricky part is folding at the right granularity. nang should fold to ng as a whole word but not when it's buried inside nangagsisiibig, or you corrupt unrelated words. So the normalizer is word-boundary aware, not a blind find-and-replace.

I also lean on the recognizer's alternate hypotheses. The plugin returns more than just the top transcription, so when the best guess misses, I retry the search against the runner-up transcriptions:

[
  for (final a in result.alternates.skip(1))
    if (a.recognizedWords.trim().isNotEmpty) a.recognizedWords,
]
Enter fullscreen mode Exit fullscreen mode

Sometimes the correct word is the recognizer's second or third guess, and this quietly rescues the search.

What I haven't solved, the sound-level miss

Spelling normalization and alternates handle a lot. What they don't handle is the case at the top: naglalakip heard as magkakalapit. That's not a spelling variant and it's not in the alternates list, it's a genuine acoustic substitution of one whole word for another.

The approaches I'm weighing:

  • Phonetic normalization - map both the transcript and the index to a sound-based key (a Tagalog-tuned Soundex/Metaphone) so acoustically-near words collide on purpose.
  • Fuzzy matching over FTS5 - Levenshtein or trigram distance against the verse index, so a near-miss still retrieves candidates instead of nothing.
  • Tagalog-aware morphology - strip reduplication and affixes before matching, so naglalakip and its mangled cousins reduce toward a common root.

Each option has a downside. Phonetic keys match too many unrelated words. Fuzzy search across the whole Bible gets slow on cheap phones. Stripping affixes properly needs real Tagalog grammar rules, not shortcuts.

Have you hit this?

If you've built voice or search for Tagalog or any low-resource language with heavy affixation I'd like to know how you handled the acoustic misses. Phonetic keying? Fuzzy retrieval? Something smarter? The spelling and alternates layers were the easy 80%. The last 20%, where the recognizer swaps in a whole wrong word, is the part I'm still chewing on.

I'm a software engineer from the Philippines building offline-first Android apps, with a focus on Tagalog-language tooling that most speech and search stacks weren't designed for.

Top comments (0)