Whisper transcribes spoken Hindi numbers as words, not digits. So I made a library to convert them back.

A community health worker reads a blood-pressure measurement out loud. Whisper transcribes it as एक सौ दस बटा सत्तर — "one hundred ten over seventy," in words. The form I was filling needed 110/70.

That gap is the whole reason this library exists.

I've written about llmclean before — a tiny library that grew out of the same habit of copy-pasting the same cleanup pass across projects. This is its sibling, one layer down the stack. Where llmclean cleans up what a language model emits, hindi-normalize cleans up what a speech recogniser hears — specifically for Hindi.

It came out of Sakhi, an offline Hindi voice-to-form tool for ASHA workers (India's frontline community health workers). At the bottom of that pipeline, right before Whisper's transcript became "data we trust," I kept finding the same code: number words that never became digits, ५ and 5 treated as different characters, an ASCII | sitting where a danda । belonged, the same word encoded two different ways so search and dedup silently missed it.

Every Hindi speech project I touched had its own ad-hoc version. So I pulled the pass out, generalized it, and put it on PyPI.

What v0.1.0 does

from hindi_normalize import normalize_transcript

normalize_transcript("आपका BP एक सौ दस बटा सत्तर है, वजन अट्ठावन kg")
# → 'आपका BP 110/70 है, वजन 58 kg'

That one call runs the whole pipeline — collapse ASR stutter, canonicalize the Devanagari, map domain terms, convert number words to digits, tidy punctuation. Every transform is also exported on its own:

from hindi_normalize import convert_numbers, normalize_devanagari

convert_numbers("एक सौ दस बटा सत्तर")          # → '110 बटा 70'
normalize_devanagari("वजन ५८ किलो | ठीक है")   # → 'वजन 58 किलो। ठीक है'

The number words are the headline

Speech-to-text hands you spoken numbers as words, and it spells them the way they sounded, not the way a dictionary would. So the mapping has to be tolerant. पाँच, पांच, पाच all have to reach 5; सत्तर and सतर both to 70. v0.1.0 ships around 160 such variants, and composes them up to 999:

from hindi_normalize import parse_hindi_number

parse_hindi_number("नौ सौ निन्यानवे")   # → 999

The one design decision I'd defend hardest: adjacent numbers are never summed.

convert_numbers("दो तीन दिन")   # → '2 3 दिन', never '5'

In spoken Hindi, दो तीन दिन means "two-three days" — an approximation, the way English says "a couple days." A naive word-number parser adds adjacent tokens and turns that into five. In a medical transcript, silently inventing quantities is the worst thing this library could do, so it keeps a range a range.

Three Devanagari things that make it more than a regex

1. NFC quietly decomposes the nukta letters

The obvious first move for "make the same text compare equal" is Unicode NFC. And for most scripts, NFC composes — it pulls a base character and a combining mark into a single precomposed codepoint.

For the Devanagari nukta consonants, it does the opposite.

import unicodedata
[hex(ord(c)) for c in "क़"]                            # ['0x958']  — one codepoint
[hex(ord(c)) for c in unicodedata.normalize("NFC", "क़")]  # ['0x915', '0x93c']  — क + ़

The precomposed forms क़ ख़ ग़ … (U+0958–U+095F) sit in Unicode's composition-exclusion table, so NFC leaves them decomposed as base + U+093C. This surprises people who expect NFC to always compose. I lean into it instead of fighting it: canonicalize to the decomposed form, which is exactly what the Indic NLP Library's normalizer does too. Whichever way the character was typed, you get the same bytes out.

2. \b silently fails on most Hindi words

For collapsing ASR stutter (है है है है → है), I first reached for the textbook repeated-word regex, \b\S+\b. It missed almost everything.

The reason is subtle. Most Hindi words end in a matra — a combining vowel sign like the ी in हिंदी or the ै in है. A combining matra is not a \w character, and \b is defined as a boundary between \w and non-\w. So after a word that ends in a matra, there is no word boundary for \b to find, and the match quietly fails on exactly the words Hindi uses most.

The fix is to drop \b entirely and anchor on whitespace instead:

(\S+)(?:\s+\1){n,}

I hit this as a failing unit test while extracting the library. Which is the good outcome — it's the kind of bug that otherwise ships and looks like a flaky model.

3. The one-liner that deletes the language

There is a tempting way to "clean up" Unicode text: strip the combining marks — filter out everything in the Unicode Mark (M) category. Run that over Devanagari and you delete every matra and every virama:

import unicodedata
"".join(c for c in "हिंदी" if unicodedata.category(c)[0] != "M")
# → 'हद'

The output still looks like Devanagari. It's just no longer the word — or any word. This is a real bug in naive normalizers, and it has a nasty second-order effect in ML: it silently deflates Whisper's measured word error rate, because you've thrown away the exact characters the model is being scored on. hindi-normalize canonicalizes marks; it never removes the category. The lossy transforms it does offer — nukta removal, nasal folding, chandrabindu and visarga folding — are all opt-in and off by default. You don't lose information unless you ask to.

The shape of the thing

hindi-normalize lives in a small gap between bigger tools:

• For full Indic NLP — tokenization, transliteration, script conversion across a dozen languages — use the Indic NLP Library. This follows its DevanagariNormalizer for the character-level operations.
• For English number-word parsing, use text2num or word2number.

This is the pass in between: spoken Hindi numbers to digits, plus the Devanagari canonicalization an ASR or OCR transcript actually needs before you try to parse it. It composes with the bigger tools; it doesn't compete with them.

The principles it shares with its sibling llmclean:

Every function is total. Hand it a non-string and it returns the input untouched; the body is wrapped so it never raises. That means it drops into a pipeline that can't afford an exception path.

Lossless by default. The safe transforms run automatically; anything that discards information is a keyword flag you opt into.

Zero runtime dependencies. Pure standard library. Nothing for a downstream user to resolve version conflicts against.

Try it, tell me where it breaks

pip install hindi-normalize

What would genuinely help:

If your ASR produces a number-word spelling I don't map, open an issue with the raw transcript. That's how the variant list grows.

If you work in a domain other than maternal and child health, the default term dictionary is ASHA-shaped and won't fit — but it's swappable, so I'd like to know what your vocabulary looks like.

And if you've solved this with the Indic NLP Library or your own pass and think I should have just used that — that's a useful argument to have in the open too.

GitHub: Tushar-9802/hindi-normalize
PyPI: hindi-normalize on PyPI

Next on the roadmap, driven the same way — found in real transcripts first, ported second: Hindi ordinals (पहला, दूसरा), fraction words (आधा, सवा, डेढ़, पौन — everywhere in doses and quantities), and currency, date, and time normalization.