DEV Community

Vaibhav Srivastava
Vaibhav Srivastava

Posted on

Why Converting Hindi PDFs to Word Produces Gibberish (Kruti Dev ASCII vs. UTF-8 Unicode)

If you have ever tried to convert a Hindi government circular, district court petition, or scanned educational textbook from PDF into Microsoft Word, you have almost certainly experienced one of two disasters:

  1. The converted Word file replaces all Hindi paragraphs with seemingly random English characters like Hkkjr ljdkj or d + ;k.
  2. Or worse: half the vowel marks (matras) break, the top headline bar fragments, and words turn into empty □□□ tofu boxes.

Why do nearly all standard PDF converters fail catastrophically on Devanagari and Hindi PDFs—and what is actually happening beneath the hood of legacy Indian font encoding?


1. The Legacy ASCII Remap Trap: Kruti Dev, DevLys & Chanakya

Most people assume that if you can read Hindi text inside a PDF viewer, the file internally stores Hindi characters. In thousands of legacy Indian documents, that is false.

Before Unicode standardized Devanagari in UTF-8 (U+0900–U+097F), Indian DTP centers, typing institutes, and government clerks used ASCII-remapped TrueType fonts such as Kruti Dev 010, DevLys, Chanakya, and Shusha.

+---------------------------------------------------------------+
| VISUAL RENDERING ON SCREEN (With Kruti Dev Font Installed)    |
| "भारत सरकार"                                                   |
+---------------------------------------------------------------+
                               │
                 (Underlying Byte Stream in PDF)
                               ▼
+---------------------------------------------------------------+
| RAW ASCII CHARACTER CODES (Stored in PDF Content Stream)      |
| "Hkkjr ljdkj"                                                 |
+---------------------------------------------------------------+
Enter fullscreen mode Exit fullscreen mode

Because Kruti Dev maps visual Devanagari glyph shapes directly onto standard English keyboard codes (H draws भ, k draws ा, j draws र), any naive PDF-to-Word converter simply reads the raw byte codes (Hkkjr) and outputs English gibberish unless you manually reinstall Kruti Dev and re-apply font overrides—or pass the text through a legacy-to-Unicode remapping table.


2. Why Scanned Hindi OCR Breaks Standard Engines

Even when a Hindi document is a flat image scan (bypassing legacy ASCII font encodings), standard OCR models stumble over three typographic complexities unique to the Devanagari script:

A. The Continuous Shirorekha (Top Headline Bar)

In Latin scripts (hello), white space naturally isolates every individual character vertically and horizontally. In Devanagari (नमस्ते), letters are joined together across an entire word by the horizontal top line (Shirorekha). Classical connected-component segmenters frequently mistake the whole word as a single character glyph or drop characters when thin horizontal ink breaks.

B. Visual vs. Logical Vowel (Matra) Ordering

In the word हिंदी (Hindi), the Chhoti i matra (ि) is painted visually to the left of the consonant ह (Ha). However, according to the UTF-8 Unicode standard, vowel signs must be stored logically after the base consonant:

  • Visual placement left-to-right: [ ि ] + [ ह ]
  • Valid UTF-8 Unicode byte order: [ ह (U+0939) ] + [ ि (U+093F) ]

Poorly implemented converters read glyphs strictly left-to-right and emit invalid byte sequences (िह), scrambling spelling and breaking Word spellcheck/search.

C. Conjunct Consonants (Samyuktakshar) and Halant Ligatures

Half-consonants joined with virama/halant (्)—such as क्ष, त्र, ज्ञ, द्व—change shape drastically based on context. Low-resolution 150 DPI scans erode tiny diacritics like anusvara (ं) and nuqta (़) into background scanner dust.


3. Practical Checklist for Clean Hindi PDF-to-Word Conversion

Before converting an important Hindi legal deed, marksheet, or government circular:

  1. Inspect PDF Font Metadata First (Ctrl + D in Reader): Check the Fonts tab. If you see names like KrutiDev010, DevLys, Chanakya, or Shusha, simple copy-paste will yield ASCII English strings unless processed with a dedicated Kruti Dev to Unicode converter.
  2. Scan Physical Papers at 300 DPI Grayscale: Avoid 1-bit pure Black-and-White mode for Hindi scans. Pure thresholding deletes tiny diacritical dots (bindi/anusvara) that drastically alter meaning (e.g., है vs हैं). 300 DPI 8-bit grayscale preserves clean glyph boundaries.
  3. Verify Searchability & Font Normalization: After conversion to .docx, select a paragraph and check if the Word font defaults to a standard OpenType Unicode font like Mangal, Nirmala UI, or Noto Sans Devanagari. Try hitting Ctrl + F and typing a Hindi word from a smartphone or Google Input Tools—if it highlights, the output is genuine Unicode.

Summary

Converting Hindi and bilingual documents reliably requires bridging both legacy ASCII font encoding and Unicode script geometry while protecting seals, tables, and margins.

At PDF Retype, we built specialized free workflows to solve both challenges without paywalls or watermarks:

Disclosure: I am the founder of PDF Retype (https://pdfretype.com), an independent, zero-registration document processing suite built to eliminate layout breakage and regional OCR failures.

Top comments (0)