<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Riamu</title>
    <description>The latest articles on DEV Community by Riamu (@epitrochoid).</description>
    <link>https://dev.to/epitrochoid</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3683370%2Fa5e82352-0cc5-4393-b5a0-3577cc73b802.jpg</url>
      <title>DEV Community: Riamu</title>
      <link>https://dev.to/epitrochoid</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/epitrochoid"/>
    <language>en</language>
    <item>
      <title>Adding Furigana to Word Documents is a Nightmare</title>
      <dc:creator>Riamu</dc:creator>
      <pubDate>Fri, 28 Aug 2026 01:47:36 +0000</pubDate>
      <link>https://dev.to/epitrochoid/adding-furigana-to-word-documents-is-a-nightmare-4m22</link>
      <guid>https://dev.to/epitrochoid/adding-furigana-to-word-documents-is-a-nightmare-4m22</guid>
      <description>&lt;p&gt;Adding furigana to a Word document sounded like a small extension to EZFurigana.&lt;/p&gt;

&lt;p&gt;Furigana are the small pronunciation guides that appear above Japanese kanji. I read Japanese myself, and I still run into kanji that I cannot read immediately. Adding furigana manually in Microsoft Word is fine for a few words. On a longer document, it gets tedious very quickly.&lt;/p&gt;

&lt;p&gt;So I wanted EZFurigana to accept a Word document and return it with furigana automatically added.&lt;/p&gt;

&lt;p&gt;My initial mental model was:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Open the DOCX.&lt;/li&gt;
&lt;li&gt;Find the Japanese text.&lt;/li&gt;
&lt;li&gt;Generate the readings.&lt;/li&gt;
&lt;li&gt;Save the file.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;That sounded fairly straightforward.&lt;/p&gt;

&lt;p&gt;Then I looked inside a DOCX file.&lt;/p&gt;

&lt;p&gt;It soon turned out to be quite the nightmare.&lt;/p&gt;

&lt;h2&gt;
  
  
  A DOCX is not a text file
&lt;/h2&gt;

&lt;p&gt;A DOCX file is not what it seems.  It's not one document, but rather a ZIP package containing XML files, styles, images, fonts, scattered into many pieces which are glued together. &lt;/p&gt;

&lt;p&gt;For a typical document, most of the main body content is stored in:&lt;/p&gt;

&lt;p&gt;document.xml&lt;/p&gt;

&lt;p&gt;One option would have been to extract all the text, add furigana, and build a new Word document.&lt;/p&gt;

&lt;p&gt;The people who invented the DOCX file, however, didn't want people's lives to be easy.&lt;/p&gt;

&lt;p&gt;A Word document can contain tables, hyperlinks, images, and structures that are difficult to understand. Rebuilding the document would mean recreating all of them correctly.&lt;/p&gt;

&lt;p&gt;Instead, EZFurigana modifies the existing Word document and leaves everything else alone. It changes only &lt;code&gt;document.xml&lt;/code&gt;, the part containing the document's main content, while preserving the rest of the file as it is.&lt;/p&gt;

&lt;p&gt;But life keeps getting worse.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;document.xml&lt;/code&gt; does not contain text in the convenient form you might expect.&lt;/p&gt;

&lt;p&gt;Suppose Word displays:&lt;/p&gt;

&lt;p&gt;今日は東京駅に行きます。&lt;/p&gt;

&lt;p&gt;That sentence may be split across several runs, which are separate elements containing pieces of text.&lt;/p&gt;

&lt;p&gt;A formatting change can create a new run. So can hyperlinks, tables, images, and anything else Word lets you add.  Word has been secretly slicing your sentences the whole time.&lt;/p&gt;

&lt;p&gt;A single visible word can therefore span several elements.&lt;/p&gt;

&lt;p&gt;That makes this kind of operation unsafe:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;find "東京駅"&lt;/li&gt;
&lt;li&gt;replace it with annotated "東京駅"&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The characters may actually be chopped up into different parts of the file.&lt;/p&gt;

&lt;p&gt;So EZFurigana keeps track of where each piece of text came from before changing anything. If the text crosses parts of the document that are too risky to rewrite, it simply leaves it alone.&lt;/p&gt;

&lt;p&gt;This isn't perfect. But I would rather miss some furigana than fail to glue all the pieces back together.&lt;/p&gt;

&lt;h2&gt;
  
  
  Adding Word's actual furigana markup
&lt;/h2&gt;

&lt;p&gt;Microsoft Word has native support for ruby, which is the general term for annotations such as furigana.&lt;/p&gt;

&lt;p&gt;Internally, Word represents it with structures such as &lt;code&gt;w:ruby&lt;/code&gt;, &lt;code&gt;w:rubyBase&lt;/code&gt;, and &lt;code&gt;w:rt&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Conceptually:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight xml"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;w:ruby&amp;gt;&lt;/span&gt;
  &lt;span class="nt"&gt;&amp;lt;w:rt&amp;gt;&lt;/span&gt;
    &lt;span class="c"&gt;&amp;lt;!-- とうきょう --&amp;gt;&lt;/span&gt;
  &lt;span class="nt"&gt;&amp;lt;/w:rt&amp;gt;&lt;/span&gt;
  &lt;span class="nt"&gt;&amp;lt;w:rubyBase&amp;gt;&lt;/span&gt;
    &lt;span class="c"&gt;&amp;lt;!-- 東京 --&amp;gt;&lt;/span&gt;
  &lt;span class="nt"&gt;&amp;lt;/w:rubyBase&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;/w:ruby&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;w:rubyBase&lt;/code&gt; contains the original text and &lt;code&gt;w:rt&lt;/code&gt; contains the reading.&lt;/p&gt;

&lt;p&gt;If a run contains:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;A東京B
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;and only &lt;code&gt;東京&lt;/code&gt; needs furigana, EZFurigana effectively has to split it into:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;A
東京 + ruby
B
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;while keeping the original formatting.&lt;/p&gt;

&lt;p&gt;It also detects existing ruby instead of blindly adding another annotation.&lt;/p&gt;

&lt;p&gt;Eventually I had a DOCX that Microsoft Word could open and display correctly.&lt;/p&gt;

&lt;p&gt;Then I needed to preview it in the browser.&lt;/p&gt;

&lt;h2&gt;
  
  
  Rendering DOCX: The Nightmare Continues
&lt;/h2&gt;

&lt;p&gt;EZFurigana lets users review generated readings and correct them, so I needed some way to display the modified document before download.&lt;/p&gt;

&lt;p&gt;Creating a valid DOCX does not solve that.&lt;/p&gt;

&lt;p&gt;Browsers do not understand Word documents natively. Something has to reconstruct the margins, fonts, spacing, and all the other pieces, then somehow apply the secret glue that makes it look like Word again.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;modified DOCX&lt;/li&gt;
&lt;li&gt;LibreOffice&lt;/li&gt;
&lt;li&gt;PDF&lt;/li&gt;
&lt;li&gt;preview pages&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;This gets much closer to the original document layout.&lt;/p&gt;

&lt;p&gt;But LibreOffice is not Microsoft Word, and this is where furigana caused problems again.&lt;/p&gt;

&lt;p&gt;LibreOffice gets surprisingly close, but its handling of East Asian typography, especially ruby, is not quite where I want it to be. In particular, it adds extra spacing below ruby, so some documents look different from the same file opened in Microsoft Word.&lt;/p&gt;

&lt;p&gt;I don't really blame the LibreOffice maintainers. They already have the job of gluing the pieces of a DOCX nightmare back together, and ruby is only one tiny part of that mess. I am happy to let them keep whatever sanity they have left.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Ended Up Shipping
&lt;/h2&gt;

&lt;p&gt;The final flow looks roughly like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;upload DOCX&lt;/li&gt;
&lt;li&gt;map the Word document &lt;/li&gt;
&lt;li&gt;generate furigana &lt;/li&gt;
&lt;li&gt;rewrite the safe parts of the XML (keeping the glue)&lt;/li&gt;
&lt;li&gt;render a preview with LibreOffice &lt;/li&gt;
&lt;li&gt;download the modified DOCX&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;I am fine with that tradeoff if it means I can wake up from this nightmare. Plus, missing a few annotations is much better than destroying somebody's document.&lt;/p&gt;

&lt;p&gt;I started this feature thinking DOCX support would be a straightforward addition.&lt;/p&gt;

&lt;p&gt;Instead, I ended up dealing with a file split into dozens of pieces, text that refuses to stay in one place, formatting that must survive surgery, and a second office suite just to render the result.&lt;/p&gt;

&lt;p&gt;All because I didn't want to add furigana manually.&lt;/p&gt;

&lt;p&gt;Was it worth it? You can try the DOCX feature on EZFurigana and let me know. If you have a strange Word document that breaks something, I would like to hear about it. Those are usually the best way to find what I missed.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://www.ezfurigana.com/" rel="noopener noreferrer"&gt;Here's the furigana converter&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://www.ezfurigana.com/furigana-docx" rel="noopener noreferrer"&gt;Here's the converter for Word files&lt;/a&gt;&lt;/p&gt;

</description>
      <category>showdev</category>
      <category>softwaredevelopment</category>
      <category>japanese</category>
      <category>webdev</category>
    </item>
    <item>
      <title>How computers stumble over Japanese readings</title>
      <dc:creator>Riamu</dc:creator>
      <pubDate>Mon, 29 Dec 2025 02:29:50 +0000</pubDate>
      <link>https://dev.to/epitrochoid/how-computers-stumble-over-japanese-readings-2p72</link>
      <guid>https://dev.to/epitrochoid/how-computers-stumble-over-japanese-readings-2p72</guid>
      <description>&lt;p&gt;Last Saturday I was tutoring a man in English when he asked a question I hadn't heard before.&lt;/p&gt;

&lt;p&gt;"Are the words flea and flea market related?"&lt;/p&gt;

&lt;p&gt;I had to stop and think. The short answer is no - they share spelling, but their meanings and origins are different. Native speakers don't usually work that out in real time. We just… know.&lt;/p&gt;

&lt;p&gt;I wanted to explain that "just knowing" feeling, and I reached for a Japanese example, partly because I'm learning Japanese myself and this is the kind of thing I keep tripping over.&lt;/p&gt;

&lt;p&gt;Take 大分. It's two characters: 大 ("big") and 分 ("part"). But together it can be read in more than one way:&lt;br&gt;
Ōita (おおいた), the prefecture&lt;br&gt;
daibu (だいぶ), meaning "quite a bit"&lt;/p&gt;

&lt;p&gt;A Japanese reader doesn't pause and decode the kanji from scratch. They lean on context without thinking about it. Geography? Ōita. Degree or extent? daibu.&lt;br&gt;
I'm not sure my student cared about Ōita, but the example stuck with me because it's the same kind of problem I run into constantly as a learner.&lt;/p&gt;

&lt;p&gt;Some words have two plausible readings. Some have more. Even a single character can split into a handful of options. 明, for example, can be read as mei, myō, min, or akira, depending on where it appears and what the sentence is doing. For a native reader the correct choice often feels obvious. For a learner, or for a computer, it might not be.&lt;br&gt;
And even if you pick the "right" base reading, Japanese pronunciation doesn't always stay put when words combine.&lt;/p&gt;

&lt;p&gt;There are a few common sound changes that native speakers apply automatically:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Rendaku (連濁) (voicing): 花 (hana, "flower") + 火 (hi, "fire") → 花火 (hanabi, "fireworks")&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Renjō (連声) (sound linking): 観 (kan, "to observe") + 音 (on, "sound") → 観音 (kannon, a Buddhist term)&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Sokuonbin (促音便) (a small っ / doubled consonant): 学校 (gakkō, "school"), often explained as gakukō → gakkō&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A dictionary can tell you 火 can be read hi (ひ), and it can list 花火 as hanabi (はなび). But the moment you meet a compound that isn't explicitly in a dictionary, you're back to something fuzzier: patterns, probability, context.&lt;/p&gt;

&lt;h2&gt;
  
  
  Readings drift over&amp;nbsp;time
&lt;/h2&gt;

&lt;p&gt;The other part of this is that readings aren't fixed.&lt;br&gt;
Language moves. Pronunciations shift. Classical texts pull in older forms that don't match modern habits. And internet writing keeps inventing new conventions faster than any printed reference can follow.&lt;br&gt;
At some point I started thinking of furigana as a kind of agreement. Not random, but not "encoded" either. We write a reading because people, collectively, have decided that's how it's read, at least for now.&lt;/p&gt;

&lt;p&gt;You can see that agreement forming in slang:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;草 (kusa): "lol," because www (laughing) looks like grass&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;尊い (toutoi): "too cute / too precious"&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;萎える (naeru): "to lose steam / get deflated" (motivation drops)&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These weren't in dictionaries when they started spreading. They became "real" the same way most language does: usage first, and consensus later.&lt;br&gt;
Why I cared enough to build something&lt;br&gt;
I kept hitting the same wall when I tried to read actual books.&lt;/p&gt;

&lt;p&gt;I'd be moving through a page, hit a word I didn't recognize, and have to stop. Copy it. Paste it into a dictionary. Scroll through multiple readings. Try to figure out which one made sense in that sentence. Then do it again three lines later.&lt;/p&gt;

&lt;p&gt;One paragraph could easily turn into ten minutes of lookups. I wasn't reading anymore. I was bouncing between the text and a search box.&lt;/p&gt;

&lt;p&gt;I didn't want a possible reading. I wanted the reading that made sense here, right now.&lt;/p&gt;

&lt;p&gt;Until computers understand language the way people do, furigana is still one of the best bridges between written Japanese and comprehension. I wanted a bridge that held up a little better in messy, real-world text, basically a furigana converter that could survive the stuff I actually read.&lt;/p&gt;

&lt;h2&gt;
  
  
  Dictionaries help, until they&amp;nbsp;don't
&lt;/h2&gt;

&lt;p&gt;Most furigana systems start with huge dictionaries. They collect common words and compounds and store their expected readings. With enough coverage, you can get pretty far.&lt;/p&gt;

&lt;p&gt;But dictionaries don't solve the core ambiguity problem. When 明 appears on its own, is it the Ming dynasty? A given name like Akira? Something else? A person figures it out by glancing at what's around it. A program has to learn how to do that explicitly.&lt;br&gt;
Rules didn't scale for&amp;nbsp;me&lt;/p&gt;

&lt;p&gt;My first instinct was to write rules: look for historical terms or dates and bias toward min; look for person-ish clues and bias toward akira. It works in a few handpicked cases, and then it starts to feel like trying to handwrite language. It's slow to extend, easy to break, and it fails as soon as the text gets novel.&lt;br&gt;
So I moved to machine learning.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why I didn't reach for a neural&amp;nbsp;network
&lt;/h2&gt;

&lt;p&gt;I considered neural networks because that's what most modern language systems lean on. They're powerful and flexible, and they can pick up subtle grammatical signals.&lt;/p&gt;

&lt;p&gt;But I wasn't trying to build a general-purpose language model. I was trying to build a furigana converter that's fast and light enough to be practical. For that, a simpler model made more sense.&lt;br&gt;
I ended up using logistic regression. It's not glamorous, but it's quick, easy to reason about, and surprisingly strong when the features are chosen carefully.&lt;/p&gt;

&lt;h2&gt;
  
  
  Data, and the ways it goes&amp;nbsp;wrong
&lt;/h2&gt;

&lt;p&gt;Once you decide to learn from context, you inherit a data problem.&lt;/p&gt;

&lt;p&gt;The text you train on needs range: novels, news, legal writing, casual posts, dialogue. Too broad and the model learns mush; too narrow and it breaks outside its comfort zone. And real-world text is messy (emoji, numbers, odd punctuation, inconsistent formatting), so you often end up building a cleaning pipeline before you can even start training.&lt;br&gt;
The part I didn't appreciate at first is how specific the failure modes can be.&lt;/p&gt;

&lt;p&gt;A good example is 紅葉. It can be read as もみじ (momiji) or こうよう (kōyō). The meanings overlap: momiji is usually "maple / red leaves," and kōyō is the broader "autumn foliage" idea. Close enough that you don't always get a clean context signal, which means your training data has to be careful.&lt;/p&gt;

&lt;p&gt;While I was building the dataset, I found out a bunch of it was basically contaminated by a novel that had a character written as 紅葉（もみじ）. Once those examples got mixed in, a lot of otherwise useful sentences stopped being usable. The model started learning weird person-ish context around 紅葉, and then it would reach for name-like cues even in ordinary sentences about trees.&lt;/p&gt;

&lt;p&gt;Nothing was "broken," exactly. The outputs just slowly started getting stranger until I noticed the pattern. After that I spent an unglamorous amount of time filtering, re-checking, and rebuilding the examples so the model wasn't training on accidental fiction trivia.&lt;/p&gt;

&lt;p&gt;Even after cleaning, there's a practical question that never fully goes away: when do you trust a dictionary lookup, and when do you fall back to the statistical model?&lt;/p&gt;

&lt;h2&gt;
  
  
  A patchwork, like most&amp;nbsp;things
&lt;/h2&gt;

&lt;p&gt;To deal with names, I added another component that tries to tag spans of text that look like personal names. Once something is tagged as a name, I hand it off to different logic for resolving the reading.&lt;br&gt;
At that point the whole project started to look like what most software looks like in the end: not one elegant idea, but a bunch of smaller pieces that cover each other's weak spots.&lt;/p&gt;

&lt;p&gt;In practice, what I do is pretty simple.&lt;br&gt;
I try the dictionary approach first. If that doesn't settle it, I look at nearby particles, prefixes, and suffixes that tend to give the reading away. If it smells like a name, I route it through the name path. And if it's one of the characters I kept getting wrong over and over, that's when I let the model make the call.&lt;/p&gt;

&lt;h2&gt;
  
  
  Training and&amp;nbsp;checking
&lt;/h2&gt;

&lt;p&gt;I iterated over a lot of Japanese text, mostly Aozora Bunko (an online repository of public-domain books) and Wikipedia. I'd run the algorithm, use an LLM to flag outputs that looked suspicious, then manually verify and correct those cases. Cross-referencing open-source dictionaries helped me build a base dictionary set with massive coverage.&lt;/p&gt;

&lt;p&gt;I did track accuracy, but mostly as a sanity check for myself, not as a formal benchmark. Over time it climbed into the mid-90s on the test set I kept around. Adding the name handling pushed it into the high-90s. It still makes mistakes, and I still find edge cases, but it crossed the line where I could read with it without babysitting it.&lt;/p&gt;

&lt;p&gt;This started as a small personal furigana converter and slowly turned into something I kept reaching for, so I put it up here: &lt;a href="https://www.ezfurigana.com/" rel="noopener noreferrer"&gt;www.ezfurigana.com&lt;/a&gt;.&lt;br&gt;
The moment it started to feel&amp;nbsp;real&lt;/p&gt;

&lt;p&gt;I fed it a couple sentences with 辛い, a character that can mean either "spicy" (karai) or "painful / difficult" (tsurai) depending on context.&lt;br&gt;
Something like 辛いカレー (spicy curry) versus この仕事は辛い (this job is tough).&lt;/p&gt;

&lt;p&gt;Then I tried 大分 in a few different shapes too: prefecture, adverb, the usual ambiguity.&lt;br&gt;
It got them right.&lt;/p&gt;

&lt;p&gt;Not just once, but consistently. Different sentences, different contexts, and it kept landing on the correct reading. That's when I stopped thinking of it as a project and started thinking of it as a tool I could actually use.&lt;/p&gt;

&lt;p&gt;Then I ran it on a full ebook. It generated furigana for the entire thing and kept the formatting intact. I opened it on my reader and just… started reading.&lt;br&gt;
That was the moment I thought: okay. I can actually use this now.&lt;/p&gt;

</description>
      <category>nlp</category>
      <category>machinelearning</category>
      <category>japanese</category>
      <category>webdev</category>
    </item>
  </channel>
</rss>
