DEV Community

Doyoon Kim
Doyoon Kim

Posted on

The HWP-to-Markdown Problem Nobody Talks About (and a FreeType Gotcha)

Korean .hwp files are everywhere in government, legal, and enterprise workflows in Korea — and almost nowhere in the LLM tooling ecosystem. If you've ever tried to feed a .hwp file into a RAG pipeline or an LLM context window, you've probably hit the same wall I did: there's no pdfplumber-equivalent for HWP, and most "document to text" libraries just skip the format entirely.

I ended up building a HWP→Markdown path into a small internal tool (doc2md) and ran into two problems worth sharing, because they're the kind of thing that eats a whole afternoon if you don't know they're coming.

Problem 1: HWP isn't one format, it's a ZIP with opinions

Modern .hwp (and .hwpx) files are structured containers, closer to OOXML than to a flat binary blob. That's good news — it means a Python library can actually parse them without reverse-engineering a proprietary binary spec from scratch. I used rhwp-python (MIT-licensed), which handles the container parsing and gives you structured text/paragraph access instead of raw bytes.

The gotcha: HWP's paragraph and table model doesn't map 1:1 to Markdown. Tables in particular need their own conversion pass — naively dumping cell text in document order silently reorders table data if you don't track row/column position explicitly.

Problem 2: a font-rendering library crashed the process, for a reason that had nothing to do with fonts

The parsing step pulled in a FreeType dependency for font metrics, and it segfaulted intermittently under load — but only on the server, never locally. Classic "works on my machine."

The actual cause: FreeType's shared library was being loaded twice by two different code paths in the same process (once via the HWP library, once via an unrelated image library), and the two loads disagreed on symbol versions. The fix wasn't LD_PRELOAD (my first guess) — it was preloading the FreeType library explicitly via ctypes before either import path could load its own copy, forcing both to share one instance. A few lines, but the debugging-to-fix ratio was brutal.

Why this was worth doing

Once HWP is normalized into Markdown, it goes through the exact same pipeline as PDF/PPTX/DOCX — same chunking, same LLM ingestion, same output shape. That consistency is the actual payoff: you stop writing format-specific glue code every time a new document type shows up, and "can I RAG this file" stops depending on whether someone exported it as HWP or DOCX.

We packaged this (HWP + ~20 other formats → Markdown) as a free tool if you want to try it on your own files without setting up the pipeline yourself: https://www.knowverse.net/en/util/ — no signup needed for quick conversions.

Curious if anyone else here has fought with HWP, or with FreeType's shared-library loading quirks in a Python service — always interested in comparing notes on the boring-but-necessary parts of document pipelines.

Top comments (0)