Parsing and Rebuilding EPUB Files in Python: Lessons Learned from Building an AI Book Translator

How we used ebooklib and lxml to extract, translate, and reconstruct EPUBs while preserving formatting and structure.

We started building LectuLibre, an AI-powered book translation service, with a simple premise: users upload an EPUB or PDF, we translate it using large language models (Claude, DeepSeek), and they download a translated copy. Sounds straightforward—but the devil is in the details, especially when you promise to preserve the original formatting, fonts, images, and structure. Here’s what we learned about parsing and rebuilding EPUB files in Python, and how we solved the trickiest parts.

The Problem: Translating Books While Keeping Everything Else Intact

EPUB is essentially a ZIP archive containing XHTML files, CSS, images, and metadata. To translate a book, we needed to:

1. Extract the text from each XHTML file in reading order.
2. Send the text to an LLM for translation (in chunks, but that's another story).
3. Replace the original text with the translated text, preserving the DOM structure, inline tags, and styling.
4. Rebuild the EPUB with updated metadata, navigation, and all assets intact.

Naively, you might think you can just unzip, regex-replace the text, and zip it back. But that breaks as soon as you encounter nested tags, split sentences across tags, or special characters. We needed a proper parser.

Evaluating Libraries: Why We Chose ebooklib + lxml

We looked at several options:

• ebooklib: High-level library for reading/writing EPUBs. Handles the ZIP container, manifest, spine, and TOC. It gives you EpubBook and EpubItem objects but doesn't parse the HTML content—you get raw bytes/strings.
• epubfile: Another library, but less maintained and less flexible.
• Calibre's ebook-convert: Powerful but designed as a command-line tool, not for programmatic control inside a web service.
• Direct zipfile + lxml: Full control but we'd have to implement the EPUB spec ourselves (manifest, spine, etc.).

We settled on ebooklib for the container-level work and lxml for parsing and modifying the XHTML content. This combination gave us the best balance of convenience and control.

Reading and Iterating Over EPUB Content

Here's a minimal example of loading an EPUB and iterating over its spine items (documents in reading order):

from ebooklib import epub
from lxml import etree

book = epub.read_epub('input.epub')

for item in book.get_items_of_type(epub.ITEM_DOCUMENT):
    # item.content is bytes
    content = item.content.decode('utf-8')
    tree = etree.fromstring(content.encode('utf-8'))
    # Now work with tree...

But there's a catch: namespaces. XHTML files declare a default namespace (xmlns="http://www.w3.org/1999/xhtml"), and lxml requires you to handle that. If you just do tree.xpath('//p'), you'll get nothing. You need to map the namespace.

ns = {'xhtml': 'http://www.w3.org/1999/xhtml'}
paragraphs = tree.xpath('//xhtml:p', namespaces=ns)

This was our first big "gotcha". We wrote a helper to strip namespaces or always use the namespace map.

Extracting Text in Paragraph Order

For translation, we needed to extract text chunks that made sense. Instead of grabbing all text nodes indiscriminately (which would flatten everything and lose structure), we walked the DOM and collected text per block-level element (p, h1, h2, etc.) while preserving inline tags.

Here's a simplified version of our extraction function:

def extract_paragraphs(tree):
    ns = {'x': 'http://www.w3.org/1999/xhtml'}
    blocks = []
    for elem in tree.xpath('//x:p | //x:h1 | //x:h2 | //x:h3 | //x:h4 | //x:h5 | //x:h6', namespaces=ns):
        text = elem.text_content().strip()
        if text:
            blocks.append(text)
    return blocks

But this flattened inline tags—losing <em> or <span> information. For translation, we actually wanted to preserve those tags because they carry meaning (emphasis, quotes, etc.). So we devised a more granular approach: we serialized the inner content of each block element to a compact markup string, sent that to the LLM with instructions to translate text but keep tags, and then parsed the result back.

That was complex and error-prone, so for v1 we settled on a simpler method: we translate the plain text of each paragraph, then insert the translated text into the original block element, stripping all inline tags. This meant losing italics/bold in translated text, which was acceptable for MVP but we knew we'd need to improve later. It's a trade-off we documented clearly.

Replacing Text Without Breaking the DOM

To replace text while preserving the block-level structure, we used lxml to set the text content of each block element. For a paragraph <p>Hello <em>world</em>!</p>, if we want to replace with "Bonjour le monde !", we could do:

new_p = etree.SubElement(parent, '{http://www.w3.org/1999/xhtml}p')
new_p.text = translated_text
# Replace old_p with new_p
old_p.getparent().replace(old_p, new_p)

But that removes any attributes or classes on the original paragraph. Better approach: keep the element, but remove all children and set .text:

for child in old_p:
    old_p.remove(child)
old_p.text = translated_text

This preserves the tag, attributes, and tail text (though tail text needed careful handling; we often used etree.strip_elements or manual tail merging).

Batch Translation and Reconstruction

In our pipeline, we don't translate paragraph by paragraph; that would be too slow and lose context. Instead, we extract all paragraphs from the entire book (in spine order), then send them to the LLM in batches (e.g., 50 paragraphs per request). The LLM returns translated paragraphs in the same order, and we map them back.

Here's how we apply the translations back to the document tree:

def replace_paragraphs(tree, translated_texts):
    ns = {'x': 'http://www.w3.org/1999/xhtml'}
    blocks = tree.xpath('//x:p | //x:h1 | //x:h2 | //x:h3 | //x:h4 | //x:h5 | //x:h6', namespaces=ns)
    for elem, new_text in zip(blocks, translated_texts):
        # Remove all children (inline tags) but keep the element itself
        for child in elem:
            elem.remove(child)
        elem.text = new_text
    return tree

We keep a global list of all block elements across all XHTML files, in order, and after translation, we iterate over the files again and update each element with the corresponding translated text. This ensures the reading order is preserved.

Handling Images and CSS

Images and CSS are separate EPUB items. We didn't need to modify them, just make sure they are preserved when rebuilding. ebooklib does this automatically: when you read an EPUB, all items are stored in the book object, and when you write, they are included. However, we had to ensure that the items we modified (the XHTML documents) maintained their correct names and paths.

One issue: when we modified an item's content, we had to set it back:

item.content = etree.tostring(tree, encoding='utf-8', xml_declaration=True)

We also had to be careful about the xml:lang attribute and the DOCTYPE declaration. ebooklib sometimes added or removed them inconsistently. We ended up post-processing the serialized string to fix it.

Rebuilding the Navigation (TOC)

EPUB3 uses a nav.xhtml file for the table of contents. ebooklib has a book.toc property, but it's often out of sync or not updated when you change document structure. Since we weren't changing the structure (just text inside paragraphs), we didn't need to modify the TOC. But for some books, we had to add language metadata or update the title.

We used ebooklib's metadata API:

book.set_title('Translated Title')
book.set_language('fr')
book.add_metadata('DC', 'description', 'Translated by LectuLibre')

Performance Numbers

For a typical 300-page novel (~100,000 words), our parsing and rebuilding pipeline (excluding LLM translation time) takes about 2-3 seconds on a standard VPS. The bottleneck is actually the epub.read_epub and epub.write_epub operations, which involve ZIP compression/decompression. The lxml DOM operations are fast (milliseconds per file). Since translation via LLM takes minutes, the overhead is negligible.

Challenges and Workarounds

1. Namespaces: As mentioned, always use namespace maps.
2. Self-closing tags: lxml sometimes converts <p></p> to <p/>, which can break some EPUB readers. We had to force full closing tags by using etree.tostring(..., method='html') or using the HTML parser. Actually, for XHTML we should use XML serialization, but we found that many EPUBs are not strictly valid XML, so we used the HTML parser for resilience.
3. Encoding: Most EPUBs use UTF-8, but occasionally we saw Windows-1252 or other encodings. ebooklib doesn't handle this well; we added detection using chardet.
4. Fixed-layout EPUBs: Some EPUBs are fixed-layout (images per page). Our text-based approach doesn't work for those. We detect them and fall back to a different pipeline (OCR or PDF-like processing). This is still an open area.
5. Broken EPUBs: Many EPUBs in the wild are not well-formed. We had to implement fallbacks: try strict XML parsing, then fall back to lxml.html parser, which is more forgiving.

Lessons Learned

• Use ebooklib for container management, lxml for content manipulation. Don't try to reinvent the ZIP/OPF layer.
• Always test with a diverse set of EPUBs. We collected a corpus of 50+ books from different sources and used them for regression testing.
• Preserve the original DOM as much as possible. Even small changes to the structure (like removing empty spans) can break the layout.
• Plan for inline tags early. Stripping them was a temporary solution; we're now working on a tag-preserving translation using a custom markup language and XML-aware replacement.

Open Question for the Community

How do you handle translation of text with inline formatting (bold, italics, links) without losing the semantics? We've experimented with sending serialized HTML to the LLM and asking it to return translated HTML, but reliability is hit-or-miss. If you've solved this elegantly, we'd love to hear about it.