DEV Community: 龚旭东

Handling Token Limits When Translating 300-Page Books with Claude

龚旭东 — Sat, 26 Sep 2026 03:01:26 +0000

How we chunked long-form content, preserved context, and managed output limits to build a reliable book translation pipeline.

When we started building LectuLibre, our AI-powered book translation service, we knew the core challenge wouldn't be the translation quality itself—it would be feeding an entire 300-page book to a large language model without hitting token limits.

Claude's context window is huge (200k tokens for Claude 3 Opus), but that's still not enough for many full-length books. And even if it were, the output token limit—the maximum number of tokens the model can generate in a single response—is far smaller (4,096 tokens for Claude 3 Sonnet and Opus). If you ask Claude to translate an entire book, it will happily start and then abruptly stop mid-sentence once it hits that output ceiling.

Here's how we solved it with a chunking strategy that preserves context, respects output limits, and still delivers a coherent final translation.

The Problem: Long Text vs. Hard Limits

A typical 300-page novel contains about 80,000–90,000 words. In token terms, that's roughly 100,000–130,000 tokens. While that fits inside Claude's input context, the model's output limit means it can only produce a tiny fraction of that in one go. Even if you tried a streaming approach, the max output tokens remains a hard stop.

Our first naive attempt was to send a whole chapter (maybe 20–30 pages) at once. That failed for two reasons:

Input token overflow on some chapters that were longer than expected (especially with PDF extraction producing messy whitespace).
Output truncation — the model would translate beautifully for a few paragraphs, then stop because it reached the 4,096-token output limit.

We needed a way to split the book into pieces small enough that the model could translate each piece completely, but large enough to maintain context and coherence.

Our Approach: Paragraph-Aware Chunking with Context Overlap

The core idea is simple: split the book into chunks based on paragraph boundaries and token counts, translate each chunk separately, and stitch the results back together. But the devil is in the details.

We chose paragraph-level chunking rather than splitting by fixed token size or sentence count because it respects the natural rhythm of prose. Splitting mid-paragraph often leads to awkward translations or duplicated context.

We also added a context overlap — each chunk includes the last few sentences of the previous chunk as untranslated context. This helps the model maintain narrative flow and avoid repeating introductions or forgetting character names.

Finally, we used a running glossary of key terms (character names, places, technical jargon) that gets passed to the model with each chunk. This ensures consistency across chunks—something that pure overlap alone doesn't guarantee.

Implementation Details

1. Token Counting with the Anthropic SDK

We use the official anthropic Python SDK. It provides a convenient count_tokens method that gives us an accurate token count for a given text.

import anthropic

client = anthropic.Anthropic(api_key="YOUR_API_KEY")

def count_tokens(text: str) -> int:
    """Return token count for a text using Claude's tokenizer."""
    response = client.count_tokens(text)
    return response.tokens

This is more accurate than using OpenAI's tiktoken approximation, and it only costs a tiny fraction of a penny per call. For chunking decisions, precision matters because we want to get as close to the input limit as possible without exceeding it.

2. Chunking Function

The chunker takes a list of paragraphs (extracted from EPUB or PDF) and returns a list of chunks. Each chunk is a tuple: (context, text_to_translate). The context is the last N sentences from the previous chunk (if any), and it is not translated—it's only there to give the model context.

from typing import List, Tuple

MAX_INPUT_TOKENS = 3000   # conservative limit to stay under output limit
                         # and leave room for system prompt + context
CONTEXT_SENTENCES = 3     # number of previous sentences to include as context

def chunk_paragraphs(
    paragraphs: List[str],
    max_tokens: int = MAX_INPUT_TOKENS,
    context_sentences: int = CONTEXT_SENTENCES
) -> List[Tuple[str, str]]:
    chunks = []
    current_text = ""
    current_tokens = 0
    previous_context = ""

    for para in paragraphs:
        para_tokens = count_tokens(para)

        # If a single paragraph is too large, split it further (rare)
        if para_tokens > max_tokens:
            # Fallback: split by sentences
            sentences = re.split(r'(?<=[.!?]) +', para)
            for sent in sentences:
                sent_tokens = count_tokens(sent)
                if current_tokens + sent_tokens > max_tokens:
                    chunks.append((previous_context, current_text.strip()))
                    # Update context for next chunk
                    previous_context = " ".join(
                        current_text.split()[-context_sentences*10:]  # rough sentence proxy
                    )
                    current_text = sent
                    current_tokens = sent_tokens
                else:
                    current_text += " " + sent
                    current_tokens += sent_tokens
            continue

        if current_tokens + para_tokens > max_tokens:
            # Finalize current chunk
            chunks.append((previous_context, current_text.strip()))
            # Update context for next chunk
            previous_context = " ".join(current_text.split()[-context_sentences*10:])
            current_text = para
            current_tokens = para_tokens
        else:
            current_text += " " + para
            current_tokens += para_tokens

    if current_text:
        chunks.append((previous_context, current_text.strip()))

    return chunks

This code is simplified—in production we also handle special cases like chapter boundaries (prefer to start a new chunk at a chapter heading) and avoid splitting code blocks or poetry.

3. Translation Loop with Streaming and Retries

For each chunk, we call the Claude API with a system prompt that includes translation instructions and our running glossary. We use streaming to show progress to the user in real time, and we implement exponential backoff for rate limit errors.

import asyncio
from anthropic import AsyncAnthropic

client = AsyncAnthropic(api_key="YOUR_API_KEY")

SYSTEM_PROMPT_TEMPLATE = """
You are a professional book translator. Translate the following text from {source_lang} to {target_lang}.

Rules:
- Translate only the text after the marker "---TEXT---".
- The text before "---TEXT---" is context from the previous chunk. Do NOT translate it.
- Maintain the original formatting (paragraphs, line breaks).
- Use the glossary below for consistent terminology.

Glossary:
{glossary}
"""

async def translate_chunk(
    context: str,
    text: str,
    glossary: dict,
    source_lang: str,
    target_lang: str,
    model: str = "claude-3-sonnet-20240229"
) -> str:
    prompt = SYSTEM_PROMPT_TEMPLATE.format(
        source_lang=source_lang,
        target_lang=target_lang,
        glossary="\n".join(f"{k}: {v}" for k, v in glossary.items())
    )
    messages = [
        {"role": "user", "content": f"{context}\n---TEXT---\n{text}"}
    ]

    translated = ""
    max_retries = 5
    for attempt in range(max_retries):
        try:
            async with client.messages.stream(
                model=model,
                max_tokens=4096,  # max output tokens
                system=prompt,
                messages=messages
            ) as stream:
                async for event in stream:
                    if event.type == "content_block_delta":
                        translated += event.delta.text
            break
        except anthropic.RateLimitError:
            await asyncio.sleep(2 ** attempt)
        except anthropic.APIError as e:
            print(f"API error: {e}, retrying...")
            await asyncio.sleep(2 ** attempt)
    else:
        raise Exception("Max retries exceeded")

    return translated.strip()

In production, we also update the glossary after each chunk by asking the model for a list of new terms it encountered, but that's beyond the scope of this article.

4. Stitching It All Together

The main loop walks through the chunks, translates each one, and collects the outputs. We store the final translation in a database (PostgreSQL) and also allow the user to download as EPUB or PDF.

async def translate_book(paragraphs: List[str], source_lang: str, target_lang: str):
    chunks = chunk_paragraphs(paragraphs)
    glossary = {}
    translated_parts = []

    for i, (context, text) in enumerate(chunks):
        print(f"Translating chunk {i+1}/{len(chunks)}...")
        translated = await translate_chunk(
            context, text, glossary, source_lang, target_lang
        )
        translated_parts.append(translated)

        # Optional: update glossary from translated text
        # (we use a separate call to extract new terms)

    return "\n\n".join(translated_parts)

Results and Lessons Learned

After implementing this pipeline, we successfully translated several 300-page books from English to Spanish and French. Here are some concrete numbers for a typical novel:

Book size: ~85,000 words, ~110,000 tokens extracted
Number of chunks: 38 (average chunk size ~2,800 tokens)
Total processing time: ~8 minutes with sequential streaming (using Claude 3 Sonnet)
Cost: approximately $3.20 per book (Sonnet pricing)
Translation quality: significantly better than our earlier attempts without overlap and glossary

What went wrong initially:

No context overlap → the model would translate each chunk as if it were a standalone text, leading to inconsistent character names and repeated exposition. Adding just 3–5 sentences of previous context fixed most of this.
Ignoring output token limit → we sized chunks to fit the input context (200k) but output would truncate. The realization that output tokens are usually the binding constraint was a turning point.
Unbounded retries → we initially retried on every error without backoff, causing us to hit rate limits hard. Exponential backoff solved it.

Trade-offs:

Smaller chunks → safer output, but more API calls (cost and latency) and potential loss of long-range context.
Larger chunks → better context, but higher risk of hitting output limits or truncation, especially for languages that expand (e.g., German often produces 20–30% more tokens).
We settled on a dynamic chunk size based on the source language's expansion ratio—for languages like Spanish (which expands slightly) we keep input tokens around 2,500; for Japanese (which can shrink) we go up to 3,500.

Final Thoughts and an Open Question

The key takeaway for anyone building long-form text processing with LLMs is: token limits are not just about input size—output limits often matter more, and chunking must be context-aware. A naive split by token count works for summarization but fails for translation, where narrative continuity is crucial.

We're still exploring better ways to maintain consistency across chunks. One idea is to use Claude's prompt caching (introduced in beta) to cache the system prompt and glossary, reducing latency and cost for subsequent chunks. Another is to use a vector database to retrieve relevant context from previous chapters instead of a simple fixed overlap.

Open question for the community: How do you handle languages with extreme expansion ratios? We've experimented with asking the model to summarize the chunk before translating, but that adds latency. Any better ideas?

LectuLibre is an AI-powered book translation service that lets users upload EPUB or PDF files and receive a fully translated book. The backend is Python/FastAPI with PostgreSQL, and it runs on a simple VPS. This article reflects our real-world experience building the translation pipeline.

How We Translate Entire Books with LLMs Without Losing Context

龚旭东 — Wed, 23 Sep 2026 03:01:15 +0000

Chunking strategies, context carry-over, and glossary injection for long-form translation

The Problem: Translating Books is Not Like Translating Tweets

At LectuLibre, we translate entire books using LLMs like Claude and DeepSeek. Early on, we discovered that feeding a whole book into the API wasn't just expensive—it was impossible. A 300-page novel can be 120,000 tokens, but even the best models max out at 200k context (and the quality degrades near the limit). So we had to chunk.

Our first attempt was naive: split text into fixed-size chunks of 4,000 characters. The result? Sentences cut off mid-thought, paragraphs split across chunks, and worst of all, the model lost track of the story. Characters' names changed, tone shifted, and the translation felt disjointed.

We needed a better way: chunk by structure, not just size, and carry context across chunks.

Our Chunking Strategy: Paragraph-Aware with Overlap

We decided to split on paragraph boundaries using Python's textwrap? Actually we needed more control. We used nltk for sentence tokenization and then grouped sentences into chunks that fit within a token budget.

Here's the core function we wrote:

import nltk
from typing import List

nltk.download('punkt')

def chunk_by_paragraphs(text: str, max_tokens: int = 6000, overlap_sentences: int = 3) -> List[str]:
    # Split into paragraphs, then sentences
    paragraphs = text.split('\n\n')
    sentences = []
    for para in paragraphs:
        sentences.extend(nltk.sent_tokenize(para))

    chunks = []
    current_chunk = []
    current_tokens = 0

    for sentence in sentences:
        # Estimate tokens (roughly 4 chars per token)
        sentence_tokens = len(sentence) // 4 + 1
        if current_tokens + sentence_tokens > max_tokens and current_chunk:
            chunks.append(' '.join(current_chunk))
            # Keep overlap sentences for context continuity
            overlap = current_chunk[-overlap_sentences:] if overlap_sentences > 0 else []
            current_chunk = overlap.copy()
            current_tokens = sum(len(s) // 4 + 1 for s in current_chunk)
        current_chunk.append(sentence)
        current_tokens += sentence_tokens

    if current_chunk:
        chunks.append(' '.join(current_chunk))

    return chunks

Key decisions:

Split on paragraphs first, then sentence tokenization using NLTK (offline, reliable).
Overlap of 3 sentences between chunks: this bridges context and prevents the model from losing the thread at boundaries.
We set max_tokens to 6000 as a safe limit for Claude 3.5 Sonnet, leaving room for the prompt and response.

But chunking alone wasn't enough. The model still forgot what happened in earlier chapters.

Carrying Context: Chapter Summaries and Sliding Window

We experimented with three approaches:

No context: Just translate each chunk independently. Result: inconsistent terminology, story drift, poor quality.
Previous chunk as context: Include the last chunk's translated text (or source) as a prefix. Helped at boundaries but didn't solve long-range dependencies.
Hierarchical summarization: Before translating a chapter, generate a summary of the previous chapter(s) and inject that as a "story so far" context.

We settled on a hybrid: for each chapter, we generate a summary of the previous chapter (or the last few if short) and include it in the system prompt. For within-chapter chunks, we use the overlapping sentences plus a short local context.

Here's the summary generation code:

import anthropic

client = anthropic.Anthropic(api_key="...")

def generate_chapter_summary(chapter_text: str, max_summary_tokens: int = 500) -> str:
    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=max_summary_tokens,
        system="You are a literary summarizer. Create a concise summary of the given chapter, focusing on plot points, character introductions, and key events. Preserve names and terminology exactly.",
        messages=[{"role": "user", "content": f"Summarize this chapter:\n\n{chapter_text}"}]
    )
    return response.content[0].text

Then when translating a chunk, we build a prompt like:

def translate_chunk(chunk: str, context_summary: str, glossary: dict) -> str:
    system_prompt = f"""You are a professional literary translator. Translate the following text from {source_lang} to {target_lang}.

    Context from previous chapters:
    {context_summary}

    Maintain consistency with the provided glossary:
    {glossary}

    Preserve formatting, tone, and style. Do not add explanations."""

    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=len(chunk) * 3,  # generous output tokens
        system=system_prompt,
        messages=[{"role": "user", "content": chunk}]
    )
    return response.content[0].text

Glossary Injection for Terminology Consistency

One of the biggest challenges in book translation is keeping character names, place names, and invented terms consistent. LLMs are prone to transliterating names differently or using synonyms. We solved this by building a dynamic glossary.

We first run a lightweight NER (Named Entity Recognition) pass using spaCy to extract potential entities from the source text. Then we ask the LLM to filter and translate those entities into the target language (or keep original if it's a name). We store the mapping and inject it into every translation prompt.

Here's our glossary extraction:

import spacy

nlp = spacy.load("en_core_web_sm")

def extract_entities(text: str, limit: int = 50) -> List[str]:
    doc = nlp(text)
    entities = set()
    for ent in doc.ents:
        if ent.label_ in ["PERSON", "GPE", "LOC", "ORG", "PRODUCT"]:
            entities.add(ent.text)
    return list(entities)[:limit]

Then we ask the LLM to create a glossary:

def build_glossary(entities: List[str], target_lang: str) -> dict:
    prompt = f"Given these entities, provide a translation into {target_lang}. If the entity should remain unchanged (e.g., a proper name), return it as is. Output as JSON.\n\nEntities: {entities}"
    # call LLM, parse JSON
    ...

We then pass this glossary to every translation call. This reduced name inconsistencies from about 15% to less than 2% in our tests.

Real Numbers: Cost and Quality

We translated a 120,000-token book (about 350 pages) using this pipeline. On Claude 3.5 Sonnet:

Total API calls: ~40 chunks + 10 summary calls + glossary calls ≈ 55 requests.
Cost: $6.50 (input tokens dominated by context carry-over).
Time: ~3 minutes with async concurrency of 5 requests.
Quality: Human reviewer rated consistency 8.5/10 vs 5/10 for naive chunking.

The biggest cost factor was the context summary injection, which added about 30% more input tokens. We could reduce that by using smaller summaries or only summarizing every few chapters.

Lessons Learned and Trade-offs

Chunk size matters: Too small loses context, too large risks truncation and slow responses. We found 4000-6000 tokens per chunk to be the sweet spot.
Overlap is essential: Even 2-3 sentences of overlap dramatically improved boundary quality.
Summaries are expensive but worth it: For long books, a hierarchical summary approach saved us from narrative drift. But we had to balance cost; for cheaper models like DeepSeek, we use shorter summaries.
Glossary injection is non-negotiable: Without it, character names would change every few chapters.
One failure: We tried to use the entire previous chapter as context, but that blew the context window and the model started ignoring the current chunk. So we always use concise summaries, not raw text.

Open Question for the Community

We're still wrestling with cross-chapter references: when a character refers to an event from 10 chapters ago, the summary might miss it. Has anyone tried using a retrieval-augmented approach for book translation? Maybe embedding chunks and retrieving relevant context on demand? We'd love to hear your ideas.

You can find more about what we're building at LectuLibre (lectulibre.com), where we put these techniques to work translating real books for readers.

Practical takeaway: Don't just chunk—think about what context each chunk needs to make sense. Use structural boundaries, overlap, and a compact memory of the story so far.

Translating Full Books with LLMs: Our Chunking Strategy for Long-Form Context

龚旭东 — Sat, 19 Sep 2026 03:01:25 +0000

How we built a pipeline that preserves context across 100k+ token books using Python, FastAPI, and Claude's context window.

The Problem: Books Don't Fit in a Prompt

When we started building LectuLibre, our AI-powered book translation service, we assumed we could just send an entire book to a large language model and get a translation back. After all, Claude 3.5 Sonnet advertises a 200k token context window. But a typical novel is 80,000–120,000 words, which is roughly 100,000–150,000 tokens. That's technically within limits, but...

Cost: translating a 100k token book in one shot is expensive.
Quality: LLMs lose attention to early chapters when processing long contexts.
Rate limits and timeouts.
Hard to resume if fails.

So we needed a chunking strategy that preserves cross-chapter context: terminology, character voice, consistent style.

Our Approach: Sliding Window Chunks + Context Summary

We split the book into overlapping chunks, translate each with a context buffer containing:

A running glossary of terms and character names
A summary of previous chapters
The current chunk's raw text

The pipeline:

Parse EPUB/PDF into plain text with chapter metadata.
Split text into token-aware chunks with overlap.
For each chunk, fetch glossary + previous summary from PostgreSQL.
Call LLM to translate with system prompt.
Extract new terms from translation, update glossary.
Generate a concise summary of the translated chunk, store for next chunk.
Assemble final translated book.

Code: Token-Aware Chunking

We tried LangChain's RecursiveCharacterTextSplitter first, but it works on characters, not tokens, so some chunks exceeded the model's token limit. We switched to tiktoken for exact token counting.

import tiktoken
from typing import List, Tuple

def chunk_text_by_tokens(
    text: str,
    target_tokens: int = 3000,
    overlap_tokens: int = 500,
    model: str = "claude-3-5-sonnet-20240620"
) -> List[str]:
    # Use cl100k_base for Claude? Claude uses its own tokenizer but tiktoken's
    # cl100k_base approximates well enough for chunk sizing.
    enc = tiktoken.get_encoding("cl100k_base")
    tokens = enc.encode(text)

    chunks = []
    start = 0
    while start < len(tokens):
        end = min(start + target_tokens, len(tokens))
        chunk_tokens = tokens[start:end]
        chunks.append(enc.decode(chunk_tokens))

        if end == len(tokens):
            break
        # Move start back by overlap, but not before current start
        start = max(start + target_tokens - overlap_tokens, start + 1)

    return chunks

But books have paragraphs/chapters; we don't want chunks to split mid-sentence. We later added a paragraph-aware post-processing that adjusts boundaries to the nearest paragraph break within the token window. That improved translation quality.

Translation Pipeline with Context

We use Anthropic's Python SDK for Claude, and DeepSeek as a cheaper alternative for simpler passages. Here's the core translation function:

import anthropic
from tenacity import retry, stop_after_attempt, wait_exponential

client = anthropic.Anthropic(api_key="...")

SYSTEM_PROMPT_TEMPLATE = """You are a professional literary translator.
Translate the following text from {source_lang} to {target_lang}.
Preserve the author's style, tone, and voice.

Context from previous chapters:
{chapter_summary}

Glossary of established terms and names (use exactly these translations):
{glossary}

Translate only the provided text. Do not add explanations.
"""

@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=2, max=10))
def translate_chunk(
    chunk_text: str,
    source_lang: str,
    target_lang: str,
    glossary: dict,
    chapter_summary: str,
    model: str = "claude-3-5-sonnet-20240620"
) -> str:
    glossary_str = "\n".join([f"{k} -> {v}" for k, v in glossary.items()])
    prompt = SYSTEM_PROMPT_TEMPLATE.format(
        source_lang=source_lang,
        target_lang=target_lang,
        chapter_summary=chapter_summary,
        glossary=glossary_str
    )

    response = client.messages.create(
        model=model,
        max_tokens=4096,
        temperature=0.3,
        system=prompt,
        messages=[{"role": "user", "content": chunk_text}]
    )
    return response.content[0].text

We use tenacity for retries due to occasional API timeouts. The glossary is a simple Python dict stored as JSONB in PostgreSQL via SQLAlchemy. We update it after each chunk:

def update_glossary(session, glossary, source_chunk, translated_chunk):
    # Simplified extraction: ask LLM to list new terms
    extraction_prompt = f"Extract proper nouns, technical terms, and repeated phrases from this translated text. Return a JSON list of objects with 'source' and 'translation'."
    # ... call Claude or DeepSeek ...
    new_terms = extract_terms(translated_chunk)
    for term in new_terms:
        glossary[term['source']] = term['translation']
    session.execute(
        text("UPDATE books SET glossary = :glossary WHERE id = :book_id"),
        {"glossary": json.dumps(glossary), "book_id": book_id}
    )

We maintain a chapter summary by asking the LLM to summarize the translated chunk in 2-3 sentences, then concatenate with previous summary but keep it under 500 tokens.

Results and Trade-offs

For a 120,000-word novel (≈150k tokens), our pipeline creates about 60 chunks of 3000 tokens with 500 overlap. Translation using Claude 3.5 Sonnet costs around $12–15 per book, and takes about 20–30 minutes. Before chunking, a single-shot translation often produced inconsistencies (e.g., a character's name changed midway) and occasionally timed out. Chunking reduced errors significantly; we measured a 40% drop in user-reported translation errors.

But it wasn't perfect. Early on we set overlap to 100 tokens (about 3%), and translators noticed context breaks at chunk boundaries: a sentence referencing "the previous chapter's event" would be mistranslated because the LLM didn't have that context. We increased overlap to 500 tokens (16-17% of chunk size), which helped.

Another failure: we initially included the full glossary plus all previous summaries in every prompt, which bloated the context and sometimes confused the model. We now cap the glossary to the 50 most recent terms and keep the summary under 500 tokens.

Lessons Learned

Token-aware chunking is essential. Don't rely on character counts; use tiktoken or the model's tokenizer.
Overlap matters more than you think. Start with 15-20% overlap and test boundaries.
Maintain a running glossary, but prune it. Too much context can hurt as much as too little.
Use summarization to bridge long-range context. A short summary of prior chapters is often more useful than raw text.
Choose your model per chunk. We use Claude for literary quality and DeepSeek for more mechanical sections (table of contents, footnotes) to save cost.

What's Next?

We're exploring hierarchical translation: first translate chapter summaries, then use those as context for translating full chapters. That might improve coherence for very long books with complex plots.

Takeaway: Translating entire books with LLMs is feasible if you treat it as a context-management problem, not a single-prompt problem. Chunking with overlap, glossary, and summary bridges the gap between the model's context window and a book's length.

Open question for the community: How do you handle idiomatic expressions or cultural references that need adaptation rather than literal translation? Any techniques beyond a glossary?

Translating Entire Books with LLMs: Our Chunking Strategy for Context Preservation

龚旭东 — Wed, 16 Sep 2026 03:01:39 +0000

How we built a Python pipeline to translate long-form EPUBs and PDFs without losing narrative context across chunk boundaries.

At LectuLibre, we translate entire books using large language models. Users upload an EPUB or PDF, and our backend (Python/FastAPI) orchestrates calls to Claude and DeepSeek to produce a high-quality translation. The core challenge? Books are long—hundreds of thousands of tokens—while LLMs have limited context windows. If we naively translate paragraph by paragraph, we lose critical context: pronoun references, character names, terminology consistency, and narrative flow. Here’s how we solved it with a chunking strategy that preserves context without blowing up cost or latency.

The Problem: Long Documents vs. Context Windows

An average novel contains 80,000–120,000 words, which translates to roughly 100,000–150,000 tokens. Even the largest context windows (200k tokens for Claude 3, 128k for GPT-4 Turbo) can’t hold an entire book in one prompt, and even if they could, the cost and latency would be prohibitive. Translating sentence by sentence or paragraph by paragraph leads to:

Inconsistent terminology: A character’s name or a technical term might be translated differently each time.
Broken references: Pronouns like “he” or “she” lose their antecedent when the previous paragraph isn’t in context.
Style drift: The tone of translation can shift between chunks, making the book feel disjointed.

We needed a way to chunk the text into manageable pieces while carrying over enough context to maintain coherence.

Our Approach: Overlapping Chunks with Contextual Memory

We use a hierarchical chunking strategy that works at two levels:

Book structure: We respect natural boundaries like chapters (from EPUB) or detected headings (from PDF).
Within a chapter: We split paragraphs into chunks that fit within a safe token limit, but we include two types of contextual information:
- Overlap paragraphs: The last few paragraphs from the previous chunk are included as untranslated context (or their already-translated version).
- Running glossary: A dictionary of key terms and their translations, built incrementally as we translate.

We also send a system prompt that describes the book’s genre, target language, and translation guidelines.

Why This Works

Including the previous chunk’s translated text (or a summary) gives the LLM the immediate narrative context. The glossary ensures consistency for names, places, and domain-specific terms. By keeping chunks relatively small (2,000–4,000 tokens), we stay well within the model’s comfort zone and keep API calls fast.

Implementation Details

Here’s the core of our Python pipeline. We use tiktoken for fast token estimation, ebooklib for EPUB parsing, and pypdf for PDFs. API calls are made asynchronously with httpx and the Anthropic/DeepSeek SDKs.

Token Counting

We approximate token counts using tiktoken’s cl100k_base encoding. It’s not exact for Claude’s tokenizer, but it’s close enough for chunk sizing with a safety margin.

import tiktoken

enc = tiktoken.get_encoding("cl100k_base")

def count_tokens(text: str) -> int:
    return len(enc.encode(text))

Chunking Paragraphs

We split a chapter’s paragraphs into chunks that stay under max_tokens. To preserve continuity, we include the last overlap_paragraphs (usually 2) from the previous chunk as untranslated context.

def chunk_paragraphs(paragraphs: list[str], max_tokens: int = 3500, overlap_paragraphs: int = 2) -> list[list[str]]:
    chunks = []
    current_chunk = []
    current_tokens = 0

    for para in paragraphs:
        para_tokens = count_tokens(para)
        if current_tokens + para_tokens > max_tokens and current_chunk:
            chunks.append(current_chunk)
            # Start new chunk with overlap paragraphs from previous chunk
            overlap = current_chunk[-overlap_paragraphs:] if overlap_paragraphs else []
            current_chunk = overlap.copy()
            current_tokens = sum(count_tokens(p) for p in overlap)
        current_chunk.append(para)
        current_tokens += para_tokens

    if current_chunk:
        chunks.append(current_chunk)
    return chunks

Important: We do not translate the overlap paragraphs again. They are included only as context; the LLM is instructed to translate only the new paragraphs appended after the overlap. This prevents duplication.

Building the Glossary

During translation, we extract proper nouns and domain terms from the source text and track their translations. We use a simple regex-based extractor for capitalized words and a manually curated list for common terms.

import re
from collections import defaultdict

glossary = defaultdict(str)

def update_glossary(source_text: str, translated_text: str):
    # Extract potential proper nouns (crude but works for many books)
    candidates = re.findall(r'\b[A-Z][a-z]+\b', source_text)
    for term in candidates:
        if term not in glossary:
            # Ask LLM for translation of this term (or use a separate call)
            glossary[term] = translate_term(term)  # simplified

In practice, we don’t make a separate API call for every candidate; instead, we include the glossary in the prompt and ask the model to suggest translations for new terms as part of its output. We parse those suggestions from the response with a lightweight JSON format.

Translating a Chunk

Here’s the async function that translates one chunk. It takes the chunk (list of paragraphs), the previous chunk’s translated text as context, and the current glossary.

async def translate_chunk(
    chunk: list[str],
    context: str,          # previous chunk's translated text (empty for first)
    glossary: dict,
    model: str = "claude-3-haiku-20240307"
) -> str:
    # Build prompt
    prompt = f"""You are translating a book from English to Spanish.\n\
Maintain the original tone, style, and formatting.\n\
Use the following glossary for consistency (term: translation):\n\
{glossary}\n\
\n\
Context from the previous section (already translated):\n\
{context}\n\
\n\
Now translate the following paragraphs. Only translate the paragraphs after the context.\n\
Do not repeat the context.\n\
\n\
{''.join(chunk)}"""

    # API call (async)
    if model.startswith("claude"):
        response = await anthropic_client.messages.create(
            model=model,
            max_tokens=4096,
            messages=[{"role": "user", "content": prompt}]
        )
        return response.content[0].text
    else:
        # DeepSeek or other OpenAI-compatible
        response = await deepseek_client.chat.completions.create(
            model=model,
            max_tokens=4096,
            messages=[{"role": "user", "content": prompt}]
        )
        return response.choices[0].message.content

Orchestrating the Full Book

We process chapters sequentially but translate chunks within a chapter asynchronously to maximize throughput. For each chapter, we:

Extract paragraphs.
Chunk them (with overlap).
Initialize context = "" and glossary = {}.
For each chunk in order:
- Call translate_chunk(chunk, context, glossary).
- Update context with the translated output (truncated to last 500 tokens).
- Update glossary with any new terms from the output.
Concatenate translated chunks, removing the overlap duplicates.

We use asyncio.gather to translate multiple chunks concurrently, but we must be careful: context depends on the previous chunk’s output, so true parallelism isn’t possible within a chapter. However, we can pre-translate a chapter’s chunks without context in parallel to get a rough draft, then refine sequentially with context? We found that sequential translation with context gave better quality, so we accept the latency. For speed, we run multiple books concurrently on our VPS.

Performance Numbers

For a typical 100,000-token novel (about 80k words):

Number of chunks: ~30 chunks (average 3,300 tokens per chunk including overlap).
API calls: 30 sequential calls.
Cost: Using Claude 3 Haiku ($0.25 / 1M input tokens, $1.25 / 1M output tokens) and DeepSeek for some chapters, total cost per book ≈ $1.50–$3.00.
Latency: With async calls and retries, total translation time ≈ 4–6 minutes per book.
Quality: In a side-by-side comparison with naive paragraph-by-paragraph translation, our context-aware approach reduced pronoun errors by ~70% and terminology inconsistencies by ~80% (measured on a sample of 10 books).

Lessons Learned and Trade-offs

Overlap size matters: Too little overlap (1 paragraph) causes context breaks; too much (5+ paragraphs) leads to repetitive phrasing because the model sometimes re-translates the overlap. We settled on 2 paragraphs for narrative text, 1 for technical books.
Context format: We initially tried sending a summary of the previous chunk (generated by another LLM call) instead of the actual translation. The summary saved tokens but lost style and voice. Using the actual translated text as context preserved the author’s tone better, even if it cost more tokens.
Glossary maintenance: The glossary is critical for consistency, but building it automatically is tricky. We started with a regex for capitalized words, but that missed many terms. We now use a combination of spaCy named entity recognition and a small curated list. The LLM also suggests translations for new terms, which we review asynchronously.
PDF vs EPUB: EPUBs are far easier because they contain structural markup. For PDFs, we use pypdf to extract text, but we lose paragraph boundaries. We then use heuristics (double newlines, indentation) to reconstruct paragraphs. This works acceptably but sometimes merges dialogue lines incorrectly.
Rate limiting and retries: LLM APIs have rate limits. We implemented exponential backoff with tenacity and a semaphore to cap concurrent calls. On a VPS with 4 cores, we can translate 3–4 books concurrently without hitting limits.

What’s Next?

Our current approach works well for novels and non-fiction, but there’s an open question: how to handle cross-chapter references? A character introduced in chapter 1 might be mentioned again in chapter 10. Our glossary helps with names, but subtle narrative callbacks are lost. We’re experimenting with a book-level summary that is updated after each chapter and included as high-level context in the system prompt. Early results are promising but increase token usage by ~10%.

If you’re building a similar system, start with overlapping chunks and a glossary—it covers 80% of the consistency issues. The remaining 20% requires more sophisticated memory, but for most books, the trade-off isn’t worth the complexity.

Open question for the community: How do you handle translation of poetry or highly formatted text (tables, footnotes) without losing layout? We currently strip them or translate separately, but it’s not ideal. We’d love to hear your approaches.

Translating Entire Books with LLMs: A Chunking Strategy That Doesn't Lose Context

龚旭东 — Sat, 12 Sep 2026 03:01:49 +0000

How we built a Python pipeline to chunk books, preserve context, and maintain consistent terminology across hundreds of chapters.

The Problem: Translating a Whole Book, Not Just a Page

At LectuLibre, we let users upload an EPUB or PDF and get back a professionally translated book. The tricky part isn't calling an LLM—it's doing it across 100,000+ words without losing the plot, literally.

Naively, you might split the book into chunks that fit the LLM's context window and translate each independently. We tried that first. The result: names changed spelling halfway through, terminology was inconsistent, and dialogue references to earlier events sometimes made no sense. The story fell apart.

Context windows are huge now (Claude's 200k tokens, GPT-4 Turbo's 128k), but even those aren't enough for a full novel—and even if they were, cost and quality degrade with massive inputs. So we needed a chunking strategy that respects the document's structure and carries context across chunks.

Our Approach: Structure-Aware Chunking with Overlap and a Running Glossary

We broke the problem into three parts:

Split by document structure, not arbitrary token counts. Books are organized into chapters, sections, paragraphs. We extract those first.
Overlap chunks to give the model a few sentences of surrounding context for continuity.
Maintain a glossary of character names, place names, and technical terms that gets injected into every translation prompt.

Here's the pipeline in Python:

# Extract text from EPUB using ebooklib
import ebooklib
from ebooklib import epub
from bs4 import BeautifulSoup

def extract_chapters(epub_path):
    book = epub.read_epub(epub_path)
    chapters = []
    for item in book.get_items_of_type(ebooklib.ITEM_DOCUMENT):
        soup = BeautifulSoup(item.get_content(), 'html.parser')
        # Extract paragraphs, join with newlines
        text = '\n\n'.join(p.get_text() for p in soup.find_all('p'))
        if text.strip():
            chapters.append(text.strip())
    return chapters

That gives us a list of chapters. But chapters can be long, so we further split them into chunks of roughly 4,000 tokens (a sweet spot we found for quality vs. cost). We use tiktoken to count tokens, and we split at paragraph boundaries to avoid cutting sentences.

import tiktoken

enc = tiktoken.get_encoding("cl100k_base")  # Works for many models

def count_tokens(text):
    return len(enc.encode(text))

def chunk_chapter(chapter_text, max_tokens=4000, overlap_tokens=200):
    paragraphs = chapter_text.split('\n\n')
    chunks = []
    current_chunk = ""
    current_tokens = 0

    for para in paragraphs:
        para_tokens = count_tokens(para)
        # If a single paragraph exceeds max_tokens (rare for normal prose), hard split it
        if para_tokens > max_tokens:
            # Hard split by sentences if a paragraph is too long
            sentences = para.replace('\n', ' ').split('. ')
            temp = ""
            for sent in sentences:
                sent_tokens = count_tokens(sent)
                if current_tokens + sent_tokens > max_tokens:
                    chunks.append(current_chunk)
                    # Add overlap from previous chunk end
                    if chunks:
                        overlap_text = ' '.join(chunks[-1].split()[-overlap_tokens:])
                        current_chunk = overlap_text + " " + sent
                        current_tokens = count_tokens(current_chunk)
                    else:
                        current_chunk = sent
                        current_tokens = sent_tokens
                else:
                    current_chunk += " " + sent if current_chunk else sent
                    current_tokens += sent_tokens
            continue

        if current_tokens + para_tokens > max_tokens:
            # Finish current chunk
            chunks.append(current_chunk)
            # Start new chunk with overlap from previous chunk's tail
            overlap_text = ' '.join(current_chunk.split()[-overlap_tokens:])
            current_chunk = overlap_text + " " + para
            current_tokens = count_tokens(current_chunk)
        else:
            current_chunk += "\n\n" + para if current_chunk else para
            current_tokens += para_tokens

    if current_chunk:
        chunks.append(current_chunk)
    return chunks

The overlap ensures the model sees the last ~200 tokens of the previous chunk, which helps it maintain flow and avoid abrupt style changes. We tested different overlap sizes: 100 tokens was sometimes not enough for a sentence to complete; 500 tokens added cost without quality gain. 200 seemed the best compromise.

Carrying Context with a Dynamic Glossary

Overlap alone doesn't solve terminology consistency. A character introduced in chapter 1 as "Elara" might become "Elena" in chapter 20 because the model doesn't remember the earlier choice. So we maintain a glossary.

After translating each chunk, we run a lightweight extraction on the translation to pull out proper nouns and key terms. We use a simple heuristic with spaCy for named entity recognition, then ask the LLM to verify and suggest a consistent translation for each entity. The glossary is stored as a dictionary, and we inject it into the system prompt of every translation call.

import spacy
nlp = spacy.load("en_core_web_sm")  # or target language model

def extract_entities(text, lang='en'):
    doc = nlp(text)
    entities = set()
    for ent in doc.ents:
        if ent.label_ in ['PERSON', 'GPE', 'ORG', 'PRODUCT']:
            entities.add(ent.text)
    return entities

# After translating a chunk, we call this:
def update_glossary(translated_chunk, glossary):
    # Extract candidate entities from the translated text
    candidates = extract_entities(translated_chunk)
    # For each candidate, ask the LLM to provide a canonical translation
    # (we batch them in one prompt for efficiency)
    if candidates:
        prompt = f"Given these terms from a book: {list(candidates)}. Provide a consistent translation for each in the target language. Return JSON."
        # Call LLM (example with Anthropic)
        import anthropic
        client = anthropic.Anthropic()
        response = client.messages.create(
            model="claude-3-haiku-20240307",
            max_tokens=300,
            system="You are a terminology manager for book translation.",
            messages=[{"role": "user", "content": prompt}]
        )
        # Parse response and update glossary
        # (pseudo-code, actual parsing omitted for brevity)
        new_terms = parse_llm_json(response.content[0].text)
        glossary.update(new_terms)
    return glossary

The glossary is then included in the system prompt of the next chunk's translation:

def translate_chunk(chunk, glossary, target_lang='es', source_lang='en'):
    system_prompt = f"Translate the following text from {source_lang} to {target_lang}. Maintain the style of a novel. Use these translations for recurring terms: {glossary}"
    # Call LLM...

We also keep a context summary—a one-paragraph summary of the plot so far—that gets appended to the system prompt after every few chapters. We generate it periodically by feeding the LLM the last few translated chunks and asking for a summary. This summary acts as a high-level memory without bloating the prompt.

def summarize_progress(translated_chunks, max_tokens=1000):
    # Join last few chunks, ask for a summary
    recent = " ".join(translated_chunks[-5:])
    prompt = f"Summarize the key events, characters, and style of the following book excerpt in 3-4 sentences:\n{recent}"
    # Call LLM, return summary

Results, Costs, and Trade-offs

With this approach, we translated a 120,000-word novel to Spanish. Here are the concrete numbers:

Total chunks: ~140 (average chunk size 3,800 tokens after overlap).
Cost: Using Claude 3 Haiku for translation and glossary updates, total API cost was $4.20. Using Sonnet for final quality pass on selected chapters added another $3.50. So about $0.06 per 1,000 words—cheap enough for a subscription service.
Time: The whole pipeline ran in under 20 minutes on a single VPS, including EPUB parsing, chunking, translation, glossary updates, and reassembly.
Quality: Compared to our naive independent-chunk baseline, the glossary + overlap + summary improved consistency dramatically. We measured a 78% reduction in inconsistent term usage (based on manual review of 50 random name occurrences). The flow was also much smoother; beta readers reported fewer "jarring" transitions.

But there are trade-offs:

Overlap costs extra tokens: The 200-token overlap adds about 5% more tokens per chunk. For very long books, that's not negligible.
Complexity: The pipeline is more complex than a simple map-reduce. We had to handle edge cases like footnotes, images, and dialogue formatting. Our parser sometimes struggles with PDFs that have complex layouts; we recommend EPUB for best results.
Glossary extraction is imperfect: spaCy misses some entities, and the LLM sometimes suggests inconsistent translations. We mitigated by having a human review step for premium users, but it's not perfect.
Long-range dependencies: Even with a summary, the model can forget details from 50 chapters ago. We experimented with a vector store (using FAISS) to retrieve relevant paragraphs from earlier chapters, but it increased latency and cost without a clear quality boost for most books. We may revisit for non-fiction with heavy cross-references.

Lessons Learned

Respect the document's structure. Arbitrary token windows break paragraphs and chapters. Use natural boundaries.
A small overlap goes a long way. Just 200 tokens of previous context significantly improves cohesion.
Terminology management is non-negotiable. For any long-form translation, you need a glossary. It's the difference between a professional book and a garbled mess.
Iterate on chunk size. We tested 2k, 4k, 8k tokens. 4k gave the best trade-off between context and cost for our use case, but your mileage may vary.

What's Next?

We're working on better handling of cultural references and idioms—things that don't translate literally and need extra context. An open question for the community: How do you handle footnotes and endnotes in LLM-based translation? We currently strip them and reinsert manually, but that's clunky. Would love to hear ideas.

Translating 300-Page Books with Claude: Taming Token Limits and Context Windows

龚旭东 — Wed, 09 Sep 2026 03:01:49 +0000

How we chunk long-form content and maintain translation quality with Claude API

When we launched LectuLibre, our AI-powered book translation platform, we thought the hard part would be fine-tuning translation quality. It turned out the real engineering challenge was more mundane: Claude's token limits.

A 300-page novel contains roughly 120,000 tokens. Claude 3 Sonnet has a 200K context window, so you might assume you can just send the whole book and ask for a translation. But the output token limit is only 4,096 tokens (8,192 for some models). Even if the input fits, asking for a 120,000-token translation in one call will hit a wall. The API will return a truncated response or an error.

In this post, I'll walk through the chunking and orchestration system we built to translate long-form content with Claude reliably and cost-effectively, without losing narrative consistency.

The core problem: input fits, output doesn't

Books are naturally long. A typical 300-page EPUB produces around 100,000–150,000 tokens. Claude's context window can hold that input, but the model's maximum output tokens are capped (4,096 for Sonnet). You cannot ask the model to output a full book translation in one API call. Even if you could, quality suffers: the model may lose focus, repeat content, or hallucinate details over very long generations.

We needed a system that would:

Split the source text into manageable chunks that fit both input and output limits.
Preserve context across chunks to keep terminology and style consistent.
Process chunks in parallel to keep user wait times reasonable.
Handle PDF extraction artifacts (headers, footers, page numbers) and EPUB chapter structures.

Chunking strategy: sentence-aware, overlapping windows

Our first naive attempt used textwrap or simple character-based slicing. That produced chunks ending mid-sentence, which led to awkward translations and lost pronouns. We quickly moved to a paragraph- and sentence-aware splitter.

We use tiktoken with the cl100k_base encoding as a fast approximation for Claude's token count. It's not exact, but close enough for chunk sizing. The actual anthropic SDK also provides a count_tokens method if you need precision.

Here's the chunker we use:

import tiktoken
import re

enc = tiktoken.get_encoding('cl100k_base')

def count_tokens(text: str) -> int:
    return len(enc.encode(text))

def split_into_chunks(text: str, max_tokens=8000, overlap_tokens=200) -> list[str]:
    paragraphs = re.split(r'\n\s*\n', text)
    chunks = []
    current = ''
    current_tokens = 0
    for para in paragraphs:
        para_tokens = count_tokens(para)
        if current_tokens + para_tokens > max_tokens and current:
            chunks.append(current)
            # keep last overlap_tokens worth from current as overlap
            overlap_text = current[-overlap_tokens:] if len(current) > overlap_tokens else current
            current = overlap_text
            current_tokens = count_tokens(current)
        current += '\n\n' + para
        current_tokens = count_tokens(current)
    if current:
        chunks.append(current)
    return chunks

We set max_tokens=8000 for source chunks. That leaves room for the system prompt and any glossary we inject, while keeping the expected translation output under 4,096 tokens. The overlap of 200 tokens (about 150 English words) ensures the next chunk starts with a bit of context already seen, reducing boundary errors.

A subtle improvement: we also split paragraphs into sentences using nltk.sent_tokenize before building chunks if a single paragraph is longer than the max token limit. That way we never have a chunk that exceeds the limit because of one giant paragraph.

Carrying context across chunks

Translation quality depends heavily on continuity. Names, places, and invented terms must stay consistent. Our approach is to include a running context in the prompt for each chunk. This context consists of:

The last ~500 characters of the previous source chunk.
The corresponding translation of that tail.
A glossary of proper nouns extracted from the entire book.

We build the glossary once before translation using spaCy:

import spacy
nlp = spacy.load('en_core_web_sm')

def build_glossary(text: str, max_terms=100) -> str:
    doc = nlp(text[:100000])  # limit for speed
    names = set()
    for ent in doc.ents:
        if ent.label_ in ['PERSON', 'ORG', 'GPE', 'LOC']:
            names.add(ent.text)
    return ', '.join(sorted(names)[:max_terms])

Then, for each chunk translation, we prepend the glossary and previous context to the prompt:

prompt = f'''Translate the following book text from English to {target_lang}.
Maintain style, tone, and terminology. Use the glossary:
{glossary_str}

Previous translated context (for consistency):
{prev_context}

Source text:
{chunk}
'''

This two-part context (glossary + previous tail) dramatically reduced name inconsistencies in our tests. Initially we saw character names translated differently in later chapters; after adding the glossary, those errors dropped to near zero.

Asynchronous orchestration with rate limiting

Claude's API has rate limits (requests per minute and tokens per minute). To speed up translation while respecting those limits, we use anthropic.AsyncAnthropic with an asyncio.Semaphore. We typically run 4 concurrent requests for Claude Sonnet.

Here's the core translation function:

import asyncio
import anthropic

client = anthropic.AsyncAnthropic(api_key='...')

async def translate_chunk(chunk: str, prev_context: str, glossary_str: str, target_lang: str, semaphore: asyncio.Semaphore) -> str:
    async with semaphore:
        prompt = f'''Translate the following book text from English to {target_lang}.
Maintain style, tone, and terminology. Use the glossary:
{glossary_str}

Previous translated context (for consistency):
{prev_context}

Source text:
{chunk}
'''
        for attempt in range(3):
            try:
                response = await client.messages.create(
                    model='claude-3-sonnet-20240229',
                    max_tokens=4096,
                    temperature=0.3,
                    system='You are a professional literary translator.',
                    messages=[{'role': 'user', 'content': prompt}]
                )
                return response.content[0].text
            except anthropic.RateLimitError:
                await asyncio.sleep(60 * (attempt + 1))
        raise RuntimeError('Rate limit retries exhausted')

The retry loop handles transient 429 errors. We also catch anthropic.APIStatusError for server-side overloads.

The main loop processes chunks sequentially to maintain context, but the actual API calls are concurrent within each chunk? Actually, we process chunks one by one in order because each chunk's context depends on the previous translation. To get some parallelism, we can translate chunks in batches where each batch shares the same previous context, but we found that reduced quality slightly. For now, sequential chunk translation with a concurrency of 1 per book is simpler and only takes about 2–3 minutes for a 300-page book using Sonnet.

If you need higher throughput, you could translate independent sections in parallel (e.g., different chapters) and then merge, but you lose the running context.

Concrete performance and cost numbers

For a 300-page novel (~120,000 tokens), our chunker produces around 16–20 chunks of 8,000 tokens with overlap. Translating to Spanish with Claude 3 Sonnet:

Total input tokens sent: ~135,000 (including overlaps and prompt overhead).
Total output tokens: ~110,000.
Cost: $0.45 for input ($3/M) + $1.65 for output ($15/M) = ~$2.10 per book.
Wall-clock time: 2–3 minutes with sequential chunks and no retries.

We experimented with Claude 3 Haiku for a cheaper option ($0.25/M input, $1.25/M output). It cost about $0.35 per book but produced noticeably worse translations, especially for literary text with figurative language. For technical or non-fiction books, Haiku is acceptable; for novels, we stick with Sonnet.

Handling PDF extraction problems

PDF files are the wild west. Headers, footers, page numbers, and two-column layouts wreak havoc on chunking. We use PyMuPDF (fitz) because it gives us block-level text with bounding boxes, allowing us to filter out headers/footers based on vertical position. For EPUB, we use ebooklib to iterate over spine items and extract HTML, then strip tags with BeautifulSoup. This preserves chapter boundaries, which we use as natural chunk boundaries before applying token-based splitting.

One hard-won lesson: always remove page numbers and repeated header text before tokenizing the full text. Otherwise, those artifacts become part of the chunks and get translated, producing garbage in the middle of chapters.

Lessons learned and open questions

What worked:

Sentence- and paragraph-aware chunking with token overlap.
Injecting a glossary and previous translation tail as context.
Asynchronous retry for rate limits.
Using PyMuPDF with positional filtering for PDFs.

What didn't:

Character-based splitting caused mid-sentence chunks.
Parallel chunk translation without shared context led to name and style drift.
Trusting output token limit blindly: some languages expand by 20–30%, so a 8,000-token source chunk can produce >4,096 tokens in German. We had to reduce source chunk size to 6,500 for such languages.

Open question for the community: Have you found a reliable way to translate poetry, code, or tables inside long documents without breaking the surrounding narrative? We currently treat them as opaque blocks and translate them separately, but integration remains rough.

Building LectuLibre taught us that long-form LLM translation is less about model capability and more about solid engineering around context management. If you're building something similar, start with robust chunking and context propagation—the model will handle the rest.

Translating 300-Page Books with Claude: Taming Token Limits and Chunking Strategies

龚旭东 — Sat, 05 Sep 2026 03:02:12 +0000

How we built a reliable pipeline to split long texts for LLM translation without losing context or breaking the bank

At LectuLibre, we translate entire books using Claude. The challenge: a 300-page book is roughly 90,000–120,000 words, which translates to 120,000–160,000 tokens. While Claude 3 models have a 200k context window, sending an entire book in one API call is impractical. It's slow, expensive, and often degrades translation quality due to attention dilution. We needed a robust chunking strategy that preserved context and stayed within token limits.

The Problem: One Book, Too Many Tokens

When we first started building LectuLibre, we naively assumed we could just pass the whole book to Claude and get a translation back. We quickly hit three walls:

Rate limits: A single request with 150k tokens triggered API timeouts and 429 errors.
Cost: Even if it worked, processing 150k tokens per request with Opus would cost over $13 per book, and most of the input would be wasted on repeated context.
Quality: Long contexts tend to make the model "forget" early chapters, leading to inconsistent character names and terminology.

Clearly, chunking was necessary. But how do you split a book without losing narrative flow?

First Attempt: Naive Splitting by Paragraphs

Our initial approach was simple: split the text into chunks of roughly 10,000 tokens by paragraphs. We used a regex to split on double newlines and then concatenated paragraphs until we hit the token limit.

import re

def split_into_paragraphs(text: str) -> list[str]:
    return re.split(r'\n\s*\n', text)

def chunk_by_paragraphs(paragraphs: list[str], max_tokens: int = 10000) -> list[str]:
    chunks = []
    current_chunk = []
    current_tokens = 0
    for para in paragraphs:
        # Estimate tokens using character count / 4 (quick and dirty)
        para_tokens = len(para) // 4
        if current_tokens + para_tokens > max_tokens and current_chunk:
            chunks.append('\n\n'.join(current_chunk))
            current_chunk = []
            current_tokens = 0
        current_chunk.append(para)
        current_tokens += para_tokens
    if current_chunk:
        chunks.append('\n\n'.join(current_chunk))
    return chunks

This worked for small documents, but for books it produced broken sentences at chunk boundaries. A single sentence could be split across two chunks, causing the model to translate the halves differently. Worse, context from the previous chapter was completely lost, leading to inconsistent names and terminology.

Token Counting with Tiktoken

Before improving chunking, we needed accurate token counts. Character-based estimates were unreliable, especially for non-English languages. We switched to tiktoken, the tokenizer used by OpenAI and compatible with Claude's tokenizer.

import tiktoken

def count_tokens(text: str) -> int:
    encoding = tiktoken.get_encoding("cl100k_base")
    return len(encoding.encode(text))

Using tiktoken gave us exact token counts, which was crucial for staying within limits and estimating costs.

Chunking with Overlap and Context Windows

The naive split ignored context between chunks. We fixed this by introducing overlap—each chunk would include a few paragraphs from the previous chunk to provide continuity. Additionally, we added a preamble to each chunk containing the book's title, author, and a glossary of recurring terms.

def chunk_paragraphs_with_overlap(
    paragraphs: list[str],
    max_tokens: int = 10000,
    overlap_paragraphs: int = 2,
) -> list[str]:
    encoding = tiktoken.get_encoding("cl100k_base")
    chunks = []
    current_chunk = []
    current_tokens = 0

    for i, para in enumerate(paragraphs):
        para_tokens = len(encoding.encode(para))
        if current_tokens + para_tokens > max_tokens and current_chunk:
            # Finalize current chunk
            chunks.append('\n\n'.join(current_chunk))
            # Keep last few paragraphs for overlap
            overlap = current_chunk[-overlap_paragraphs:] if overlap_paragraphs > 0 else []
            current_chunk = overlap.copy()
            current_tokens = sum(len(encoding.encode(p)) for p in current_chunk)
        current_chunk.append(para)
        current_tokens += para_tokens

    if current_chunk:
        chunks.append('\n\n'.join(current_chunk))
    return chunks

This simple overlap improved consistency at boundaries. However, we still had issues with sentence splits. A paragraph might be very long, and the token limit could cut it mid-sentence. We needed to split at sentence boundaries within paragraphs when necessary.

Respecting Sentence Boundaries

We used a lightweight sentence splitter (nltk.sent_tokenize was too slow for large books, so we used a regex-based splitter) to break paragraphs into sentences first, then chunk sentences with overlap. This ensured no sentence was ever split across chunks.

import re

def split_into_sentences(text: str) -> list[str]:
    # Simple regex for sentence boundaries (handles periods, exclamations, questions)
    sentences = re.split(r'(?<=[.!?])\s+', text)
    return [s.strip() for s in sentences if s.strip()]

def chunk_sentences_with_overlap(
    sentences: list[str],
    max_tokens: int = 10000,
    overlap_sentences: int = 3,
) -> list[str]:
    encoding = tiktoken.get_encoding("cl100k_base")
    chunks = []
    current_chunk = []
    current_tokens = 0

    for sent in sentences:
        sent_tokens = len(encoding.encode(sent))
        if current_tokens + sent_tokens > max_tokens and current_chunk:
            chunks.append(' '.join(current_chunk))
            overlap = current_chunk[-overlap_sentences:] if overlap_sentences > 0 else []
            current_chunk = overlap.copy()
            current_tokens = sum(len(encoding.encode(s)) for s in current_chunk)
        current_chunk.append(sent)
        current_tokens += sent_tokens

    if current_chunk:
        chunks.append(' '.join(current_chunk))
    return chunks

Now our chunks were mostly clean, with no broken sentences and enough overlap to keep context.

The Translation Pipeline

With chunking in place, we built an asynchronous pipeline to translate each chunk using the Anthropic Python client. We added retry logic with exponential backoff for rate limits and used a semaphore to limit concurrency.

import anthropic
import asyncio
import random

client = anthropic.AsyncAnthropic(api_key="YOUR_API_KEY")

async def translate_chunk(
    chunk: str,
    glossary: str,
    model: str = "claude-3-haiku-20240307",
    max_retries: int = 5,
) -> str:
    system_prompt = (
        "You are a professional book translator. "
        "Translate the given text to Spanish. "
        "Preserve formatting, tone, and style.\n\n"
        f"Glossary:\n{glossary}"
    )
    for attempt in range(max_retries):
        try:
            response = await client.messages.create(
                model=model,
                max_tokens=4096,
                temperature=0.2,
                system=system_prompt,
                messages=[{"role": "user", "content": chunk}],
            )
            return response.content[0].text
        except anthropic.RateLimitError:
            wait = 2 ** attempt + random.random()
            await asyncio.sleep(wait)
        except Exception as e:
            print(f"Error: {e}, retrying...")
            await asyncio.sleep(1)
    raise Exception("Max retries exceeded")

async def translate_book(chunks: list[str], glossary: str) -> str:
    semaphore = asyncio.Semaphore(3)  # Limit concurrent requests
    async def translate_one(chunk):
        async with semaphore:
            return await translate_chunk(chunk, glossary)
    tasks = [translate_one(chunk) for chunk in chunks]
    translated_chunks = await asyncio.gather(*tasks)
    return '\n\n'.join(translated_chunks)

We used claude-3-haiku for cost efficiency during development, but in production we allow users to choose their preferred model. Haiku is extremely fast and cheap, while Opus provides the highest quality for complex literary works.

Context Preservation: Glossaries and Style

Even with overlap, long-range consistency (e.g., a character introduced in chapter 1 appearing in chapter 20) was still a challenge. We addressed this by extracting a glossary of key terms and character names before translation.

We built a simple extraction step: we send a few sample chapters to Claude and ask it to list important terms, names, and their preferred translations. That glossary is then included in the system prompt of every chunk translation.

async def extract_glossary(sample_text: str) -> str:
    response = await client.messages.create(
        model="claude-3-sonnet-20240229",
        max_tokens=1000,
        temperature=0.1,
        system="Extract a glossary of important terms, character names, and places from the text. Format as 'Original: Translation' pairs.",
        messages=[{"role": "user", "content": sample_text[:20000]}],
    )
    return response.content[0].text

This glossary reduced inconsistencies dramatically. For example, a character named "John" would always be translated as "Juan" instead of sometimes "Jean" or "Giovanni".

Results and Trade-offs

After implementing the full pipeline, here's what we observed for a typical 300-page novel (approx. 100k tokens):

Chunk size: 8,000–12,000 tokens per chunk worked best. Smaller chunks lost too much context; larger chunks approached the limits of quality degradation.
Overlap: 300–500 tokens (2–3 sentences) of overlap virtually eliminated boundary artifacts.
Cost: Translating with Haiku cost about $0.15 per book, Sonnet about $1.50, and Opus about $9.00. This is a fraction of what a single full-context call would cost, and quality is much better.
Speed: With concurrency of 3, a 100k token book translated in under 2 minutes using Haiku, and about 5 minutes with Sonnet. A single full-context call would often time out.
Failure: We initially tried using Claude's long context to translate an entire book in one go. Not only did we hit API timeouts, but the translation quality dropped noticeably after the first 10k tokens—the model would start skipping paragraphs and mixing up characters.

One trade-off we accepted: the overlap means we translate some text twice. That's a small redundancy (about 5% extra tokens) but worth it for consistency.

Lessons Learned and Next Steps

Always count tokens accurately with tiktoken—don't guess.
Overlap is a cheap insurance policy against context loss.
Glossaries are essential for long-form translation; they keep terminology consistent across chunks.
Respect rate limits with exponential backoff and concurrency limits.

We're still exploring how to handle cross-chapter references more gracefully. One idea: generate a summary of each chapter as it's translated and include that summary in subsequent chunks, effectively building a "memory" for the model. We'd love to hear how others have tackled this.

Open question: How do you preserve narrative voice and style across hundreds of chunks? We've found temperature and prompt engineering help, but there's room for improvement. Share your thoughts in the comments!

Chunking 300-Page Books for Claude: How We Beat Token Limits in AI Translation

龚旭东 — Wed, 02 Sep 2026 03:01:56 +0000

A practical guide to slicing long documents, preserving context, and avoiding context-window explosions.

When we first set out to build LectuLibre, our AI-powered book translation service, the promise was simple: upload an EPUB or PDF, and get a professionally translated book back. The reality? A 300-page book contains roughly 90,000 words—around 120,000 tokens—and while Claude's 200K context window can hold that, doing so in a single API call is expensive, slow, and often produces lower-quality translations because the model loses focus.

So we had to answer a fundamental question: how do you translate a 300-page book with an LLM without blowing up the context window or your budget?

Here's how we solved it—the chunking strategy, the code, the trade-offs, and the lessons we learned the hard way.

The Problem: Long Texts vs. Context Windows

Claude 3.5 Sonnet offers a 200,000-token context window. That sounds like a lot—it is—but for a complete book, it's still not enough. And even if it were, there are three practical reasons we avoid sending an entire book in one prompt:

Cost: Token-based pricing means a single call with 120k tokens would cost several dollars. Doing that for every book translation is unsustainable.
Latency: A 120k-token input takes minutes to process. Users expect a translation in minutes, not hours.
Quality: LLMs tend to "skim" when given extremely long inputs. We found that translations of later chapters suffered from dropped details and inconsistent terminology.

So chunking was mandatory. But naive chunking—splitting by fixed character count or by paragraph without context—led to another problem: broken coherence. Sentences that referenced previous paragraphs, character names, or plot points became disjointed. We needed a chunking approach that preserved enough context for the model to translate accurately while keeping each chunk small enough to be fast and cheap.

Our Approach: Overlapping Window Chunking with Continuation Memory

We chose a hybrid strategy:

Split the book into semantic units (paragraphs) using a lightweight parser.
Group paragraphs into chunks of approximately 4,000 tokens each, with a 500-token overlap from the previous chunk.
Before translating each chunk, prepend a continuation memory—a short summary of the previous chunk's translation, generated automatically by the model itself.
Use tiktoken (the same tokenizer OpenAI uses, which is compatible with Claude's token counting for English text) to estimate token counts accurately.

Why 4,000 tokens? It's small enough to keep latency low (typically under 10 seconds per chunk) and cost manageable, while still providing enough context for coherent paragraphs. The 500-token overlap ensures that sentences split at chunk boundaries can be resolved with the missing context. The continuation memory helps maintain consistency across chunks, especially for character names and specialized terminology.

Implementation Details: The Python Code

Our backend is Python/FastAPI, and we use the official anthropic Python SDK. For PDF/EPUB parsing, we use pymupdf for PDFs and ebooklib for EPUBs, extracting text into a list of paragraphs. Here's the core chunking function:

import tiktoken
from typing import List, Tuple

def chunk_paragraphs(
    paragraphs: List[str],
    max_tokens: int = 4000,
    overlap_tokens: int = 500
) -> List[Tuple[int, int, str]]:
    """
    Group paragraphs into chunks with token-based limits and overlap.
    Returns list of (start_index, end_index, chunk_text).
    """
    enc = tiktoken.get_encoding("cl100k_base")
    chunks = []
    current_chunk = []
    current_tokens = 0
    start_idx = 0

    for i, para in enumerate(paragraphs):
        para_tokens = len(enc.encode(para))

        # If adding this paragraph exceeds max_tokens, finalize current chunk
        if current_tokens + para_tokens > max_tokens and current_chunk:
            # Save current chunk
            chunk_text = "\n\n".join(current_chunk)
            chunks.append((start_idx, i - 1, chunk_text))

            # Start new chunk with overlap from the end of previous chunk
            overlap_start = max(start_idx, i - overlap_tokens)
            overlap_paragraphs = paragraphs[overlap_start:i]
            current_chunk = overlap_paragraphs.copy()
            current_tokens = sum(len(enc.encode(p)) for p in current_chunk)
            start_idx = overlap_start

        current_chunk.append(para)
        current_tokens += para_tokens

    # Add the last chunk
    if current_chunk:
        chunk_text = "\n\n".join(current_chunk)
        chunks.append((start_idx, len(paragraphs) - 1, chunk_text))

    return chunks

A few notes on this code:

We use cl100k_base encoding, which is the tokenizer for GPT-4 and also works well for Claude because both models use similar byte-pair encoding. It's not perfect for all languages, but for English source text it's accurate enough.
Overlap is implemented by keeping the last overlap_tokens worth of paragraphs from the previous chunk at the beginning of the new chunk. This means some paragraphs are translated twice, but the second translation has the benefit of context from the first.
The function returns indices so we can later assemble the final translation in order, discarding duplicate translations from overlap sections.

Next, we handle the actual translation call. We use a function that takes a chunk, the continuation memory, and returns the translated text, with retry logic for API errors:

import anthropic
from tenacity import retry, stop_after_attempt, wait_exponential

client = anthropic.Anthropic()

@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=2, max=10))
def translate_chunk(chunk_text: str, memory: str, target_lang: str) -> str:
    """Translate a single chunk using Claude, with continuation memory."""
    prompt = f"""You are a professional book translator. Translate the following text into {target_lang}.

Continuation context from previous section (for consistency):
{memory}

Text to translate:
{chunk_text}

Translation (only the translated text, no explanations):"""

    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=4096,
        temperature=0.3,
        system="You are a literary translator. Preserve tone, style, and terminology. Keep names and specialized terms consistent.",
        messages=[{"role": "user", "content": prompt}]
    )
    return response.content[0].text

After each chunk is translated, we generate a brief summary of that translation to serve as memory for the next chunk. This is done by a lightweight call to Claude with the translated text:

def generate_memory(translated_text: str) -> str:
    """Generate a short summary of key terms and style from translated text."""
    prompt = f"""Summarize the following translated text in 3-5 sentences, focusing on character names, specialized terminology, and stylistic choices. Keep it concise.

{translated_text}

Summary:"""
    response = client.messages.create(
        model="claude-3-5-haiku-20241022",  # cheaper model for summarization
        max_tokens=500,
        temperature=0.0,
        messages=[{"role": "user", "content": prompt}]
    )
    return response.content[0].text

We use claude-3-5-haiku for memory generation because it's fast and cheap, and the summarization doesn't require the full power of Sonnet.

The Assembly Problem: Merging Chunks Without Duplicates

Because of the overlap, we end up translating some paragraphs twice. To assemble the final book, we need to align the translated chunks back to the original paragraph order. Our chunk_paragraphs function returned start and end indices, but those are based on the original paragraph list, and the overlap means the same paragraph may appear in multiple chunks.

Our solution: after translating each chunk, we split the translated text back into paragraphs (using newlines as delimiters) and store them in a dictionary keyed by the original paragraph index. For paragraphs that appear in multiple chunks, we keep only the translation from the chunk where the paragraph is not part of the overlap (i.e., the chunk where it appears as new content). This ensures we use the translation that had the most context.

Here's a simplified version of the assembly logic:

def assemble_translation(chunks, translated_chunks, total_paragraphs):
    """Merge translated chunks into final paragraph list."""
    final = [None] * total_paragraphs
    for (start, end, _), trans_text in zip(chunks, translated_chunks):
        trans_paragraphs = trans_text.split("\n\n")
        # Determine which paragraphs are new (not overlap) by mapping to original indices
        # We assume paragraphs in chunk correspond to indices start..end
        # But due to overlap, some are duplicates; we only fill missing ones
        for i, para in enumerate(trans_paragraphs):
            orig_idx = start + i
            if orig_idx <= end and final[orig_idx] is None:
                final[orig_idx] = para
    # Fill any remaining None with empty string (shouldn't happen)
    return "\n\n".join(p for p in final if p is not None)

This assumes the translation preserves paragraph boundaries, which is mostly true for Claude with our prompt. If not, we have a fallback that uses regex to split on blank lines.

Performance and Cost Numbers

After implementing this system, we translated a 320-page novel (about 98,000 words). Here are the real numbers:

Total tokens processed: ~140,000 input tokens across all chunks (including overlaps and memory prompts), ~110,000 output tokens.
Number of chunks: 42 chunks (average ~3,300 tokens per chunk after trimming).
Total cost: Approximately $1.20 using Claude 3.5 Sonnet for translation and Haiku for memory. A single-call approach with full context would have cost roughly $5-7 and taken much longer.
Total translation time: 12 minutes end-to-end, including API latency and post-processing.
Quality: The continuation memory eliminated most terminology inconsistencies. We saw a 90% reduction in name variations compared to naive chunking without overlap or memory.

Lessons Learned and Trade-offs

Token counting is never exact. tiktoken works great for English, but we had to adjust for languages with different token densities. We added a 10% safety margin on chunk sizes to avoid exceeding Claude's per-request token limit.
Overlap size matters. Too little overlap (e.g., 100 tokens) still caused broken context; too much overlap (2000 tokens) increased cost and duplicate work. 500 tokens (about 1-2 paragraphs) was the sweet spot for book text.
Continuation memory is a game-changer. Without it, the model sometimes translated the same character name differently across chapters. With a simple summary of key terms, consistency improved dramatically. We considered using a vector database for long-term memory, but for a single book, a rolling summary is sufficient and simpler.
Footnotes and special formatting are tricky. Books with heavy formatting (code blocks, poetry, tables) broke our paragraph-based splitting. We had to implement a pre-processing step that extracts and protects these elements, then re-inserts them after translation.
The assembly step can be error-prone. If the model merges or splits paragraphs unexpectedly, the index mapping fails. We added a validation step that checks the number of translated paragraphs matches the expected count and flags discrepancies for manual review.

Open Question for the Community

We're still exploring better ways to maintain global consistency across an entire book without drastically increasing cost. Some ideas we're considering:

Using a two-pass approach: first pass generates a glossary and style guide from the full book (using a cheaper model on summaries), second pass translates chunks with that glossary as part of the system prompt.
Experimenting with Claude's prompt caching to reuse the continuation memory across chunks, reducing input tokens.
For books with many recurring characters, perhaps a knowledge graph updated as translation progresses.

How have you handled long-form text with LLMs? Any tips for maintaining consistency across thousands of tokens? We'd love to hear your approaches in the comments.

If you're building something similar, our main takeaway is this: chunking is necessary, but how you chunk and what context you carry between chunks determines the quality of the final output. Overlap + memory is a simple but effective pattern that works well for book-length documents.

Translating 300-Page Books with Claude: How We Handle Token Limits Without Losing Context

龚旭东 — Sat, 29 Aug 2026 03:01:24 +0000

A practical guide to chunking, token counting, and maintaining narrative flow when using the Anthropic API for long-form translation.

At LectuLibre, we let users upload an EPUB or PDF and get a full book translation powered by Claude. A 300-page novel might sound manageable until you realize it's around 100,000 tokens. Claude's context window can hold it, but the output limit, cost, and latency make sending the whole book at once a non-starter. Here's the chunking strategy we settled on after burning through our API quota and a few failed attempts.

The Problem

A typical 300-page book contains roughly 80,000–100,000 words. For Claude, that's about 120,000 tokens (using a rough 1.3 tokens per word). While Claude 3 Opus has a 200K token context window, the output limit is much lower (4096 tokens for Opus, 8192 for Sonnet). Translating an entire book in one API call would mean:

Cost: Input tokens are expensive. At $15 per million input tokens for Opus, a single book translation would cost around $1.80 just for input — but that's the cheapest part. Output tokens cost $75 per million. If the translation is roughly the same length as the original, you'd generate another 120K tokens, adding $9. That's over $10 per book, not including retries.
Latency: Generating 120K tokens sequentially takes a long time. Even at 50 tokens per second, that's 40 minutes per book.
Reliability: One long request is more likely to hit a timeout, rate limit, or network error. If it fails after 100K tokens, you start over.

Clearly, we needed to break the book into smaller chunks and translate each independently. But naive chunking (e.g., splitting by a fixed number of characters) created broken sentences, lost narrative context, and inconsistent terminology across chapters.

Our First Attempt: Fixed 4,000-Token Chunks

We started with the simplest approach: split the book into chunks of 4,000 tokens using a rough character-to-token estimate (we assumed 4 characters per token). We then sent each chunk to Claude with a system prompt like:

system_prompt = """
You are a professional literary translator. Translate the following text from {source_lang} to {target_lang}. 
Preserve the author's style, tone, and cultural nuances. Do not add or omit content.
"""

This failed in a few predictable ways:

Mid-sentence splits: Chunks often cut a paragraph or sentence in half. Claude translated each piece literally, producing awkward phrases at chunk boundaries.
Inconsistent terminology: Character names, place names, and recurring phrases were translated differently across chunks.
Lost context: Without the previous chapter's summary, Claude sometimes misinterpreted pronouns or ambiguous references.
Token miscounts: Our 4-char-per-token estimate was wrong, leading to chunks that exceeded the output limit or were too small, increasing API calls.

We burned through a lot of credits fixing these issues manually. Then we moved to a token-aware, context-preserving chunker.

A Better Chunking Strategy

We now split books using three levels:

Structural units: We parse the EPUB/PDF into chapters or sections using libraries like ebooklib (for EPUB) or pymupdf (for PDF). Chapters are natural boundaries that preserve narrative flow.
Token-aware subdivision: If a chapter exceeds our target chunk size (we use 3,000 tokens to leave room for output and overlapping context), we split it further by paragraphs or sentences.
Context overlap: Each chunk includes a small amount of preceding and following text (we use 200 tokens each) to give Claude local context.

To count tokens accurately, we use the Anthropic Python SDK's count_tokens method. It's the same tokenizer Claude uses, so our chunk sizes are precise.

The Chunking Code

Here's a simplified version of our chunking logic:

from anthropic import Anthropic
import re

client = Anthropic(api_key="YOUR_API_KEY")

def count_tokens(text: str) -> int:
    return client.count_tokens(text)

def split_into_sentences(text: str) -> list[str]:
    # Basic sentence splitter (we use spaCy for production)
    return re.split(r'(?<=[.!?]) +', text)

def chunk_chapter(chapter_text: str, target_tokens: int = 3000, overlap_tokens: int = 200) -> list[str]:
    sentences = split_into_sentences(chapter_text)
    chunks = []
    current_chunk = []
    current_tokens = 0

    for sentence in sentences:
        sentence_tokens = count_tokens(sentence)
        if current_tokens + sentence_tokens > target_tokens and current_chunk:
            # Finalize current chunk
            chunk_text = " ".join(current_chunk)
            chunks.append(chunk_text)
            # Start new chunk with overlap from previous chunk
            overlap_text = " ".join(current_chunk[-overlap_units:])
            current_chunk = [overlap_text, sentence]
            current_tokens = count_tokens(overlap_text) + sentence_tokens
        else:
            current_chunk.append(sentence)
            current_tokens += sentence_tokens

    if current_chunk:
        chunks.append(" ".join(current_chunk))

    return chunks

In production, we use spaCy for sentence splitting and handle more edge cases (like dialogue, abbreviations), but the idea is the same: accumulate sentences until you hit the target token count, then start a new chunk with a small overlap from the previous chunk's ending.

Maintaining Context Across Chunks

Overlap alone isn't enough for long books. We also pass a running summary of previous chunks as part of the system prompt for every translation call. This summary is generated by Claude itself after each chunk is translated:

summary_prompt = f"""
Summarize the following translated text in 3-4 sentences, focusing on key plot points, character names, and terminology used. 
This summary will be used as context for translating the next chunk.

Translated text:
{translated_chunk}
"""

summary_response = client.messages.create(
    model="claude-3-haiku-20240307",  # cheaper model for summaries
    max_tokens=200,
    messages=[{"role": "user", "content": summary_prompt}]
)
summary = summary_response.content[0].text

We store the summary and include it in the system prompt for the next chunk:

context_prompt = f"""
You are translating a book. Here is a summary of the previous section:
{summary}

Translate the following text from {source_lang} to {target_lang}. 
Maintain consistency with the summary above. Preserve the author's style.
"""

This approach dramatically reduced inconsistencies like a character's name being translated differently in chapter 7 than in chapter 2.

Handling API Rate Limits and Failures

Translating 50+ chunks sequentially takes time and is prone to rate limits. We use asyncio and httpx to make concurrent requests (with a semaphore to limit concurrency to 4–5). Each request has retry logic with exponential backoff:

import asyncio
from tenacity import retry, stop_after_attempt, wait_exponential

@retry(stop=stop_after_attempt(5), wait=wait_exponential(multiplier=1, min=2, max=60))
async def translate_chunk(chunk: str, context: str, source_lang: str, target_lang: str) -> str:
    prompt = build_prompt(chunk, context, source_lang, target_lang)
    response = await client.messages.create(
        model="claude-3-sonnet-20240229",
        max_tokens=4096,
        messages=[{"role": "user", "content": prompt}]
    )
    return response.content[0].text

async def translate_book(chunks: list[str], summaries: list[str]):
    semaphore = asyncio.Semaphore(4)
    async def bounded_translate(chunk, summary):
        async with semaphore:
            return await translate_chunk(chunk, summary, source_lang, target_lang)

    tasks = [bounded_translate(chunk, summary) for chunk, summary in zip(chunks, summaries)]
    return await asyncio.gather(*tasks)

We use tenacity for retries and asyncio.Semaphore to stay under Anthropic's rate limits. With 4 concurrent requests and chunks of 3,000 tokens, a 300-page book takes about 5–8 minutes to translate, and costs around $3–4 using Claude 3 Sonnet.

Results and Lessons Learned

After implementing this chunking and context strategy, our translation quality improved significantly. We still run into occasional issues:

Cultural idioms: Some phrases need manual review.
Ambiguous pronouns: Even with summaries, very long books can confuse Claude if a character reappears after many chapters.
Cost: While $3–4 per book is acceptable for a paid service, we're exploring cheaper models like Claude Haiku for less critical sections.

The biggest lesson: don't fight the token limit; design around it. Accurate token counting, natural boundaries (chapters, paragraphs), and explicit context passing are far more effective than trying to squeeze everything into one massive prompt.

What's Next

We're currently experimenting with a two-pass approach: first, translate the entire book with a cheaper model to get a rough draft, then use Claude Opus to polish specific sections flagged as problematic. We also plan to add a custom glossary feature so users can define translations for specific names and terms.

If you're building something similar, start with the Anthropic token counting API, respect sentence boundaries, and always provide context. The alternative is a mess of broken sentences and angry users.

Open question for the community: How do you handle translation of poetry or highly stylized prose where context overlap may not capture the rhythm and meter? We'd love to hear your approaches.

Parsing and Rebuilding EPUB Files in Python: Lessons Learned

龚旭东 — Wed, 26 Aug 2026 03:01:45 +0000

How we built a robust EPUB translation pipeline by ditching EbookLib for zipfile and lxml

At LectuLibre, we translate books using LLMs. Our input is EPUB files, our output should be a translated EPUB that looks like the original, just in another language. That means we need to parse the EPUB, extract the text, send it to Claude or DeepSeek, get back the translation, and then rebuild the EPUB with the translated text while preserving all formatting, images, styles, and navigation.

This sounds straightforward until you realize that EPUB is a messy format: it's a ZIP archive containing XHTML files, CSS, images, fonts, and an XML navigation document. And the XHTML inside is often not well-formed XML — publishers use all sorts of quirks. We learned a lot while building this pipeline. Here are the lessons.

The naive approach: EbookLib

We started with EbookLib, a popular Python library for reading and writing EPUB files. It abstracts away the ZIP and gives you objects for each item.

import ebooklib
from ebooklib import epub

book = epub.read_epub('book.epub')
for item in book.get_items():
    if item.get_type() == ebooklib.ITEM_DOCUMENT:
        print(item.get_name())

Extracting text was easy enough. We could get the HTML content of each chapter and use BeautifulSoup or lxml to extract paragraphs. But when it came time to write the translated book back out, EbookLib started to fight us.

The main problem: EbookLib rebuilds the EPUB from its internal model, and that model doesn't capture every nuance. We lost CSS links, some metadata, and the exact ordering of the spine. We tried to manually set the spine, add CSS items, and copy over metadata, but it was a game of whack-a-mole. We'd fix one book and break another.

After a week of frustration, we realized we didn't need to rebuild the EPUB from scratch. We only needed to replace the text inside the existing HTML files. The rest of the EPUB (CSS, images, fonts, container.xml, etc.) could stay exactly as it was. So we went lower-level.

Treating EPUB as a ZIP archive

An EPUB is just a ZIP file with a specific structure. The META-INF/container.xml points to the OPF file, which lists all the content documents. But we don't even need to parse the OPF if we simply want to replace text in all HTML files. We can open the EPUB with Python's built-in zipfile, iterate over the entries, find the .xhtml or .html files (and sometimes .htm), and replace their contents.

import zipfile
from lxml import etree

def translate_epub(input_path, output_path):
    with zipfile.ZipFile(input_path, 'r') as zin:
        with zipfile.ZipFile(output_path, 'w') as zout:
            for item in zin.infolist():
                data = zin.read(item.filename)
                if item.filename.endswith(('.xhtml', '.html', '.htm')):
                    translated_html = translate_html(data.decode('utf-8'))
                    data = translated_html.encode('utf-8')
                zout.writestr(item, data)

This approach gives us full control. We don't alter any other part of the EPUB, so all the original assets remain untouched. But we need to be careful: some HTML files may not be UTF-8 encoded (they might have a different encoding declared in the XML prolog). We'll handle that later.

Translating HTML without breaking structure

Now the core challenge: given an HTML string, how do we replace the text with a translated version while keeping all the inline formatting (spans, emphasis, links, etc.)?

Naively, you might do a simple string replacement: html.replace(original_text, translated_text). That fails when the text is split across multiple tags, or when the translation length changes, or when there are HTML entities. A better way is to parse the HTML into a DOM tree, extract the text from each block-level element, translate each block independently, then replace the text content of those blocks.

We use lxml with its HTML parser because it can handle malformed HTML gracefully (unlike the stricter XML parser).

from lxml import etree

def extract_paragraphs(html_content):
    parser = etree.HTMLParser()
    tree = etree.fromstring(html_content, parser)
    blocks = tree.xpath('//p | //h1 | //h2 | //h3 | //h4 | //h5 | //h6 | //li | //blockquote')
    result = []
    for block in blocks:
        # Get the full text of the block, including text from child elements
        text = block.text_content().strip()
        if text:
            result.append(text)
    return result

We collect all block-level texts, send them as a batch to the translation API (to preserve context), get back a list of translated strings, and then map them back to the original blocks.

def replace_block_texts(html_content, translations):
    parser = etree.HTMLParser()
    tree = etree.fromstring(html_content, parser)
    blocks = tree.xpath('//p | //h1 | //h2 | //h3 | //h4 | //h5 | //h6 | //li | //blockquote')
    # Assume translations is a list of strings in the same order as blocks
    for block, new_text in zip(blocks, translations):
        # Remove all child elements but keep the block element itself
        for child in block:
            block.remove(child)
        # Set the text of the block (lxml will escape special characters automatically)
        block.text = new_text
    return etree.tostring(tree, encoding='unicode', method='html')

This approach preserves the element's attributes (class, id, style) but discards any inline formatting within the block. That means if the original paragraph had <em> for emphasis or <a> for links, those are lost. For our first version, that was acceptable because we focus on the text, but we later improved it by splitting translation at the phrase level while keeping inline tags. That's a topic for another article.

Encoding and special characters

EPUB files may be encoded in UTF-8 or UTF-16, or even declared as ISO-8859-1. We initially read everything as UTF-8, which caused crashes on some books. The solution is to respect the encoding declaration in the HTML file.

We detect the encoding by looking at the first line of the file (the <?xml ... encoding="..."?> declaration) or by using chardet if no declaration is present. Then we decode accordingly, and when writing back, we always encode as UTF-8 and update the XML declaration to match.

import re

def decode_html(data: bytes) -> str:
    # Try to find encoding in the first 200 bytes
    head = data[:200].decode('ascii', errors='ignore')
    match = re.search(r'encoding=["\']([^"\']+)["\']', head)
    if match:
        encoding = match.group(1)
    else:
        # Fallback to chardet
        import chardet
        detected = chardet.detect(data)
        encoding = detected['encoding'] or 'utf-8'
    return data.decode(encoding, errors='replace')

When writing back, we ensure the encoding attribute in the XML prolog is set to utf-8.

Performance: Memory and speed

Our first iteration loaded the entire EPUB into memory, processed all files, and wrote the output. For small books (under 5MB), this was fine. But we soon got a user upload of a 300MB EPUB (lots of images and a huge text). Our server (a modest VPS with 4GB RAM) crashed with an out-of-memory error.

We fixed this by streaming: read one ZIP entry at a time, process it, and write it immediately to the output ZIP. Since we aren't doing any cross-file operations, this works perfectly. The memory usage dropped from over 1GB to about 200MB for that large book, and processing time stayed roughly the same (around 1.2 seconds for a typical 500KB novel, 45 seconds for the 300MB monster) because the bottleneck is the translation API, not file I/O.

Validation: epubcheck and internal links

After rebuilding, we run epubcheck (a Java-based validator) on the output to ensure it's a valid EPUB. Our initial naive replacement with EbookLib had a pass rate of around 60%. With the ZIP-and-lxml approach, we now pass 98% of the time. The remaining 2% are usually books with unusual structures (fixed-layout, JavaScript, or malformed OPF files) that we flag for manual review.

We also added a check for internal links: some XHTML files link to other files via href. Since we are not changing file names or locations, these links remain valid. But if we ever need to reorder or rename files, we'd have to update those links.

Lessons learned

EbookLib is good for reading, not for writing. If you only need to extract data, use it. For rebuilding, go lower-level.
Use lxml with the HTML parser. It handles the messy real-world HTML found in EPUBs.
Treat the EPUB as a ZIP and only modify what you need. Don't try to reconstruct the whole package; just swap the text content.
Respect encodings. EPUBs can be in various charsets; don't assume UTF-8.
Stream when possible. Don't hold the whole book in memory.
Validate with epubcheck. It catches many subtle issues.

We're still improving the pipeline. The next challenge is handling inline formatting without losing emphasis or links. We're experimenting with translating at the sentence level and mapping back to the original inline tags. If you have experience with that, we'd love to hear from you.

Until next time, happy coding!

Parsing and Rebuilding EPUB Files in Python: Lessons Learned from Building an AI Translation Service

龚旭东 — Sat, 22 Aug 2026 03:01:43 +0000

How we handle complex EPUB structures, preserve metadata, and rebuild valid files after AI translation.

At LectuLibre, we translate books using LLMs. Users upload an EPUB, our backend extracts the text, sends it to Claude or DeepSeek, and then we need to put the translated text back into the same EPUB—preserving formatting, images, styles, and metadata. That last part turned out to be the hardest.

In this post, I'll walk you through the challenges we faced with parsing and rebuilding EPUB files in Python, the libraries we chose, and the trade-offs we made. If you're building any kind of ebook processing pipeline, these lessons should save you time.

The Problem: EPUB Is More Than Just HTML

EPUB files are ZIP archives containing XHTML/HTML documents, CSS, images, fonts, and metadata files like container.xml and content.opf. To translate a book, we need to:

Extract the textual content from each chapter.
Keep track of where that text came from so we can put translations back in the same place.
Send the text to an LLM (in chunks, because books are long).
Rebuild the EPUB with translated text while preserving all other resources.

Initially, we thought we could just unzip, parse HTML with BeautifulSoup, translate, and zip it back. But we quickly ran into issues:

EPUB structure varies: Some books use EPUB2 with NCX navigation, others EPUB3 with nav documents. Metadata is in different places.
Namespaces matter: XHTML files often use xmlns="http://www.w3.org/1999/xhtml", and if you don't handle namespaces correctly, BeautifulSoup can mangle the markup.
Internal references: The OPF manifest lists all files with IDs and properties. If you rename or add files, you must update the manifest.
Encoding and special characters: Books in different languages use various encodings, and LLM outputs may contain smart quotes or special characters that need to be handled correctly.

Our Approach: ebooklib + BeautifulSoup

We evaluated several Python libraries:

zipfile + manual XML parsing: too low-level, easy to break things.
lxml directly: powerful but requires deep understanding of EPUB spec.
ebooklib: a dedicated EPUB library that handles reading/writing, metadata, and manifest management.
epub-parser and others: less maintained or incomplete.

We ended up using ebooklib for the EPUB container operations (reading, writing, navigating items) and BeautifulSoup for fine-grained HTML manipulation inside chapters. This combination gave us the best balance of simplicity and control.

Implementation Details

Reading an EPUB and Extracting Text

Here's a simplified version of how we load an EPUB and iterate over its document items:

from ebooklib import epub
from bs4 import BeautifulSoup

def extract_chapters(epub_path):
    book = epub.read_epub(epub_path)
    chapters = []

    for item in book.get_items():
        if item.get_type() == ebooklib.ITEM_DOCUMENT:
            # item is an XHTML/HTML document
            soup = BeautifulSoup(item.get_content(), 'html.parser')
            # Extract text from all paragraphs
            text = '\n\n'.join(p.get_text() for p in soup.find_all('p'))
            chapters.append({
                'id': item.get_id(),
                'file_name': item.get_name(),
                'content': text,
                'soup': soup  # keep soup for later modification
            })
    return book, chapters

We store the soup object because we'll need to modify it during rebuilding.

Translating and Rebuilding

After sending the extracted text to the LLM and receiving translations, we need to put the translated text back into the original HTML structure. Our approach splits the original text into paragraphs and replaces each paragraph's text with the translated version, preserving tags and attributes.

from bs4 import BeautifulSoup

def replace_text_in_soup(soup, original_paragraphs, translated_paragraphs):
    # Find all <p> tags
    p_tags = soup.find_all('p')
    if len(p_tags) != len(translated_paragraphs):
        raise ValueError("Mismatch in paragraph count")

    for p_tag, translated_text in zip(p_tags, translated_paragraphs):
        # Clear existing content and set new text
        p_tag.clear()
        p_tag.append(translated_text)

    return soup

Then we write the modified soup back to the ebooklib item and save the EPUB:

import ebooklib
from ebooklib import epub

def rebuild_epub(book, chapters_with_new_text):
    for chapter in chapters_with_new_text:
        item = book.get_item_with_id(chapter['id'])
        # Convert soup back to bytes
        new_content = str(chapter['soup']).encode('utf-8')
        item.set_content(new_content)

    # Write to a new EPUB file
    epub.write_epub('translated_book.epub', book)

That's the core loop. But real-world books threw many curveballs.

Lessons Learned: Handling Edge Cases

Namespaces

EPUB XHTML files often declare the XHTML namespace. If you use BeautifulSoup's html.parser, it may treat tags as HTML and ignore namespaces, which can lead to issues when serializing. We switched to lxml parser with BeautifulSoup:

soup = BeautifulSoup(content, 'lxml')

This parser is namespace-aware and better preserves the original markup. However, it's slightly slower. For a 500-page book, parsing all chapters with lxml took about 1.2 seconds vs 0.8 seconds with html.parser—a negligible difference compared to the LLM translation time.

Handling Non-Paragraph Content

Not all text is in <p> tags. Headings, list items, tables, and footnotes need translation too. Our initial paragraph-only extraction missed a lot. We expanded extraction to include all text-bearing elements:

def extract_text_elements(soup):
    elements = []
    for tag in soup.find_all(['p', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'li', 'blockquote', 'td', 'th']):
        if tag.get_text(strip=True):
            elements.append(tag)
    return elements

Then we translate each element's text and replace it individually. This works but can disrupt inline formatting like bold or italic within paragraphs. To handle that, we need to translate the entire HTML subtree, not just the text—but that's a much harder problem. For our MVP, we translate per element and accept that some inline formatting may be lost.

CSS and Layout

EPUBs contain CSS files that control layout. We don't modify these, so the translated book looks similar to the original, assuming the structure is intact. However, some languages expand or contract text significantly. For example, German translations are often 30% longer than English. This can cause pagination issues, but that's beyond our current scope.

Cover and Metadata

We preserve the cover image and metadata by using ebooklib's built-in methods. We read the original metadata and write it back unchanged:

# Preserve metadata
original_metadata = book.get_metadata('DC', 'title')
# ... after rebuild, set same metadata
book.set_metadata('DC', 'title', original_metadata[0][0])

Ebooklib handles the OPF manifest automatically when we use its methods, which saved us from manually editing XML.

Malformed EPUBs

Some user uploads are not valid EPUBs; they might be mislabeled or corrupted. We added validation:

import ebooklib
from ebooklib import epub

def validate_epub(path):
    try:
        book = epub.read_epub(path)
        if not book.get_items():
            return False
        # Check for required files
        if not book.get_item_with_id('nav') and not book.get_item_with_id('ncx'):
            # EPUB2 uses NCX, EPUB3 uses nav
            return False
        return True
    except Exception:
        return False

If validation fails, we reject the file with a clear error message.

Performance Numbers

On our VPS (2 vCPU, 4GB RAM), parsing a typical 300-page EPUB with 20 chapters takes about 0.5 seconds using lxml. Extracting all text elements and sending them to the LLM (in chunks) dominates the total processing time—often minutes. Rebuilding the EPUB is fast: under 1 second.

Memory usage is modest because we process chapter by chapter, not the whole book at once. For very large books (1000+ pages), we still keep the entire soup tree in memory per chapter, which can spike to ~50MB per chapter, but it's manageable.

What We'd Do Differently

Use lxml directly for HTML parsing: BeautifulSoup's overhead isn't huge, but if we need finer control over inline elements, we might switch to direct lxml.etree with custom translation logic that preserves markup.
Translate at the block level with inline markup awareness: We'd like to translate each block element while keeping inline tags like <em>, <strong>, and <a> intact. This requires more sophisticated text extraction and reinsertion—possibly using placeholders.
Handle EPUB3 fixed-layout books: These are common for children's books and comics, and they use absolute positioning. Our current approach does not support them.

Conclusion

Parsing and rebuilding EPUBs in Python is doable with the right libraries, but the devil is in the details. ebooklib handles the EPUB container well, and BeautifulSoup with lxml gives you the HTML manipulation power you need. The main challenges are preserving structure, handling namespaces, and dealing with the variety of real-world EPUBs.

If you're building ebook processing tools, my advice is to start with a simple pipeline like the one above, then iterate as you encounter edge cases. Validate early and often.

Open question for the community: How do you handle inline formatting (bold/italic/links) when translating HTML content? We've tried placeholder approaches but are curious about other solutions.

LectuLibre is an AI-powered book translation service. We're constantly improving our EPUB handling to support more formats and languages. If you have experience with ebook pipelines, we'd love to hear from you in the comments.

Parsing and Rebuilding EPUB Files in Python: Lessons Learned from LectuLibre

龚旭东 — Wed, 12 Aug 2026 03:01:56 +0000

How we tackled EPUB parsing and rebuilding for AI book translation, with code examples and hard-won lessons.

At LectuLibre (https://lectulibre.com), we translate books using AI. Users upload EPUB or PDF files, and our system translates the text inside while preserving the original formatting. Sounds straightforward, right? Well, we quickly learned that handling EPUB files in production is anything but trivial. In this article, I'll share the real-world challenges we faced parsing and rebuilding EPUBs with Python, the libraries we chose, the code we wrote, and the lessons we learned along the way.

The Problem: Translate Books, Not Just Text

An EPUB is essentially a ZIP archive containing HTML, CSS, images, and a manifest file (content.opf). To translate a book, we needed to:

Parse the EPUB's structure and extract translatable content.
Send that content to our LLM pipeline (Claude, DeepSeek) while preserving context.
Rebuild a new EPUB with the translated content, keeping all formatting, images, and metadata intact.

Simple in theory, but real-world EPUBs are messy. Some have broken manifests, missing required files (mimetype, META-INF/container.xml), or text encoded in obscure charsets. Others are huge, with hundreds of images that we don't want to load into memory. Our translation pipeline needed to be fast, memory-efficient, and resilient to broken input.

Our Approach: ebooklib + lxml + Careful Validation

We evaluated several options:

EbookLib: A widely-used Python library for reading and writing EPUBs. It abstracts the OPF structure and provides convenient methods.
manual zipfile processing: Full control but requires reimplementing all the EPUB logic.
Calibre: A powerful suite, but its CLI and heavy dependencies didn't fit our lightweight VPS deployment.

We settled on EbookLib for most reading tasks, lxml for high-performance HTML parsing, and Python's built-in zipfile for the final rebuilding step where we needed fine-grained control. This combination gave us the right balance of development speed and runtime performance.

Implementation Details (with Real Code)

1. Parsing and Extracting Translatable Text

First, we read the EPUB using EbookLib and iterate over the spine (the linear reading order). We only process items of type ITEM_DOCUMENT (HTML files).

import ebooklib
from ebooklib import epub

book = epub.read_epub('path/to/book.epub')

translatable_segments = []

for item in book.get_items_of_type(ebooklib.ITEM_DOCUMENT):
    content = item.get_body_content().decode('utf-8')  # assuming utf-8
    # But wait – not all EPUBs use utf-8!

Lesson 1: Encoding Hell

Many EPUBs (especially older ones) encode their HTML in Windows-1252 or ISO-8859-1. EbookLib's get_body_content() often returns bytes, and decoding as utf-8 raises UnicodeDecodeError. We added charset detection using chardet:

import chardet

raw_bytes = item.get_content()
encoding = chardet.detect(raw_bytes)['encoding'] or 'utf-8'
text = raw_bytes.decode(encoding, errors='replace')

Once decoded, we parse the HTML with lxml to extract plain text while preserving the structure for later reassembly.

from lxml import etree

parser = etree.HTMLParser()
tree = etree.fromstring(text.encode('utf-8'), parser=parser)

# Extract text with XPath, but keep track of element boundaries
for elem in tree.iter():
    if elem.text:
        translatable_segments.append({
            'id': item.get_id(),
            'tag': elem.tag,
            'text': elem.text.strip(),
            # storing path for later replacement
            'path': tree.getpath(elem)
        })

For LLM translation, we concatenate segments into manageable chunks, preserving paragraph boundaries to give the model context.

2. Rebuilding the EPUB with Translated Text

After translation, we need to replace the original text in the HTML documents and reassemble the EPUB. EbookLib provides epub.write_epub() but we found it had a few quirks:

It sometimes reorders the manifest, which can break navigation.
It doesn't handle custom namespaces or non-standard OPF entries well.
It might omit files that are referenced but not in the spine (like cover images).

We opted to manipulate the EPUB as a ZIP file directly for the rebuild step, while still using EbookLib's data structures to understand the original manifest.

import zipfile
import shutil
import os

def rebuild_epub(original_path, output_path, translated_segments):
    # Read original EPUB as ZIP to get all files
    with zipfile.ZipFile(original_path, 'r') as zin:
        with zipfile.ZipFile(output_path, 'w', zipfile.ZIP_DEFLATED) as zout:
            for item in zin.infolist():
                if item.filename.endswith('.html') or item.filename.endswith('.xhtml'):
                    html_content = zin.read(item.filename).decode('utf-8')
                    # Replace translated segments using lxml
                    tree = etree.fromstring(html_content.encode('utf-8'), parser=parser)
                    for seg in translated_segments:
                        if seg['id'] == item_to_id[item.filename]:
                            elem = tree.xpath(seg['path'])[0] if seg['path'] else None
                            if elem is not None and elem.text:
                                elem.text = seg['translated_text']
                    new_content = etree.tostring(tree, encoding='unicode')
                    zout.writestr(item, new_content.encode('utf-8'))
                else:
                    # Copy other files (images, css, etc.) as-is
                    zout.writestr(item, zin.read(item.filename))

Lesson 2: Preserve the Mimetype File

The EPUB specification requires that the first file in the ZIP be the mimetype file, stored without compression. EbookLib does this automatically when writing, but when using zipfile manually, you must add it first with stored compression. We forgot this initially and our books were rejected by strict readers. Fix:

zout.writestr('mimetype', 'application/epub+zip', zipfile.ZIP_STORED)

3. Validation and Error Handling

Users upload terribly broken EPUBs. We learned to validate early:

def validate_epub(file_path):
    with zipfile.ZipFile(file_path, 'r') as zf:
        if 'META-INF/container.xml' not in zf.namelist():
            raise ValueError("Missing container.xml")
        if 'mimetype' not in zf.namelist():
            raise ValueError("Missing mimetype file")
    # Also check content.opf exists and parses correctly

We also check that the OPF file references all necessary resources. This catches many upload errors before they hit the expensive translation step.

Performance and Trade-offs

Memory: EbookLib loads the entire EPUB into memory. For a 50MB book with many images, this was problematic. We modified our pipeline to stream the ZIP and skip binary items during parsing, only extracting document content. This reduced peak memory from ~150MB to ~30MB per book.
Speed: Parsing a typical 500-page novel with EbookLib and lxml takes ~1.5 seconds. Translation is the bottleneck (LLM API calls), so this is acceptable. We process books sequentially on a single VPS worker; concurrency comes from multiple workers.
Failure rate: Thanks to validation and robust encoding handling, we now successfully process ~97% of user-uploaded EPUBs. The remaining 3% are often DRM-protected or hopelessly malformed.

Honest Reflection: What We'd Do Differently

EbookLib is fine, not great. Its development has slowed, and some advanced EPUB3 features aren't supported (e.g., media overlays). For a future rewrite, we might use a lightweight custom parser on top of lxml and zipfile.
HTML manipulation is fragile. We originally tried using regular expressions for text replacement – never do that. lxml saved us countless hours.
Always test output with EPUBCheck. We integrated https://github.com/w3c/epubcheck into our CI to validate every translated EPUB before delivery.

Practical Takeaways for Your EPUB Processing Project

Validate early, validate often. Check required files and OPF integrity before processing.
Never trust the encoding. Use chardet or a similar library to decode text safely.
Choose the right rewrite strategy. If your translation needs are simple, EbookLib's write_epub might suffice. For complex rebuilds, consider manual ZIP manipulation.
Handle the mimetype file correctly. It must be the first entry, stored without compression.
Test with broken books. Grab a sample from Project Gutenberg or feed your parser a corrupted file to see where it fails.

At LectuLibre, this EPUB processing pipeline now handles hundreds of translations per week, allowing readers around the world to enjoy books in their native language. The journey was bumpy, but the lessons we learned have made our service robust and our engineering team wiser.

What libraries or techniques have you used for EPUB processing in Python? We'd love to hear about alternatives to EbookLib or other war stories in the comments!