Translating 300-Page Books with Claude: How We Handle Token Limits Without Losing Context

A practical guide to chunking, token counting, and maintaining narrative flow when using the Anthropic API for long-form translation.

At LectuLibre, we let users upload an EPUB or PDF and get a full book translation powered by Claude. A 300-page novel might sound manageable until you realize it's around 100,000 tokens. Claude's context window can hold it, but the output limit, cost, and latency make sending the whole book at once a non-starter. Here's the chunking strategy we settled on after burning through our API quota and a few failed attempts.

The Problem

A typical 300-page book contains roughly 80,000–100,000 words. For Claude, that's about 120,000 tokens (using a rough 1.3 tokens per word). While Claude 3 Opus has a 200K token context window, the output limit is much lower (4096 tokens for Opus, 8192 for Sonnet). Translating an entire book in one API call would mean:

• Cost: Input tokens are expensive. At $15 per million input tokens for Opus, a single book translation would cost around $1.80 just for input — but that's the cheapest part. Output tokens cost $75 per million. If the translation is roughly the same length as the original, you'd generate another 120K tokens, adding $9. That's over $10 per book, not including retries.
• Latency: Generating 120K tokens sequentially takes a long time. Even at 50 tokens per second, that's 40 minutes per book.
• Reliability: One long request is more likely to hit a timeout, rate limit, or network error. If it fails after 100K tokens, you start over.

Clearly, we needed to break the book into smaller chunks and translate each independently. But naive chunking (e.g., splitting by a fixed number of characters) created broken sentences, lost narrative context, and inconsistent terminology across chapters.

Our First Attempt: Fixed 4,000-Token Chunks

We started with the simplest approach: split the book into chunks of 4,000 tokens using a rough character-to-token estimate (we assumed 4 characters per token). We then sent each chunk to Claude with a system prompt like:

system_prompt = """
You are a professional literary translator. Translate the following text from {source_lang} to {target_lang}. 
Preserve the author's style, tone, and cultural nuances. Do not add or omit content.
"""

This failed in a few predictable ways:

1. Mid-sentence splits: Chunks often cut a paragraph or sentence in half. Claude translated each piece literally, producing awkward phrases at chunk boundaries.
2. Inconsistent terminology: Character names, place names, and recurring phrases were translated differently across chunks.
3. Lost context: Without the previous chapter's summary, Claude sometimes misinterpreted pronouns or ambiguous references.
4. Token miscounts: Our 4-char-per-token estimate was wrong, leading to chunks that exceeded the output limit or were too small, increasing API calls.

We burned through a lot of credits fixing these issues manually. Then we moved to a token-aware, context-preserving chunker.

A Better Chunking Strategy

We now split books using three levels:

1. Structural units: We parse the EPUB/PDF into chapters or sections using libraries like ebooklib (for EPUB) or pymupdf (for PDF). Chapters are natural boundaries that preserve narrative flow.
2. Token-aware subdivision: If a chapter exceeds our target chunk size (we use 3,000 tokens to leave room for output and overlapping context), we split it further by paragraphs or sentences.
3. Context overlap: Each chunk includes a small amount of preceding and following text (we use 200 tokens each) to give Claude local context.

To count tokens accurately, we use the Anthropic Python SDK's count_tokens method. It's the same tokenizer Claude uses, so our chunk sizes are precise.

The Chunking Code

Here's a simplified version of our chunking logic:

from anthropic import Anthropic
import re

client = Anthropic(api_key="YOUR_API_KEY")

def count_tokens(text: str) -> int:
    return client.count_tokens(text)

def split_into_sentences(text: str) -> list[str]:
    # Basic sentence splitter (we use spaCy for production)
    return re.split(r'(?<=[.!?]) +', text)

def chunk_chapter(chapter_text: str, target_tokens: int = 3000, overlap_tokens: int = 200) -> list[str]:
    sentences = split_into_sentences(chapter_text)
    chunks = []
    current_chunk = []
    current_tokens = 0

    for sentence in sentences:
        sentence_tokens = count_tokens(sentence)
        if current_tokens + sentence_tokens > target_tokens and current_chunk:
            # Finalize current chunk
            chunk_text = " ".join(current_chunk)
            chunks.append(chunk_text)
            # Start new chunk with overlap from previous chunk
            overlap_text = " ".join(current_chunk[-overlap_units:])
            current_chunk = [overlap_text, sentence]
            current_tokens = count_tokens(overlap_text) + sentence_tokens
        else:
            current_chunk.append(sentence)
            current_tokens += sentence_tokens

    if current_chunk:
        chunks.append(" ".join(current_chunk))

    return chunks

In production, we use spaCy for sentence splitting and handle more edge cases (like dialogue, abbreviations), but the idea is the same: accumulate sentences until you hit the target token count, then start a new chunk with a small overlap from the previous chunk's ending.

Maintaining Context Across Chunks

Overlap alone isn't enough for long books. We also pass a running summary of previous chunks as part of the system prompt for every translation call. This summary is generated by Claude itself after each chunk is translated:

summary_prompt = f"""
Summarize the following translated text in 3-4 sentences, focusing on key plot points, character names, and terminology used. 
This summary will be used as context for translating the next chunk.

Translated text:
{translated_chunk}
"""

summary_response = client.messages.create(
    model="claude-3-haiku-20240307",  # cheaper model for summaries
    max_tokens=200,
    messages=[{"role": "user", "content": summary_prompt}]
)
summary = summary_response.content[0].text

We store the summary and include it in the system prompt for the next chunk:

context_prompt = f"""
You are translating a book. Here is a summary of the previous section:
{summary}

Translate the following text from {source_lang} to {target_lang}. 
Maintain consistency with the summary above. Preserve the author's style.
"""

This approach dramatically reduced inconsistencies like a character's name being translated differently in chapter 7 than in chapter 2.

Handling API Rate Limits and Failures

Translating 50+ chunks sequentially takes time and is prone to rate limits. We use asyncio and httpx to make concurrent requests (with a semaphore to limit concurrency to 4–5). Each request has retry logic with exponential backoff:

import asyncio
from tenacity import retry, stop_after_attempt, wait_exponential

@retry(stop=stop_after_attempt(5), wait=wait_exponential(multiplier=1, min=2, max=60))
async def translate_chunk(chunk: str, context: str, source_lang: str, target_lang: str) -> str:
    prompt = build_prompt(chunk, context, source_lang, target_lang)
    response = await client.messages.create(
        model="claude-3-sonnet-20240229",
        max_tokens=4096,
        messages=[{"role": "user", "content": prompt}]
    )
    return response.content[0].text

async def translate_book(chunks: list[str], summaries: list[str]):
    semaphore = asyncio.Semaphore(4)
    async def bounded_translate(chunk, summary):
        async with semaphore:
            return await translate_chunk(chunk, summary, source_lang, target_lang)

    tasks = [bounded_translate(chunk, summary) for chunk, summary in zip(chunks, summaries)]
    return await asyncio.gather(*tasks)

We use tenacity for retries and asyncio.Semaphore to stay under Anthropic's rate limits. With 4 concurrent requests and chunks of 3,000 tokens, a 300-page book takes about 5–8 minutes to translate, and costs around $3–4 using Claude 3 Sonnet.

Results and Lessons Learned

After implementing this chunking and context strategy, our translation quality improved significantly. We still run into occasional issues:

• Cultural idioms: Some phrases need manual review.
• Ambiguous pronouns: Even with summaries, very long books can confuse Claude if a character reappears after many chapters.
• Cost: While $3–4 per book is acceptable for a paid service, we're exploring cheaper models like Claude Haiku for less critical sections.

The biggest lesson: don't fight the token limit; design around it. Accurate token counting, natural boundaries (chapters, paragraphs), and explicit context passing are far more effective than trying to squeeze everything into one massive prompt.

What's Next

We're currently experimenting with a two-pass approach: first, translate the entire book with a cheaper model to get a rough draft, then use Claude Opus to polish specific sections flagged as problematic. We also plan to add a custom glossary feature so users can define translations for specific names and terms.

If you're building something similar, start with the Anthropic token counting API, respect sentence boundaries, and always provide context. The alternative is a mess of broken sentences and angry users.

Open question for the community: How do you handle translation of poetry or highly stylized prose where context overlap may not capture the rhythm and meter? We'd love to hear your approaches.