DEV Community: 龚旭东

How We Translate Entire Books with LLMs Without Losing Context

龚旭东 — Thu, 18 Jun 2026 23:19:00 +0000

Our chunking strategy that keeps chapters coherent, respects context windows, and handles multi-lingual books.

The problem: books don’t fit in a prompt

At LectuLibre, we translate entire books — novels, technical manuals, poetry — using large language models. It sounds simple: feed each paragraph to an LLM, concatenate results, done. But the moment we tried a 300‑page EPUB, chaos ensued. Chapters bled into each other, sentences were chopped mid‑word, and the translation of chapter 5 had no idea what happened in chapter 4.

LLMs have limited context windows. Even the massive 200K token window of Claude 3 can’t hold a whole 150K‑word book. And even if it could, the cost and latency would be absurd. We needed a way to split the book into manageable chunks while preserving enough context so that the translation remains coherent across thousands of pages.

Here’s how we designed a chunking pipeline that respects your wallet, the context window, and the book’s narrative flow.

Step 1: extract structure, not just text

Naively splitting by character count is a recipe for disaster. Instead, we first parse the document to understand its logical units: chapters, sections, headings. For EPUB, we use ebooklib; for PDF, pdfplumber. Both give us a stream of items (paragraphs, headings) that we then organize into a tree of chapters and sub‑sections.

import ebooklib
from ebooklib import epub

def get_chapters(epub_path):
    book = epub.read_epub(epub_path)
    chapters = []
    for item in book.get_items_of_type(ebooklib.ITEM_DOCUMENT):
        # Simplified: each document is a chapter
        content = item.get_content().decode('utf-8')
        chapters.append(content)
    return chapters

In practice, we use BeautifulSoup to extract <body> text and identify heading tags (<h1>–<h6>) to build a table of contents. This way, even if a chapter is 20,000 tokens, we keep it together as a single unit until later splitting.

Step 2: sentence‑aware splitting with token budgets

A chapter still needs to be broken down to fit the model’s context window. But we never split mid‑sentence. We use spaCy to tokenize the text into sentences, then greedily group them until we hit a token limit.

Why not simple character‑based splitting? Because sentences carry semantic boundaries. Breaking inside a sentence occasionally produces artefacts like “He walked to the sta‑” / “‑tion.” LLMs are forgiving but not that forgiving.

import spacy
from transformers import AutoTokenizer  # for accurate token count

nlp = spacy.load("en_core_web_sm")
tokenizer = AutoTokenizer.from_pretrained("claude-tokenizer")  # custom tokenizer for Claude

def sentence_split(text):
    doc = nlp(text)
    return [sent.text for sent in doc.sents]

def chunk_sentences(sentences, max_tokens=1800, overlap_sentences=5):
    chunks = []
    current_chunk = []
    current_token_count = 0

    for i, sent in enumerate(sentences):
        sent_tokens = len(tokenizer.encode(sent))
        if current_token_count + sent_tokens > max_tokens:
            # Store chunk with a sliding overlap
            chunks.append(current_chunk)
            # Overlap: take last `overlap_sentences` from the chunk just concluded
            current_chunk = sentences[i - overlap_sentences : i] if i - overlap_sentences > 0 else []
            current_token_count = sum(len(tokenizer.encode(s)) for s in current_chunk)
        current_chunk.append(sent)
        current_token_count += sent_tokens
    if current_chunk:
        chunks.append(current_chunk)
    return chunks

We set max_tokens to 1800, leaving room for the system prompt, context from previous chunks, and the model’s response. That’s for Claude Haiku, which has a 32K context window. For longer‑context models we’d scale up, but keeping chunks smaller also means faster, cheaper API calls.

Step 3: passing context across chunks

The real magic is what we do between chunks. A standalone translation of chunk #5 has no clue that the protagonist just entered a dark cave in chunk #4. Two techniques solved this:

Sliding window of previous sentences — we include the last 5–10 sentences from the preceding chunk directly in the prompt as “context left.”
A running summary — after translating a chunk, we ask the LLM to generate a one‑sentence summary of that chunk. This summary is accumulated and fed into every subsequent prompt, so the model remembers high‑level events.

def build_prompt(chunk, previous_context_sentences, summary_so_far):
    context_left = " ".join(previous_context_sentences)
    prompt = f"""You are translating a book. Here is a summary of the story so far:
    {summary_so_far}

    And the previous text (for immediate context):
    "{context_left}"

    Now translate the following text to Spanish, preserving tone and style:
    {chunk}"""
    return prompt

The summary is generated using a separate, cheap call (we use DeepSeek for summaries, even if the main translation uses Claude). This keeps the context token usage minimal while still giving long‑range coherence.

Why not just include the entire previous chunk? That doubles the token count per call. On a 200K‑word book, that adds up to hundreds of dollars. Summaries cut that cost by ~80% with negligible quality loss.

The translation loop then looks like this:

overall_summary = ""
previous_context = []
full_translation = []

for chapter_chunks in all_chunks_by_chapter:
    chapter_summary = ""
    for i, chunk in enumerate(chapter_chunks):
        prompt = build_prompt(
            " ".join(chunk),
            previous_context,
            chapter_summary + "\n" + overall_summary if i > 0 else ""
        )
        translated = call_llm(prompt)
        full_translation.append(translated)

        # Update context: keep last 5 sentences of the translated chunk as next context
        trans_sents = sentence_split(translated)
        previous_context = trans_sents[-5:]

        # Generate chunk summary asynchronously to save time
        chunk_summary = call_llm(f"Summarize this passage in one sentence: {chunk}")
        chapter_summary += chunk_summary + " "
    overall_summary += chapter_summary

We process chunks concurrently using asyncio and httpx to keep translation times reasonable.

Real‑world results and trade‑offs

Translating a 120K‑word Spanish novel (“El Quijote”) into English took about 4 minutes end‑to‑end with Claude 3 Haiku. Total API cost: $0.67. The translation was surprisingly fluid — chapters felt connected, and the occasional flashback or pronoun reference (“she” referring to a character introduced three pages earlier) was correctly resolved. Without the context pipeline, the same book would have been riddled with inconsistencies.

We experimented with other models: DeepSeek‑V3 gave similar quality at half the price but with higher latency, making it better for batch jobs where speed isn’t critical. GPT‑4 Turbo reproduced stylistic flourishes more naturally, but its 16K context window forced us to use even smaller chunks, which sometimes fragmented dialogue. Claude struck the best balance.

But it’s not perfect. Humor and idioms still occasionally fall flat because the summary can’t encapsulate a running joke. Code blocks and tables inside technical books need special handling — we’re working on a parser that detects them and wraps them in [CODE] markers so the LLM doesn’t try to translate variable names. And poetry, with its line breaks and meter, remains a challenge; we’re considering a dedicated poetry‑aware chunker.

The key takeaway

If you’re building long‑document translation using LLMs, invest in a pipeline that:

Respects document structure (chapters, paragraphs) before splitting.
Splits on sentences, and always leaves room for context.
Provides both immediate context (last few sentences) and global context (summaries) to each chunk.
Uses separate, cheap models for auxiliary tasks like summarization to keep costs down.

Our code is not open‑source yet, but we plan to release the core chunking library once we’ve battle‑tested it on more formats.

How do you handle context in LLM translations? We’re especially curious about handling highly technical books with equations, footnotes, and cross‑references. Drop your ideas in the comments — let’s figure this out together.

The Hidden Masterpieces of Japanese Crime Fiction: Novels Still Waiting for English Translation

龚旭东 — Thu, 18 Jun 2026 08:42:59 +0000

Discover the works of Seishi Yokomizo, Soji Shimada, and other giants of the genre that remain out of reach for English-only readers.

If you’re a fan of Japanese mystery novels in English, you’ve probably noticed a golden age of translation in recent years. Publishers like Pushkin Vertigo, Bento Books, and Locked Room International have brought us long-awaited classics by Seishi Yokomizo, Soji Shimada, and other masters of Japanese crime fiction. But for every Honjin Murders or Tokyo Zodiac Murders that finally lands on our shelves, dozens of equally brilliant works remain hidden behind the language barrier.

I still remember the frustration of finishing Yokomizo’s The Inugami Curse and discovering that only a handful of his 77 Kosuke Kindaichi novels had been translated. The same goes for Shimada’s intricate locked-room puzzles—the English-speaking world has tasted only a fraction of his genius. This article is for readers who want to go deeper, who refuse to let language be a wall between them and the stories they crave. We’ll explore the untranslated jewels of Japanese mystery, and I’ll show you practical ways to access them now.

Why Japanese Crime Fiction Captivates the World

Japanese mystery novels aren’t just puzzles with a Tokyo backdrop. They blend meticulous plotting with a literary sensibility that often feels more adult, more psychologically nuanced than their Western counterparts. Classic honkaku (orthodox) mysteries by authors like Yokomizo and Shimada treat the whodunit as a rigorous intellectual game, where every clue is laid out fairly for the reader. Yet the atmosphere is soaked in history, folklore, and post-war tension—elements that give these stories a depth beyond the puzzle.

Meanwhile, the social crime novels of Seicho Matsumoto and the psychological thrillers of Natsuo Kirino have already proven their international appeal. But beneath the well-translated surface lies a vast reservoir of mid-century and contemporary works that Anglophone readers have never seen. Many of these are considered landmarks in Japan, yet they remain invisible abroad simply because no publisher has taken a chance on them.

The Translation Gap: A Numbers Game

Let’s talk numbers. Seishi Yokomizo alone wrote over 70 novels featuring the scruffy, brilliant detective Kosuke Kindaichi. So far, Pushkin Vertigo has translated about 10 of them—impressive, but just the tip of the iceberg. Soji Shimada, the godfather of the logic-obsessed shin-honkaku movement, has had only a handful of his dozens of novels appear in English. And these are just the most famous names.

The reasons are complex. Japanese-to-English literary translation is costly and time-consuming. Publishers are often risk-averse, preferring to invest in authors with proven international sales records. Kulturkampf titles that require background knowledge of Japanese society can seem daunting to market. But the result is a frustrating feast-and-famine cycle: readers devour the few available translations and are left hungry for more.

The Authors Whose Best Work Remains Locked Away

Seishi Yokomizo: More Than the Honjin Murders

Yokomizo’s popularity in the West has soared since the translation of his debut, The Honjin Murders. Fans adore his macabre imagination, his rural settings steeped in family curses, and his detective Kindaichi, with his stutter and unassuming charm. But many of Yokomizo’s most acclaimed novels have never been translated.

For example, Akuma no Temari Uta (The Devil’s Hand-Tapping Song) is considered one of the finest locked-room mysteries in the Japanese canon. It involves a series of murders linked to an old nursery rhyme, and its denouement is both shocking and emotionally devastating. Another untranslated masterpiece is Byoinzaka no Kubikukuri no Ie (The House of the Hanging on Hospital Hill), a labyrinthine tale of family secrets and ritualized death. While we can hope Pushkin Vertigo will eventually bring these to light, for now they exist only in Japanese.

Soji Shimada: Puzzles Waiting to Be Solved

Soji Shimada’s Mitarai Kiyoshi series revolutionized the genre in the 1980s by embracing absurdly complex tricks and a Holmes-like detective. The Tokyo Zodiac Murders and Murder in the Crooked House have already achieved cult status in English. Yet some of Shimada’s most daring puzzles remain untranslated.

Consider Knight of the Underground Temple (Chikashitsuden no Kishi), in which a body is found inside a sealed underground chamber that appears both physically and magically locked. Or The Kobe Beef Murder Case (Kōbe Gyū Satsujin Jiken), a deliciously titled story involving a impossible crime on a train. Shimada’s ability to stretch plausibility to its breaking point while still playing fair is on full display in these works. Reading them is like watching a master magician reveal his secrets.

Other Giants of Untranslated Japanese Mystery

Beyond the two most famous names, the well of Japanese crime fiction goes deep. Tetsuya Ayukawa, known for his train alibi puzzles, has a vast oeuvre almost entirely untouched by English publishers. His detective, Inspector Onitsura, stars in dozens of stories that manipulate timetables with devilish precision. Shizuko Natsuki, a female pioneer of psychological suspense, wrote chilling novels like The Third Lady—but most of her bibliography, including her award-winning debut, Whispering Room, remains inaccessible. And Akimitsu Takagi’s later works, which blend horror with detection, also await discovery.

Specific Untranslated Novels You Should Know About

Here are five extraordinary Japanese mysteries that English readers can’t legally purchase in translation—and why they deserve to be on your radar.

Akuma no Temari Uta (The Devil’s Hand-Tapping Song) by Seishi Yokomizo

Why it matters: Often ranked among the greatest Japanese locked-room mysteries, this novel uses a macabre children’s song as the pattern for a series of murders. The solution is both ingenious and emotionally resonant, cementing Yokomizo’s reputation as a master of plotting.
Underground Temple Knight (Chikashitsuden no Kishi) by Soji Shimada

Why it matters: A quintessential Shimada locked-room extravaganza featuring a killing in a chamber sealed with magical symbols. The trick is characteristically wild yet logical, and the novel explores themes of obsession and illusion.
The Village of Doctor Ayakazu (Ayakazu-sense no Mura) by Tetsuya Ayukawa

Why it matters: An intricate alibi puzzle set in a remote village where a doctor’s murder forces Inspector Onitsura to untangle a web of train schedules and lies. Ayukawa’s work is a must for fans of Crofts or Freeman.
Whispering Room (Sasayaku heya) by Shizuko Natsuki

Why it matters: A tense psychological thriller about a woman whose new marriage hides dark secrets. Natsuki’s debut won the prestigious Edogawa Rampo Prize, yet it has never been published in English.
The Invisible Mask (Mienai Kamen) by Akimitsu Takagi

Why it matters: A later Takagi novel where a Noh mask seems to hold the key to an impossible disappearance. Blending Japanese tradition with pure detection, it shows a master at the height of his powers.

How to Read These Novels Now (Without Waiting for Publishers)

So what can you do if you’re dying to read these stories but don’t have strong Japanese? The good news is that technology and community have opened doors that didn’t exist a decade ago.

1. AI-Powered Translation Tools

If you can obtain a digital Japanese text (e.g., from an ebook store like BookWalker or a digital library), tools like LectuLibre let you upload the file and receive a full machine translation in minutes. While AI translations aren’t perfect—they can miss nuance, especially with cultural references or idiomatic dialogue—they have improved dramatically and can now produce readable, coherent prose. Many fans have already used these tools to enjoy untranslated manga and novels. If you’re patient and willing to gloss over occasional awkward phrasing, services like LectuLibre can be your personal key to the locked treasure chest of Japanese mystery.

2. Learn Japanese Through Your Love of Books

Obviously, learning Japanese is the ultimate solution. The journey from zero to reading a Yokomizo novel is long, but it’s a rewarding hobby that pays dividends across all your interests. Start with graded readers, move on to young adult mysteries, and gradually work your way up. The honkaku style, with its repetitive vocabulary around crime scenes and alibis, can actually be a helpful entry point for genre-focused learners.

3. Join Fan Translation Communities

Online forums like Reddit’s r/JapaneseMystery and dedicated Discord servers bring together bilingual fans who sometimes produce unofficial translations. While the quality varies, some are excellent and keep the torch burning for forgotten classics. Just remember to support official releases when they finally arrive.

4. Advocate and Wait

The more we talk about these books, the more likely publishers are to take notice. Review translated works, request them at your library, and mention untranslated titles on social media. Pushkin Vertigo and others have shown they listen to reader demand—The Honjin Murders itself was published after years of fan campaigning.

The Future of Japanese Mystery in English

The recent translation wave gives us hope. Yokomizo’s entire Kindaichi series may eventually see the light of day. Shimada might finally get the broader recognition his genius deserves. And new voices like Keigo Higashino continue to build bridges between cultures.

But there will always be more stories than there are translators and publishing slots. That’s where passion comes in. Whether you use AI tools, start learning Japanese, or simply voice your enthusiasm, you’re part of a global conversation that brings these novels across borders.

In the meantime, I’ll be over here trying to decipher the scene of the crime in Akuma no Temari Uta with a machine translation and a kanji dictionary. Won’t you join me?

Happy reading—or, should I say, happy detecting.