LongLoom: Stop Losing Facts When Your Documents Exceed the Context Window

Sumit Nemade — Sat, 07 Nov 2026 11:49:44 +0000

I built an open-source library that processes documents of any length through LLM map/merge/repair — with measured, reproducible proof that no facts are lost. Here's why I built it, how it works, and what the benchmarks show.

The problem nobody talks about honestly

Every LLM has a context window. Every enterprise has documents that blow past it.

The standard advice is "just chunk it and summarize each piece." That works — until you actually measure what's lost. We did, with a planted-fact test: seed a document with known identifiers, numbers, and dates, then check how many survive a naive chunk-summarize pipeline.

The result: naive summarization drops 2–10% of facts. Different facts every run. A contract ID here, a dollar amount there. Gone. And because it's stochastic, you can't even predict what disappears.

For a chatbot, 8% fact loss is annoying. For a compliance pipeline, a legal discovery, or a financial audit — it's disqualifying.

The fix: stop asking the LLM to summarize

LongLoom's core insight is simple: the pipeline should transport information losslessly, and the LLM should only condense when you explicitly allow it.

The library splits into two modes:

Mode	What the LLM does	Fact preservation	Use when
Summarizing (default)	Reads each chunk, produces a summary	Measured 2–10% loss	Readable reports, chat contexts
Verbatim extraction (`for_extraction()`)	Copies fact-bearing lines verbatim	100.0% at scale (4,367/4,367 planted facts)	Compliance, legal, audit, search indexing

The verbatim mode works because the extraction prompt tells the LLM to copy lines containing facts rather than paraphrase them. The merge stage deduplicates exact copies. The result: the pipeline adds zero information loss — verified by a deterministic fake LLM that provably echoes its input, plus real NVIDIA Nemotron and OpenAI GPT-4o-mini runs.

How it works

big document
     │
     ▼
CHUNK ──► line-aware splitting, persisted to your storage
     │
     ▼
MAP ──── one LLM call per chunk (parallel, checkpointed)
     │
     ▼
MERGE ── hierarchical combine, window-sized groups
     │
     ▼
REPAIR ─ bounded condensation, only if still too big
     │
     ▼
final result + verification report

The context window only ever sees chunks and compact intermediate results. The document never has to fit.

It works with what you already have

Any LLM. Not a wrapper around one vendor — a dependency-injection seam:

from longloom import init_llm

# 9 provider integrations, or bring your own:
llm = init_llm("nvidia", "nvidia/nemotron-3-ultra-550b-a55b", api_key=..., thinking=False)
llm = init_llm("openai", "gpt-4o-mini", api_key=...)
llm = init_llm("anthropic", "claude-sonnet-4-20250514", api_key=...)
llm = init_llm("custom", model="my-model", base_url="http://vllm.internal:8000/v1",
               headers={"X-Tenant": "acme"})     # vLLM, LiteLLM, Azure APIM — anything OpenAI-compatible
llm = lambda prompt: my_gateway(prompt)          # even a bare function

Any storage. Your databases, your schema, your tables:

from longloom import create_store

store = create_store("postgresql://app@host/mydb",
                     table="app_chunks",           # your table
                     columns={"id": "chunk_pk"},   # your column names
                     namespace="legal-team")       # multi-tenant safe

9 storage targets: PostgreSQL, MySQL, MSSQL, Oracle, MongoDB, Supabase, SQLite, JSONL, in-memory. Or subclass one ABC with four methods for anything else.

What the benchmarks show

All benchmarks are reproducible — the benchmark script ships in the repo:

python examples/20_benchmark.py --sections 50000

Benchmark	Result
Pipeline losslessness	83,023/83,023 facts preserved at 6.7M chars / 2.16M tokens / 3,795 chunks — 100.0%
Parallel map speedup	≈4× (measured with real NVIDIA API, 8 workers)
Real NVIDIA extraction (91.7%)	nvidia/nemotron-3-ultra at scale, verbatim mode
Real OpenAI extraction	gpt-4o-mini, same pipeline, same verbatim mode
CI coverage	100% line coverage, gated at 99% in CI on Python 3.10–3.13

The 83,023-fact stress test processes a document 265× larger than the NVIDIA context window — and preserves every single fact through chunk → map → hierarchical merge → repair.

Enterprise-ready, not enterprise-themed

The things security reviews actually check:

Explicit credentials — api_key= is a constructor argument. The library contains zero os.environ reads for secrets (verified by test).
Zero telemetry — no phone-home, no analytics, no update pings.
Your databases — chunks live in your tables, your columns, your namespaces. No dedicated database required.
Injection-validated — table and column names are regex-validated before any query is built.
XSS-hardened — generated HTML viewers escape content and disable script execution.
Structured logging — longloom.* stdlib loggers; prompts, completions, and credentials are never logged (regression-tested).
SECURITY.md — data-flow disclosure, private vulnerability reporting.
CI — pytest matrix on Python 3.10–3.13, pip-audit supply-chain scan, 100% line coverage gate, MongoDB service container for live integration tests.

Honest limitations

Map/merge aggregates; it doesn't reason globally. Cross-chunk conflicts are preserved side by side, not reconciled.
Real-LLM verbatim extraction achieves ~92% fact preservation at scale (the pipeline adds zero loss; the model's quality is the model's quality).
The map phase is sequential per worker; parallelize via max_workers for large corpora.
tiktoken approximates non-OpenAI tokenizers — keep 2× headroom on chunk sizes.

Try it

pip install longloom

from longloom import MapReduceProcessor, SQLiteContextStore, init_llm, PromptTemplates

processor = MapReduceProcessor(
    llm=init_llm("nvidia", "nvidia/nemotron-3-ultra-550b-a55b",
                 api_key=os.environ["NVIDIA_API_KEY"], thinking=False),
    store=SQLiteContextStore("ctx.db"),
    prompts=PromptTemplates.for_extraction(),
    max_workers=4,
)
result = processor.process(your_500_page_document, task="Extract all obligations")

GitHub: github.com/sumitnemade/longloom
PyPI: pypi.org/project/longloom
Landing page: sumitnemade.github.io/longloom
Benchmark script: examples/20_benchmark.py

MIT licensed. 88 tests. Zero telemetry. Star it if useful — contributions welcome.

LongLoom is built on LangGraph and langchain-core. It works with NVIDIA Nemotron, OpenAI GPT, Anthropic Claude, Groq, Ollama, Mistral, DeepSeek, Together AI, and any OpenAI-compatible endpoint.

DEV Community: Sumit Nemade