One agent web-fetch tool returned 4.6M tokens for a 3.9MB CSV. Another returned 909.

Rererr — Sun, 12 Jul 2026 15:10:40 +0000

TL;DR — I gave the same 3.9 MB Japanese-government CSV to five web-fetch tools that coding agents use. Jina Reader returned 4,593,027 tokens (instant context death). amenbo — my MCP server — returned a header preview plus a pointer to the raw file: 909 tokens. That ~5,000× gap is the headline, but the more useful finding is why general-purpose fetchers break on Japanese pages at all. Harness + raw logs are in the repo; every number below is reconciled against them.

The problem this started from

Point a coding agent at Japanese web pages — a gov data page, a Shift_JIS text archive, a CSV/PDF link — and you hit failures the English-tuned tools rarely warn you about:

Shift_JIS / EUC-JP pages come back as mojibake (they're still everywhere in Japan).
A single long page dumps hundreds of thousands of tokens, nav + ranking widgets + banners and all, and your context dissolves.
PDF/CSV URLs return raw bytes, or the tool downloads 16 MB to disk and stops.

So I stopped hand-waving about being "good at Japanese" and measured it, on real URLs, against the tools people actually use.

The setup

Five tools, one JSON-RPC harness, seven Japanese URLs picked to be the failure-prone types:

Tool	Form	Why included
amenbo (mine)	MCP (stdio)	two-stage fetch + CJK-tuned extraction
mcp-server-fetch	MCP (uvx)	the official reference impl — the baseline
Jina Reader (r.jina.ai)	HTTP API	the most common URL→Markdown service
Playwright MCP	MCP (npx)	the browser-rendering representative
PixelRAG `pixelshot`	CLI	2026's "stop parsing HTML, screenshot everything" idea

URLs: a Zenn tech article, a table-heavy Wikipedia page, a Shift_JIS 1.2 MB novel (Aozora Bunko), a government open-data HTML page, a Japanese PDF, a 3.9 MB CSV, and a site that refuses plain HTTP clients. macOS / Node v26 / cold cache / one hit each. Text tokens use one shared CJK-aware estimator; images use Claude's width × height ÷ 750 visual-token estimate.

The numbers (tokens an agent actually pays)

URL	amenbo	mcp-server-fetch	Jina Reader	Playwright MCP	pixelshot
zenn	4,679	2,627 ⚠️truncated	4,216	9,049	11,462 🖼️10 imgs
wiki	5,337 (p1/4)	2,078 ⚠️truncated	38,014	55,970	1,178 ⚠️incomplete
aozora	7,992 (p1/25)	1,400 ❌mojibake	329,380 ⚠️full dump	453,263	263,819 🖼️221 imgs
gov	1,774 ✅+data links	519	26,983	25,382	13,355 🖼️12 imgs
pdf	810	1,329 ❌raw bytes	802	❌can't	❌875×1px img
csv	909 ✅handoff	2,669 ⚠️raw dump	4,593,027	❌16MB to disk	1,186 ⚠️first screen only

A few things worth reading slowly.

The 3.9 MB CSV (the 5,000×). Feeding an LLM a 3.9 MB CSV is pointless. amenbo returns the header row, a short preview, and a pointer ("110_home-visit-care [15.4MB] — , 104 more via links filter:'*.csv'") so the agent can curl + script the aggregation instead. Jina returned all 4,593,027 tokens. Same file, ~5,000× apart.

The Shift_JIS novel (1.2 MB). amenbo and Jina both decode it; the official fetch garbles it:

amenbo:            吾輩は猫である。名前はまだ無い。どこで生れたか…
mcp-server-fetch:  #### �� | �@��y�i�킪�͂��j�͔L�ł���B…   (mojibake, UTF-8 assumed)
Jina Reader:       吾輩（わがはい）は猫である。名前はまだ無い…       (decodes, but keeps furigana inline)

The catch is token volume. Jina returns the whole novel in one shot — 329,380 tokens, furigana and all (as of a 2026-07-12 re-run; it was a 503 error before). Playwright MCP paid 453,263 tokens for the accessibility tree and pixelshot 263,819 visual tokens across 221 image tiles. amenbo strips the ruby and splits it into 25 token-budgeted pages (7,992 for page 1, fetch page: 2 for more). A fetch that can return 300–450k tokens breaks your context-management assumptions regardless of averages.

PDF & CSV. Japanese public-sector data is made of PDFs and CSVs. amenbo extracts PDF text (810 tok) and hands off CSVs with a preview (909 tok). The official fetch returns %PDF-1.7 %��… raw bytes; Playwright can't snapshot the PDF at all.

Progressive disclosure: read the map before the territory

The numbers above are full Markdown. But amenbo can return an outline first — the heading tree with a per-section token estimate — so the agent fetches only the sections it actually needs. On the same pages, mode: "outline" costs 75–97% fewer tokens than full Markdown:

page	markdown	outline	saved
Wikipedia (prefecture populations)	5,337	659	87.7%
Aozora novel (Shift_JIS)	7,992	408	94.9%
Zenn tech article	4,679	1,141	75.6%
MDN JavaScript guide (English)	2,816	389	86.2%

(Re-measured 2026-07-12 on v0.2.0; logs in bench/logs-2026-07-12/.) For an agent researching across many pages, deciding how much to read before reading it is what keeps the context window alive — and it's language-agnostic, so it helps on English pages too.

Where it fits (and where it doesn't)

This isn't "replace everything":

Jina Reader / Firecrawl are great hosted crawling APIs. amenbo is the opposite trade-off: local, free, MIT, no API key, built to fetch a handful of pages precisely. (On plain Markdown quality, Jina is genuinely equal — on the Zenn article both reproduced all 24 table rows and 5 code blocks. amenbo's edge is holding that quality while dropping nav/banners and capping the page budget.)
The official fetch is the baseline everyone has; amenbo is what you reach for when it returns mojibake or raw bytes.
Playwright / Browserbase are for browser automation; amenbo is for reading.
PixelRAG is a screenshot-based retrieval index — a different, complementary layer, not a competitor to live fetching.

The niche, as far as I can tell, is unoccupied: the polite, local, free fetcher that also survives the Japanese web.

The honest part (weaknesses included)

Axis	amenbo	note
Token use	◎	per-page cap + progressive disclosure; orders of magnitude on long/CSV
Extraction quality	○	equal to Jina + banner removal — not a blowout
CJK handling	◎	decodes Shift_JIS, strips furigana, paginates (Jina decodes too but keeps furigana & dumps the whole novel; official fetch garbles it)
Non-HTML (PDF/CSV)	◎	only tool of the five doing both
Error handoff	○	classifies the error and suggests the next move
JS-heavy / anti-bot pages	△	Playwright et al. win here; amenbo has a system-Chrome fallback but implements no anti-bot circumvention — it stops honestly
Search / indexing	✕	PixelRAG's domain; amenbo has no search (use both)
Speed	△	politeness first (per-domain serial, 1 req/sec) — not for fast crawling, by design

It identifies as a bot, respects robots.txt / Crawl-Delay, and when a site blocks it, it says so instead of working around it. That's a design choice, not a gap.

Try it

npm install -g amenbo
# browser features (SPA rendering, screenshots), one-time ~170MB:
npx -y amenbo install-browser

There's also a CLI sharing the same core (amenbo fetch <url> > page.md), and a copy-paste prompt for CLAUDE.md / AGENTS.md in the README so the agent uses progressive disclosure by default.

If your agents touch the Japanese web, I'd like to know where amenbo breaks for you. The harness and raw logs are public — counter-tests welcome.

Repo: https://github.com/Rererr/amenbo · Benchmark: bench/

amenbo (アメンボ) is Japanese for "water strider" — the insect that skims across water without making ripples. MIT licensed. This is an English write-up of a Japanese benchmark article.

DEV Community: Rererr