TL;DR — I gave the same 3.9 MB Japanese-government CSV to five web-fetch tools that coding agents use. Jina Reader returned 4,593,027 tokens (instant context death). amenbo — my MCP server — returned a header preview plus a pointer to the raw file: 909 tokens. That ~5,000× gap is the headline, but the more useful finding is why general-purpose fetchers break on Japanese pages at all. Harness + raw logs are in the repo; every number below is reconciled against them.
The problem this started from
Point a coding agent at Japanese web pages — a gov data page, a Shift_JIS text archive, a CSV/PDF link — and you hit failures the English-tuned tools rarely warn you about:
- Shift_JIS / EUC-JP pages come back as mojibake (they're still everywhere in Japan).
- A single long page dumps hundreds of thousands of tokens, nav + ranking widgets + banners and all, and your context dissolves.
- PDF/CSV URLs return raw bytes, or the tool downloads 16 MB to disk and stops.
So I stopped hand-waving about being "good at Japanese" and measured it, on real URLs, against the tools people actually use.
The setup
Five tools, one JSON-RPC harness, seven Japanese URLs picked to be the failure-prone types:
| Tool | Form | Why included |
|---|---|---|
| amenbo (mine) | MCP (stdio) | two-stage fetch + CJK-tuned extraction |
| mcp-server-fetch | MCP (uvx) | the official reference impl — the baseline |
| Jina Reader (r.jina.ai) | HTTP API | the most common URL→Markdown service |
| Playwright MCP | MCP (npx) | the browser-rendering representative |
PixelRAG pixelshot
|
CLI | 2026's "stop parsing HTML, screenshot everything" idea |
URLs: a Zenn tech article, a table-heavy Wikipedia page, a Shift_JIS 1.2 MB novel (Aozora Bunko), a government open-data HTML page, a Japanese PDF, a 3.9 MB CSV, and a site that refuses plain HTTP clients. macOS / Node v26 / cold cache / one hit each. Text tokens use one shared CJK-aware estimator; images use Claude's width × height ÷ 750 visual-token estimate.
The numbers (tokens an agent actually pays)
| URL | amenbo | mcp-server-fetch | Jina Reader | Playwright MCP | pixelshot |
|---|---|---|---|---|---|
| zenn | 4,679 | 2,627 ⚠️truncated | 4,216 | 9,049 | 11,462 🖼️10 imgs |
| wiki | 5,337 (p1/4) | 2,078 ⚠️truncated | 38,014 | 55,970 | 1,178 ⚠️incomplete |
| aozora | 7,992 (p1/25) | 1,400 ❌mojibake | 329,380 ⚠️full dump | 453,263 | 263,819 🖼️221 imgs |
| gov | 1,774 ✅+data links | 519 | 26,983 | 25,382 | 13,355 🖼️12 imgs |
| 810 | 1,329 ❌raw bytes | 802 | ❌can't | ❌875×1px img | |
| csv | 909 ✅handoff | 2,669 ⚠️raw dump | 4,593,027 | ❌16MB to disk | 1,186 ⚠️first screen only |
A few things worth reading slowly.
The 3.9 MB CSV (the 5,000×). Feeding an LLM a 3.9 MB CSV is pointless. amenbo returns the header row, a short preview, and a pointer ("110_home-visit-care [15.4MB] — , 104 more via links filter:'*.csv'") so the agent can curl + script the aggregation instead. Jina returned all 4,593,027 tokens. Same file, ~5,000× apart.
The Shift_JIS novel (1.2 MB). amenbo and Jina both decode it; the official fetch garbles it:
amenbo: 吾輩は猫である。名前はまだ無い。どこで生れたか…
mcp-server-fetch: #### �� | �@��y�i�킪�͂��j�͔L�ł���B… (mojibake, UTF-8 assumed)
Jina Reader: 吾輩（わがはい）は猫である。名前はまだ無い… (decodes, but keeps furigana inline)
The catch is token volume. Jina returns the whole novel in one shot — 329,380 tokens, furigana and all (as of a 2026-07-12 re-run; it was a 503 error before). Playwright MCP paid 453,263 tokens for the accessibility tree and pixelshot 263,819 visual tokens across 221 image tiles. amenbo strips the ruby and splits it into 25 token-budgeted pages (7,992 for page 1, fetch page: 2 for more). A fetch that can return 300–450k tokens breaks your context-management assumptions regardless of averages.
PDF & CSV. Japanese public-sector data is made of PDFs and CSVs. amenbo extracts PDF text (810 tok) and hands off CSVs with a preview (909 tok). The official fetch returns %PDF-1.7 %����… raw bytes; Playwright can't snapshot the PDF at all.
Progressive disclosure: read the map before the territory
The numbers above are full Markdown. But amenbo can return an outline first — the heading tree with a per-section token estimate — so the agent fetches only the sections it actually needs. On the same pages, mode: "outline" costs 75–97% fewer tokens than full Markdown:
| page | markdown | outline | saved |
|---|---|---|---|
| Wikipedia (prefecture populations) | 5,337 | 659 | 87.7% |
| Aozora novel (Shift_JIS) | 7,992 | 408 | 94.9% |
| Zenn tech article | 4,679 | 1,141 | 75.6% |
| MDN JavaScript guide (English) | 2,816 | 389 | 86.2% |
(Re-measured 2026-07-12 on v0.2.0; logs in bench/logs-2026-07-12/.) For an agent researching across many pages, deciding how much to read before reading it is what keeps the context window alive — and it's language-agnostic, so it helps on English pages too.
Where it fits (and where it doesn't)
This isn't "replace everything":
- Jina Reader / Firecrawl are great hosted crawling APIs. amenbo is the opposite trade-off: local, free, MIT, no API key, built to fetch a handful of pages precisely. (On plain Markdown quality, Jina is genuinely equal — on the Zenn article both reproduced all 24 table rows and 5 code blocks. amenbo's edge is holding that quality while dropping nav/banners and capping the page budget.)
-
The official
fetchis the baseline everyone has; amenbo is what you reach for when it returns mojibake or raw bytes. - Playwright / Browserbase are for browser automation; amenbo is for reading.
- PixelRAG is a screenshot-based retrieval index — a different, complementary layer, not a competitor to live fetching.
The niche, as far as I can tell, is unoccupied: the polite, local, free fetcher that also survives the Japanese web.
The honest part (weaknesses included)
| Axis | amenbo | note |
|---|---|---|
| Token use | ◎ | per-page cap + progressive disclosure; orders of magnitude on long/CSV |
| Extraction quality | ○ | equal to Jina + banner removal — not a blowout |
| CJK handling | ◎ | decodes Shift_JIS, strips furigana, paginates (Jina decodes too but keeps furigana & dumps the whole novel; official fetch garbles it) |
| Non-HTML (PDF/CSV) | ◎ | only tool of the five doing both |
| Error handoff | ○ | classifies the error and suggests the next move |
| JS-heavy / anti-bot pages | △ | Playwright et al. win here; amenbo has a system-Chrome fallback but implements no anti-bot circumvention — it stops honestly |
| Search / indexing | ✕ | PixelRAG's domain; amenbo has no search (use both) |
| Speed | △ | politeness first (per-domain serial, 1 req/sec) — not for fast crawling, by design |
It identifies as a bot, respects robots.txt / Crawl-Delay, and when a site blocks it, it says so instead of working around it. That's a design choice, not a gap.
Try it
npm install -g amenbo
# browser features (SPA rendering, screenshots), one-time ~170MB:
npx -y amenbo install-browser
Register with your agent (.mcp.json): { "mcpServers": { "amenbo": { "command": "amenbo" } } }
There's also a CLI sharing the same core (amenbo fetch <url> > page.md), and a copy-paste prompt for CLAUDE.md / AGENTS.md in the README so the agent uses progressive disclosure by default.
If your agents touch the Japanese web, I'd like to know where amenbo breaks for you. The harness and raw logs are public — counter-tests welcome.
Repo: https://github.com/Rererr/amenbo · Benchmark: bench/
amenbo (アメンボ) is Japanese for "water strider" — the insect that skims across water without making ripples. MIT licensed. This is an English write-up of a Japanese benchmark article.
Top comments (0)