A new project called leanrea benchmarked what happens when you replace a 207-tool MCP server (REA) with a single 8 KB markdown skill that lets the agent run raw CLI commands directly through Bash.
On Claude Code 2.1 (Sonnet 5.5, macOS) across four reverse-engineering tasks:
- 38% less wall time (388s vs 627s)
- 35% lower cost (.17 vs .36)
- 60% fewer tokens (2.40M vs 5.93M)
- Same score (74/75 vs 75/75)
Frontier models already know standard flags for , , , , and . When you wrap every Unix flag in a custom JSON schema, you pay context tax on every prompt just to parse an unfamiliar API abstraction.
Full writeup and breakdown: https://reidmarlow.com/stop-wrapping-cli-tools-in-200-mcp-endpoints
Top comments (3)
The benchmark matches what we saw from the other direction, with one carve-out. For local tools the model already knows the flags, so a schema is pure tax. For a remote data API it does not know the auth header, the pagination cursor or which 20 of 90 fields exist, and a bare curl skill makes it guess those on every call. What worked for us was neither 207 endpoints nor raw curl: one tool per job (search, lookup, count), a description under 350 characters, and the field list fetched on demand instead of shipped in tools/list. Context cost landed near the size of a skill file and the pagination errors went to zero, because there was nothing left to guess. Did leanrea's 8 KB skill carry the error handling, or did the agent re-derive it on each task?
The leanrea skill baked the curl failure modes directly into the script rather than making the model handle them. It checked the HTTP status on every request and retried 429s with exponential backoff before returning JSON to the model. Having the tool wrapper handle network retries meant the model only saw clean records or explicit hard errors.
That split is the right one. The state I'd add to the wrapper: a 429 it gave up on after backoff has to come back as its own explicit state, not folded into "hard error" and not into "no record", because downstream both read as a miss. In one storm we measured two rows per hundred lost that way, and nothing in the dataset showed it. The outcome table and where each should route: dev.to/kevinhall/four-kinds-of-emp...