DEV Community: Arman Nourifar

How AI context compression actually works, part 1

Arman Nourifar — Sun, 06 Sep 2026 17:51:27 +0000

I build Compresso, a local AI compression layer, using different techniques like Headroom, Rtk and others we cut down on token usage costs.

Every request your coding agent sends is mostly things you've already paid for. The file it read three turns ago. The build log it scrolled past. The 500-row JSON blob a tool returned when the model needed four fields. All of it rides along on every single request, and all of it is billed again.

Context compression is the fix, and it's worth being precise about what it is, because the word invites a misunderstanding.

Compression here does not mean summarizing. No model paraphrases your context into something shorter and hopes the meaning survives.

Compression means the same information in fewer tokens: selecting what matters verbatim, re-encoding what's repetitive, and pointing back to what's already there. Densifying, not discarding.

Here's how that actually works, technique by technique.

Route first, compress second

The first step isn't compression at all, it's recognition. A local ML classifier looks at each block of content and decides what it is: a JSON payload, a build log, source code, a diff, prose. That takes a few milliseconds, on your machine.

This matters because there is no universal compressor. What's safe to do to a build log, collapse 400 lines of Compiling... into one, would be vandalism on source code. So each content type is routed to a compressor built for its shape, and anything the router isn't confident about passes through untouched.

This is also why honest benchmark tables have zeros in them. Grep results that are already dense, and source code the model actually needs to see: the router looks at both and does nothing. Doing nothing is a feature.

Repetitive structure: the JSON problem

Tool outputs love to return arrays of near-identical objects. A hundred log events, five hundred search results, every one carrying the same keys and mostly the same values.

A structure-aware compressor analyzes the array statistically: which fields vary, which are constant, where the values change. Then it keeps the head, the tail, the change points, and drops the verbatim duplicates in between. Every item it keeps is an untouched original, same schema, same bytes, no generated text mixed in.

Before, 500 items:

[
  {"level": "info", "service": "api", "msg": "request ok", "status": 200},
  {"level": "info", "service": "api", "msg": "request ok", "status": 200},
  ... 496 more, nearly identical ...
  {"level": "error", "service": "api", "msg": "upstream timeout", "status": 504}
]

After: the first few, the last few, and that one error, because that's where the information was. Measured on a real 500-item payload: 9,526 tokens → 1,614, in 2 milliseconds.

Logs: keep the signal, fold the noise

Build and test output is the most compressible thing an agent reads, and the most dangerous to compress naively. The rule: errors, warnings, and stack traces survive in full, including chained exceptions, where the interesting frame is often three Caused by: blocks deep. What gets folded is the repetition around them.

   Compiling serde v1.0.203
   Compiling tokio v1.38.0
   ... 180 lines like this ...
error[E0308]: mismatched types
  --> src/pipeline.rs:214:9

The error block comes through byte-for-byte. The 180 compile lines become a note that they happened. Measured: a 200-line build log went from 2,412 tokens to 148.

Lossless re-encoding: same facts, fewer bytes

Some transforms don't select at all, they re-encode, with an exact inverse. Strip ANSI color codes. Collapse a line repeated 40 times into the line plus a count. Group grep hits under one file heading instead of repeating the path on every match:

src/auth.rs:12: fn validate_token(
src/auth.rs:48: fn validate_token_expiry(
src/auth.rs:91: // validate_token is called from...

becomes

src/auth.rs:
  12: fn validate_token(
  48: fn validate_token_expiry(
  91: // validate_token is called from...

Grep stays grep, diffs stay diffs, logs stay logs, the model reads them exactly as it would have. And each of these transforms is self-checking: if reversing it doesn't reproduce the original, or the result isn't actually smaller, the content ships unchanged.

Déjà vu: the same file, four times

Here's the redundancy no per-block compressor can see. Your agent runs cat pipeline.py. Two turns later, git diff shows most of the same lines. Then a test failure prints twenty of them again. The same bytes, billed three times, on every request from then on.

Cross-turn deduplication watches the whole conversation. When a later tool output contains a large span that already appeared verbatim earlier, the later copy becomes a compact pointer: this content appeared in full above. The first occurrence is never touched, so the original is always physically in the context the model reads. And it's done carefully enough that earlier turns are never rewritten, which keeps provider-side prompt caching working, so the savings stack instead of fighting each other.

Nothing is ever truly gone

The last piece is the safety net under all of the above: reversibility. When compression does set something aside, the middle rows of that array, the folded stretch of log, it's kept, locally, and the model gets a way to ask for it. If the model decides it genuinely needs row 247, it retrieves row 247. In practice it rarely asks, because the compressors keep what mattered. But the option is what makes the whole pipeline trustworthy: compression that can be undone is compression you don't have to think about.

This is what runs on your Mac

Everything above runs locally, inside Compresso, on every request your agent makes, before the request is billed. No prompts leave your machine, and the parts that don't compress safely simply don't compress. Every plan includes the full pipeline; the tiers only mirror the Claude or ChatGPT plan you're already on.

If you want to see the technique-by-technique numbers, including the ones where compression does nothing at all, they're on the benchmarks page. Or just try it on today's session: five-day trial, no card up front.

Why Human APIs fail as MCP tools (and how to fix them)

Arman Nourifar — Thu, 28 Aug 2025 17:02:05 +0000

Introduction

The Model Context Protocol (MCP) is a powerful way to expose your systems to AI agents. But a common mistake teams make is to take an API built for humans, auto-convert it to MCP tools, and expect agents to use it effectively.

APIs designed for human developers follow a very different set of rules than APIs/Tools that serve AI agents in agentic workflows. What’s intuitive for a person is often overwhelming, inefficient, and error-prone for an LLM. If we want our agents to succeed, we need to design for them from the ground up — not just repurpose what we already have.

Setting the stage:

Imagine you want to enable agents to use your flight booking system by converting the current APIs you have, and you ask your agent to find the cheapest direct flight to Paris in June, This is roughly the api calls your agent should make:

GET /airports/search?query=paris - Find Paris airport codes
GET /airlines/list - Get all available airlines
GET /flights/search?origin=JFK&destination=CDG&departure_date=2024-06-01 - Search flights for June 1st
GET /flights/search?origin=JFK&destination=CDG&departure_date=2024-06-02 - Search flights for June 2nd
GET /flights/search?origin=JFK&destination=CDG&departure_date=2024-06-03 - Search flights for June 3rd
(Continue for each day in June...)
GET /flights/123/details - Get detailed info for each flight
GET /flights/456/details - Get detailed info for another flight
GET /flights/789/details - Get detailed info for another flight
GET /flights/filter?direct_only=true - Filter for direct flights only
GET /flights/sort?by=price&order=asc - Sort by price ascending
GET /pricing/calculate?flight_id=123&passengers=1 - Calculate total price with fees
GET /pricing/calculate?flight_id=456&passengers=1 - Calculate total price with fees

The Problem: Human APIs Don’t Fit Well with Agent Workflows

Developers want lots of small, composable endpoints they can stitch together in whatever way makes sense for their application. They often have many endpoints, parameters, and options so a developer can decide what to use and how to combine calls.

Converting each of these APIs 1:1 to an MCP tool will create friction:

Too many options → harder for the agent to decide which call is correct.
Unclear purposes → descriptions meant for humans don’t map cleanly to the agent’s decision process.
Extra steps → an agent may need multiple calls to complete something a human would do in one mental step. The result is an MCP tool that works in theory, but slows down or confuses the agent in practice.

The Solution: Think Like an Agent

Agents don’t explore the way humans do. They work toward goals by following reasoning chains. When deciding how to turn an API into MCP tools, imagine you’re telling the agent a story about what it can do and how it can act.

Instead of exposing every possible endpoint, focus on clear, goal-oriented actions. Each tool should represent something the agent can immediately understand and execute — like “find a product by name” or “book a meeting” — not just a low-level function.

Using Agent Stories

Netlify’s CEO, Mathias Biilmann, recently highlighted in his talk and article “Introducing AX: Why Agent Experience Matters” that agents are becoming one of the core “personas” they design for. Just as we think about user experience (UX) for humans, we should think about agent experience (AX) when building MCP tools.

One way to guide design is to write agent stories: short scenarios showing how the agent might solve a problem using your MCP tools.

Example:

“A user asks the agent to find the cheapest direct flight to Paris in June. The agent uses the searchFlights tool with the destination and month, then picks the best flight with selectBestFlight tool based on the criteria it has (in this case lowest price).”

By imagining these flows, you’ll see where you can simplify steps, combine related actions, or rename tools so their purpose is obvious. Now the new flow our agent will follow immediately is:

searchFlights(destination="Paris", month="June", departure_city="New York") - Returns top 5-10 cheapest direct flight options with key details
selectBestFlight(criteria="cheapest") - Picks the lowest priced option from the previous search results

# Or even all in one tool call
findCheapestDirectFlights(destination="Paris", month="June", departure_city="New York", limit=3)

Designing Flows for Agents

When building MCP tools, think in terms of flows rather than just endpoints. A flow represents the sequence of actions an agent will take from start to finish for a given task.

You can help the agent succeed by:

Grouping actions that naturally belong together.
Removing unnecessary branching that would force the agent to guess.
Providing hints in tool descriptions about what comes next or what other options are possible.
This doesn’t just make the agent faster — it makes it more reliable and less likely to get stuck.

Conclusion

Converting a human API into MCP tools is more than just an automated format change. By designing with agents in mind, using stories, flows, and goal-oriented thinking, you can create tools that work seamlessly in an agent’s reasoning process.