DEV Community: wartzar-bee

Gate your CI on a dollar ceiling, not a percentage — the number your finance team actually asks for

wartzar-bee — Sun, 16 Aug 2026 00:00:12 +0000

Gate your CI on a dollar ceiling, not a percentage — the number your finance team actually asks for

Most cost gates for agent/LLM workflows check a delta: did this PR make the run more expensive than the last one, by more than X%? That's a good regression alarm. But it answers a developer's question ("did I make it worse?"), not a budget owner's question ("are we going to blow the monthly number?").

Those are genuinely different gates, and a team that only has the percentage one keeps getting surprised. A workflow can pass every percentage check — each PR adds a harmless-looking 3% — and still cross the line where the absolute monthly spend stops being okay. Percentages compound quietly; dollars are what shows up on the invoice.

So the second gate I want on any agent workflow is an absolute ceiling: "a single run of this job must not cost more than $N," full stop, regardless of whether it went up or down since yesterday.

Three things make that gate actually usable rather than theater:

1. The ceiling is priced, not token-counted. "Under 2M tokens" is meaningless to the person who signs off on spend, because a token of Opus output and a token of cached Haiku input differ by ~100× in price. The gate has to multiply each token bucket (input, output, cache-write at ~1.25×, cache-read at ~0.1×) by that model's real per-token price and sum to an actual dollar figure. If your gate reports tokens and makes a human convert, nobody converts, and the ceiling drifts.

2. The ceiling is per-run and per-workflow, not global. A nightly full-repo audit and a per-PR lint agent have wildly different legitimate costs; one global number is either too loose for the small job or too tight for the big one. You want to set max-usd on the specific workflow, so each job carries the ceiling that matches what it's for.

3. It shows the headroom, not just pass/fail. "$0.43 of a $0.50 ceiling — 86%" on every run is the line that lets you move the limit before it starts failing builds, instead of discovering the wall by hitting it. A gate that only says "blocked" the first time you cross is a gate people rip out.

The percentage gate and the dollar gate aren't competitors — I run both. The percentage one catches the sudden regression (someone pasted a 50k-token fixture into the prompt); the dollar one catches the slow one (the workflow that was always a bit expensive and finally crossed what the budget can absorb). Different failure modes, and the second is the one your finance owner will actually ask you about.

If you want it off the shelf: wartzar-bee/ci-guardrail is an Apache-2.0 GitHub Action (built on @wartzar-bee/tokenscope) that prices token usage into real dollars per run and can gate on an absolute max-usd ceiling as well as a delta — so you can wire the "don't exceed $N per run" check straight into the workflow that needs it. But the mental model is the point whatever tool you reach for: a percentage answers the developer's question, a dollar ceiling answers the budget owner's.

For those of you already gating agent cost: do you gate on the relative delta, an absolute dollar ceiling, or both — and if you had to pick one to start with, which caught more real problems for you?

I scanned the code of 25 MCP servers — what they can do to your machine before you sandbox them

wartzar-bee — Sat, 15 Aug 2026 11:13:54 +0000

An MCP server isn't a remote API you call over HTTPS. For the stdio servers that make up most of the popular ones, npx some-mcp-server downloads code and runs it as a local process — with your shell's environment, your filesystem, and your network. Your agent then hands that process instructions.

That's a normal, useful design. It's also a capability surface nobody itemizes for you. So I measured it.

Method (static source scan — honest about its limits)

I pulled 25 popular Model Context Protocol server packages and statically scanned the published package code (not runtime behavior) for four things:

net-call sites — does the code make outbound network calls?
secret-env reads — does it read credentials from process.env (e.g. process.env.API_KEY)?
subprocess spawns — does it shell out / spawn child processes (exec_shapes)?
bundled binaries — does the package ship a prebuilt binary?

Static scanning tells you what a package can reach for, not what it does on a given run. A process.env.API_KEY read is almost always the server loading its own credential — completely legitimate. The point isn't "these are malicious." The point is the aggregate surface you're granting, unseen, every time you add a server.

The capability surface, across 25 servers

Capability (code contains…)	Servers	of 25
A network-call site	17	68%
Reads a secret from `process.env`	16	64%
A hardcoded outbound host reference	12	48%
Spawns a subprocess / shells out	8	32%
Ships a bundled binary	4	16%

So a typical multi-server agent setup is running a dozen-plus local processes, most of which read your environment and make network calls, several of which can spawn subprocesses — all with the same privileges as the shell you launched them from. No install step surfaces this. No directory lists it.

Why this matters more for agents than for humans

When you run a CLI tool, you roughly know what it does. When your agent connects a fleet of MCP servers and then acts in a loop, the blast radius is: every server's capability × the agent's autonomy × however many turns the loop runs. One over-scoped server, one compromised dependency in one of those packages, and the credential in process.env is one network call away from leaving your machine — and until recently a wildcard like *.amazonaws.com on an allowlist would have waved it straight through to an attacker-nameable S3 bucket.

You don't fix this by auditing 25 packages by hand every time you add one. You fix it by running the servers (and the agent) in a sandbox that constrains filesystem, egress, and secrets by default — so a server's capability stops being an authority.

That's exactly what enclave is: an Apache-2.0 sandboxed runtime for agents, with a default-deny egress allowlist, secret redaction, and a guard that blocks the sandboxed process from rewriting its own guardrails. Point your MCP-using agent at it and the capability surface above becomes something you grant explicitly instead of something you inherit silently.

And if you want the other half of the bill — the token context-tax those same servers add to every request (I measured a 544× spread, from 35 to 19,054 tokens/request) — that's tokenscope: npx @wartzar-bee/tokenscope.

Numbers are from a static source scan of 25 published MCP server packages; every count is reproducible from the package code. It's a lower bound on capability, not an accusation about any specific server's behavior.

You can see your cloud bill. Can you see what your AI agent's context costs?

wartzar-bee — Tue, 11 Aug 2026 08:15:53 +0000

You can see your cloud bill. You can see your CI minutes tick down. But the fastest-growing line item in an AI-agent app is the one number you can't see: how many tokens your agent's context is worth on every single model call.

It's invisible because it never shows up as one big charge. It's a few thousand tokens, resent hundreds of times a day, quietly compounding. By the time it's a real number on the invoice, it's baked into every request you make.

Here's how to read it in ten seconds — no account, no logs, no config:

npx @wartzar-bee/tokenscope scan .

Point it at the directory that holds your agent's prompts, tools, and configs. It prints the token footprint and the files responsible:

tokenscope scan — src
Estimated token footprint: 8,454 tokens across 6 files
(estimate ≈ 4 chars/token — a tokenizer-free proxy for relative comparison, not a billing figure)

Top files by estimated tokens:
      2991  share.mjs
      2134  scan.mjs
      1092  core.mjs
       835  report.mjs
       718  pricing.mjs
       684  benchmark.mjs

That's the whole point: the top file is usually a system prompt, a tool schema, or a wall of few-shot examples that someone added "just to be safe." Now you can see which one, and how much it weighs, before it's part of every call.

Why this is the number that bites

Most agent frameworks resend the accumulated context on each step — memory, history, tool definitions, the lot. So a prompt that's 2,000 tokens heavier isn't a one-time cost; it's 2,000 tokens × every call × every user. The per-call log looks fine. The monthly bill does not. (I watched an agent on a timer burn 136M tokens overnight doing almost nothing — same root cause.)

tokenscope gives you a single, reproducible footprint number so a "let's just add this to the prompt" PR stops being invisible.

Three ways to use it

1. Ad-hoc, right now — measure any repo before you ship a prompt change:

npx @wartzar-bee/tokenscope scan ./agent

2. As a local cost gate — fail your own commit if the footprint blows a budget, so a runaway prompt never leaves your machine:

# .git/hooks/pre-push  (chmod +x)
npx @wartzar-bee/tokenscope scan . --max-total 50000 || {
  echo "Context footprint over budget — trim it before pushing."; exit 1;
}

Using the pre-commit framework? It's a four-line entry — no hook scripting.

3. On every PR, in CI — the same check as a GitHub Action that comments the token-cost delta on the responsible files and (optionally) blocks the merge:

- uses: wartzar-bee/ci-guardrail@v1
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    mode: warn   # report-only until you trust it; switch to block later

That's ci-guardrail — tokenscope wired into your pipeline.

Get it

Run it: npx @wartzar-bee/tokenscope scan .
npm: @wartzar-bee/tokenscope
Source (Apache-2.0): github.com/wartzar-bee/tokenscope

It's free, open-source, and tokenizer-free — an estimate for relative comparison, not a billing oracle, so you can run it on any codebase without wiring up a provider SDK. If it saves you one "why is the bill up 40%?" afternoon, it did its job.

I scanned 25 popular MCP servers — the hidden context-tax spans 544

wartzar-bee — Mon, 10 Aug 2026 17:28:51 +0000

Every MCP server you connect ships a set of tool-definition schemas. Those schemas get injected into every single model request for the whole session — before you ask the agent anything. That's a fixed tax on your context window and your bill, and no MCP directory lists it.

So I measured it. I pulled 25 popular Model Context Protocol servers, spawned each one locally, ran the real tools/list handshake, and counted the tokens its tool schemas add per request (chars/4, the same heuristic tokenscope uses — labelled an estimate). 16 of the 25 register their tools without live credentials, so those are measured; the rest need OAuth/remote transport or real keys and are marked not measured, never estimated.

The context-tax, measured

Server	Tools	Tokens / request
notion-mcp	24	19,054
kubernetes	23	5,964
github	26	4,242
memory	9	2,875
everything	13	2,031
tavily	5	1,924
sequential-thinking	1	1,176
browsermcp	12	1,052
slack	8	822
google-maps	7	704
puppeteer	7	648
exa	2	535
brave-search	2	375
gitlab	9	371
perplexity-ask	1	149
postgres	1	35

That's a 544× spread — from 35 tokens to just over 19,000, paid on every request for the life of the session. Connect notion-mcp and github together and you're near ~23k tokens of pure schema overhead before a single user turn. On a long agent run that's not a rounding error; it's most of your context budget and a real line on the bill.

The lesson isn't "don't use big servers." It's that the cost is invisible at connect time and none of the directories that rank these servers by stars/installs will tell you. You find out from the invoice.

While I had the packages open, I also scanned what they do

Same 25 servers, static source scan (nothing executed): network egress, secret-env reads, exec/shell surface. A few things worth knowing:

16 of 25 read a secret env var; 8 have an exec/shell surface. upstash-context7-mcp reads OPENAI_APPS_CHALLENGE_TOKEN + CLIENT_IP_ENCRYPTION_KEY and egresses login.microsoftonline.com.
4 of 25 ship a bundled/minified dist (exa has 293k-char lines; notion 582k). You can't statically separate a minified blob into call-site hosts vs embedded strings, so I label those bundled — egress not statically resolvable rather than render a misleading "clean." exa's bundle references an undisclosed api.agnost.ai alongside api.exa.ai.

Getting the egress scan honest at 25 servers was most of the work — naive scans count comment/jsdoc URLs as egress (one server "phoned home" to wikipedia.org and rfc-editor.org... from doc-strings). A host only counts as egress if its URL literal sits at an actual network call site or a declared API base.

Measure your own

The context-tax above is server-side and fixed. Your own agent's per-session cost — which server's schemas are eating your window, what a run actually cost — you can measure directly:

npx @wartzar-bee/tokenscope --demo   # zero-setup: a sample cost report, no session needed
npx @wartzar-bee/tokenscope          # or point it at your most recent Claude Code session

And if you want to stop cost regressions before they ship, wartzar-bee/ci-guardrail gates token cost in CI the same way you'd gate a failing test.

Everything here is reproducible from published npm packages — npm pack the servers, run the scanner, spawn each for the live tools/list. Found a cell that's wrong? The method is open; correct it.

Part of wartzar-bee — tools for building and operating cost-efficient autonomous agents. Apache-2.0.

The cheapest way to try a CI cost gate: run it in warn mode for a week

wartzar-bee — Mon, 03 Aug 2026 09:00:47 +0000

The cheapest way to try a CI cost gate: run it in warn mode for a week

The objection I hear most about gating agent cost in CI isn't "I don't need it." It's "I'm not wiring up a new check that can fail my builds on day one." Fair. A gate that red-X's a PR the first afternoon it's installed — over a threshold nobody has calibrated yet — gets ripped back out by Friday.

So the way I actually roll one out now is: install it in report-only mode first, and let it be wrong for a week before it's allowed to block anything.

Concretely, that's three properties I'd look for in any cost gate, not just the one I work on:

1. A report-only mode that never fails the build. You want the cost delta to show up on every PR — the number, the top files, the trend — while the check itself stays green no matter what. That's the whole evaluation period: you're watching whether the gate's numbers track reality on your repo before you give it teeth. If a tool only ships in block-or-nothing form, you're calibrating in production against angry teammates.

2. A noise floor, so tiny diffs don't cry wolf. The commonest false positive for a percentage gate is a small base: a helper that goes from 10 tokens to 30 is +200%, and a naive gate fails the build over a fraction of a cent. You want an absolute floor — "only flag when the real increase is at least N tokens" — so the gate speaks up about the fixture someone pasted in, not about a two-line refactor.

3. A report that shows up even when there's no PR. Pushes to main, scheduled runs, manual dispatches — those have no PR to comment on, but they're exactly where a slow cost creep hides. A gate that also writes its table to the run summary (GitHub's $GITHUB_STEP_SUMMARY) keeps a cost record on every event, not just on pull requests.

The rollout that sticks: add it with enforcement off, watch the numbers for a sprint, set the threshold where it would have caught the two regressions you actually remember, then flip it to blocking. By then nobody argues with it, because they've watched it be right.

If you want that off the shelf: wartzar-bee/ci-guardrail is an Apache-2.0 GitHub Action (powered by @wartzar-bee/tokenscope) with a mode: warn that does exactly the report-only-never-fail thing, and it writes the same cost table to the Actions run summary on any event (push, schedule, manual) — not just on PRs — so the first PR you point it at can't break, and you decide when it earns the right to block.

The whole rollout is one step — report-only from the first PR:

# .github/workflows/agent-cost.yml
- uses: wartzar-bee/ci-guardrail@v1
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    mode: warn            # report the cost delta on every PR, never fail the build
    threshold-pct: 20      # the line it *would* flag — tune it, then flip mode off to enforce
    min-delta-tokens: 500  # noise floor: ignore tiny diffs so small bases don't cry wolf

For those of you who gate cost (or coverage, or bundle size) in CI: how long do you run a new gate in warn/report-only before you let it block — a week, a sprint, longer? Or do you just turn it to blocking on day one and eat the noise?

AutoGen's hidden token tax: why a 3-agent chat costs 15 what you expect

wartzar-bee — Thu, 30 Jul 2026 15:37:46 +0000

AutoGen's hidden token tax: why a 3-agent chat costs 15× what you expect

Cost-audit series, episode 2. This series began with an AI agent that burned 136M tokens overnight →.

AutoGen is Microsoft's multi-agent framework. It's genuinely good at orchestrating agents that hand off work to each other. But its default memory model has a cost shape that surprises almost every team that hits it in production.

This audit shows you exactly where the tokens go, with line numbers.

The setup: a 3-agent RoundRobin chat

The canonical AutoGen pattern is a RoundRobinGroupChat with N agents taking turns on a task. Here's the minimal version from the docs:

from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import MaxMessageTermination

planner  = AssistantAgent("planner",  model_client=client, system_message="You plan.")
coder    = AssistantAgent("coder",    model_client=client, system_message="You code.")
reviewer = AssistantAgent("reviewer", model_client=client, system_message="You review.")

team = RoundRobinGroupChat(
    [planner, coder, reviewer],
    termination_condition=MaxMessageTermination(max_messages=10),
)
await team.run(task="Build a web scraper for Hacker News.")

Three agents, 10 turns total (~3–4 turns each). Seems cheap. It isn't.

The default context: unbounded, per-agent

Every AssistantAgent gets its own UnboundedChatCompletionContext by default:

# autogen-agentchat/src/autogen_agentchat/agents/_assistant_agent.py, __init__ (L840)
if model_context is not None:
    self._model_context = model_context
else:
    self._model_context = UnboundedChatCompletionContext()

source

UnboundedChatCompletionContext.get_messages() returns self._messages — the full list, no cap, no truncation:

# autogen-core/.../model_context/_unbounded_chat_completion_context.py (a ~20-line file)
async def get_messages(self) -> List[LLMMessage]:
    """Get at most `buffer_size` recent messages."""
    return self._messages

source

(The docstring says "at most buffer_size" — that's a copy-paste artifact from BufferedChatCompletionContext. There is no buffer. It returns everything.)

The handoff tax: every agent sees every message

When an agent's turn arrives, on_messages_stream adds all incoming messages to its own context before calling the LLM:

# _assistant_agent.py, in on_messages_stream (STEP 1: "Add new user/handoff messages
# to the model context")
await self._add_messages_to_context(
    model_context=model_context,
    messages=messages,          # ← the full message_thread from the group manager
    ...
)

And _add_messages_to_context appends each one:

# _assistant_agent.py, static method _add_messages_to_context
await model_context.add_message(llm_msg)
...
await model_context.add_message(msg.to_model_message())

source

The group manager (BaseGroupChatManager) maintains a single _message_thread and appends every response to it:

# _base_group_chat_manager.py
self._message_thread: List[BaseAgentEvent | BaseChatMessage] = []
...
await self.update_message_thread(delta)   # called after every agent response

source

So at turn T, the agent receiving the baton gets T-1 messages added to its already-growing context. Its context now contains everything it has ever seen.

The math: O(N × T²) total tokens

Let's be precise. Define:

T = total turns in the conversation
N = number of agents
m = average tokens per message (system prompt + response, ~300 tokens is realistic for a coding task)

Each agent speaks every N turns. When agent i speaks on turn t, its context contains all t-1 prior messages (because it has been accumulating them since turn 1).

Tokens consumed by agent i on turn t:

context_tokens(t) = (t - 1) × m

Total tokens for agent i across all its turns (it speaks at turns N, 2N, 3N, … up to T):

Σ (kN - 1) × m  for k = 1 to T/N
≈ m × N × (T/N)² / 2
= m × T² / (2N)

Total tokens across all N agents:

N × m × T² / (2N) = m × T² / 2

The N cancels. Total cost scales as T² regardless of how many agents you add.

Worked example: 10 turns, 3 agents, 300 tokens/message

Turn	Agent	Context size (messages)	Tokens in this call
1	planner	0 prior + system	~300
2	coder	1 prior + system	~600
3	reviewer	2 prior + system	~900
4	planner	3 prior + system	~1,200
5	coder	4 prior + system	~1,500
6	reviewer	5 prior + system	~1,800
7	planner	6 prior + system	~2,100
8	coder	7 prior + system	~2,400
9	reviewer	8 prior + system	~2,700
10	planner	9 prior + system	~3,000
Total			~16,500 tokens

Naïve expectation (10 calls × 300 tokens each): 3,000 tokens

Actual: ~16,500 tokens — 5.5× more.

At 20 turns it's ~63,000 tokens vs 6,000 expected — 10.5× more.

At 30 turns: ~139,500 tokens vs 9,000 — 15.5× more.

The multiplier grows linearly with T. This is the same O(T²) shape as ConversationBufferMemory in LangChain — but AutoGen's version is per-agent, so it's easy to miss in per-call logs.

Why per-call logs hide this

If you're watching your LLM provider's per-call token counts, you see something like:

call 1:  300 tokens  ✓ cheap
call 2:  600 tokens  ✓ fine
call 3:  900 tokens  ✓ ok
...
call 10: 3,000 tokens  ← this one looks expensive

Each call looks like a modest increase. The cumulative total — 16,500 — only shows up when you sum across the run. Most observability dashboards show per-call costs, not per-run totals. The runaway is invisible until the bill arrives.

The fix: cap the context

AutoGen ships two bounded alternatives, named in the AssistantAgent class docstring (around L178 of _assistant_agent.py): BufferedChatCompletionContext (limits message count) and TokenLimitedChatCompletionContext (limits tokens):

Option 1: `BufferedChatCompletionContext` (sliding window)

from autogen_core.model_context import BufferedChatCompletionContext

coder = AssistantAgent(
    "coder",
    model_client=client,
    model_context=BufferedChatCompletionContext(buffer_size=5),  # last 5 messages
)

Cost shape becomes O(T × buffer_size) — linear. For buffer_size=5 and 30 turns: ~42,000 tokens vs 139,500 unbounded. 3.3× cheaper.

Option 2: `TokenLimitedChatCompletionContext` (token budget)

from autogen_core.model_context import TokenLimitedChatCompletionContext

coder = AssistantAgent(
    "coder",
    model_client=client,
    model_context=TokenLimitedChatCompletionContext(token_limit=2000),
)

Caps the context at a fixed token budget. More predictable than a message count because message sizes vary.

Which to use?

Scenario	Recommendation
Short tasks (≤10 turns)	Default is fine; monitor cumulative cost
Long tasks (>10 turns)	`BufferedChatCompletionContext(buffer_size=8–12)`
Strict cost budget	`TokenLimitedChatCompletionContext(token_limit=N)`
Need full history	Default + add per-run cost alerting (see below)

Detecting this in CI before it hits production

The pattern is detectable statically: any file that instantiates AssistantAgent without a model_context= argument is using the unbounded default.

# Flag unbounded AssistantAgent instantiations
grep -rn "AssistantAgent(" src/ | grep -v "model_context="

For dynamic detection — measuring actual token growth across a run — this is exactly what tokenscope does: it instruments LLM calls, tracks per-run cumulative cost, and can block a CI build when a PR's token delta exceeds a threshold.

The wartzar-bee/ci-guardrail GitHub Action wraps tokenscope into a one-line workflow addition:

- uses: wartzar-bee/ci-guardrail@v1
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    threshold-pct: 20   # block if the PR's predicted token cost rises >20% vs base

Summary

	Naïve expectation	Actual (unbounded)	With BufferedContext(5)
10 turns, 3 agents	3,000 tokens	~16,500 tokens	~12,000 tokens
20 turns, 3 agents	6,000 tokens	~63,000 tokens	~27,000 tokens
30 turns, 3 agents	9,000 tokens	~139,500 tokens	~42,000 tokens

The default UnboundedChatCompletionContext is correct for short tasks and full-history use cases. It becomes a cost trap in long multi-agent conversations. The fix is one constructor argument — but you have to know to add it.

The broader pattern: every major agent framework defaults to unbounded context because it's the safest correctness choice. Cost is a second-class citizen in the default config. That's the gap this series documents.

Next in the series: CrewAI — the delegation overhead. How hierarchical agent trees multiply your token bill.

tokenscope on npm · wartzar-bee/ci-guardrail · @wartzarbee on dev.to

Gate your AI agents' token cost in CI — before the bill, not after

wartzar-bee — Thu, 30 Jul 2026 14:59:43 +0000

A multi-agent run doesn't fail loudly when it gets expensive. It just quietly costs more.

One extra delegation loop, a tool that dumps a big blob into shared context, an agent that re-reads the whole history every turn — and the pipeline step that cost a few cents last week costs a few dollars this week. Nothing broke, so nothing warned you. You find out after the invoice, not before the run.

We already gate build time, bundle size, and test coverage in CI, and we fail the PR when they regress. The token/compute cost of the LLM steps themselves is the one budget we still tend to discover on the monthly bill. Here are three habits that have actually helped, then the concrete CI piece I was least sure how to do — and now do on every repo that runs agents.

1. Budget the prompt before the run, not after

Before kicking off an agent, look at what it actually carries into context: system prompt, tool schemas, accumulated task output. If a single agent walks in with 15k tokens of boilerplate it never uses, that's a fixed tax on every turn it takes. Trim it once, save it every run.

A zero-setup way to see the shape of a real session's cost — output vs cached re-reads vs cache writes — without wiring anything up:

npx @wartzar-bee/tokenscope --demo        # sample cost report, no logs needed
npx @wartzar-bee/tokenscope               # your most recent Claude Code session

2. Attribute tokens per source, not per message

"This run used 400k tokens" tells you nothing actionable. "This run used 400k tokens and 60% of them are the same three prior task outputs re-sent on every agent handoff" tells you exactly what to fix — cache it, summarize it, or stop broadcasting it. Attribution is what turns a scary number into a diff.

3. Gate the cost in CI the way you'd gate build time

This is the one that changed how I think about it. A run that suddenly costs 3x more is a regression, the same as a build that suddenly takes 3x longer — it should be able to fail a check, not surprise you in prod.

The tricky part is estimating a PR's cost statically, before you spend it. tokenscope scan does a pre-run token estimate over your agent code:

npx @wartzar-bee/tokenscope scan --dir .   # static, pre-run token estimate

and there's a GitHub Action that wraps it, comments the delta on the files responsible, and can block the PR on a regression. The honest way to adopt it is mode: warn first — it posts the same verdict and flags the breach in the Actions log, but never fails the build, so you can watch what it would block for a couple of weeks before turning enforcement on:

# .github/workflows/agent-cost.yml
name: Agent cost guardrail
on: pull_request
permissions:
  contents: read
  pull-requests: write        # so it can comment the delta on the PR
jobs:
  cost:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0       # needs the base branch to diff against
      - uses: wartzar-bee/ci-guardrail@v1
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          threshold-pct: 20    # flag if predicted tokens grow >20% vs base
          max-usd: 0.50        # hard dollar ceiling per run (0 = off)
          model: sonnet        # price the delta at the right $/1M
          mode: warn           # report only; switch to 'block' when you trust it

When you're ready to enforce, drop mode: warn (it defaults to block) and the PR can't merge until the regression is addressed. A couple of inputs worth knowing:

min-delta-tokens (default 100) is a noise floor so a tiny base — 10 → 30 tokens, +200% — doesn't fail a build over a fraction of a cent. That's the commonest cost-guardrail false positive, off by default-sane.
max-tokens / max-usd are absolute ceilings that catch a codebase that's already over budget and creeping up by a small percentage each PR — something a percentage gate alone can never flag.

Both the CLI (@wartzar-bee/tokenscope) and the Action (wartzar-bee/ci-guardrail) are Apache-2.0 and run locally / in your own runner — nothing phones home, it's reading your session logs and code. They're early and I'm actively working on them, so if you gate agent cost some other way, or the estimate is off for your setup, I genuinely want to hear it.

For those of you running agents in a real pipeline — do you gate the cost anywhere, or watch a dashboard and hope? What's caught you out?

smolagents replays its whole memory every step: the O(n ) token bill nobody mentions

wartzar-bee — Wed, 29 Jul 2026 04:37:00 +0000

smolagents replays its whole memory every step: the O(n²) token bill nobody mentions

Cost-audit series, episode 5. This series began with an AI agent that burned 136M tokens overnight →.

smolagents is Hugging Face's deliberately small agent framework — a few thousand lines, "no abstraction on top of abstraction," and 28k+ GitHub stars. Its CodeAgent is genuinely elegant: the model writes Python, the runtime executes it, the result comes back, repeat until a final answer.

The elegance hides a cost curve. A smolagents run that takes n reasoning steps does not cost n times a single step. On input tokens it costs closer to n²/2, because every step re-sends the entire accumulated memory of every previous step. The default step budget is 20. A task that genuinely needs a dozen tool calls quietly sends the model its own transcript a dozen times over.

This audit shows the exact lines, gives you a formula you can evaluate on your own workload, and shows the one-hook fix.

Where the tokens go

Every step, the agent rebuilds the full message list it sends to the model. Here is the method that does it, verbatim (agents.py:758-770, v1.26.0):

def write_memory_to_messages(
    self,
    summary_mode: bool = False,
) -> list[ChatMessage]:
    """
    Reads past llm_outputs, actions, and observations or errors from the memory into a series of messages
    that can be used as input to the LLM. Adds a number of keywords (such as PLAN, error, etc) to help
    the LLM.
    """
    messages = self.memory.system_prompt.to_messages(summary_mode=summary_mode)
    for memory_step in self.memory.steps:
        messages.extend(memory_step.to_messages(summary_mode=summary_mode))
    return messages

Read the loop: it walks every entry in self.memory.steps and appends its messages. memory.steps only ever grows — it is a plain list initialised empty and appended to, never trimmed, except by an explicit reset() between runs (memory.py:230, memory.py:232-234).

That method is called at the top of every action step, with no summary_mode, so the full history is replayed each time (agents.py:1284-1286):

memory_messages = self.write_memory_to_messages()
...
input_messages = memory_messages.copy()

The default ceiling on how many times this can happen is 20 (agents.py:300, max_steps: int = 20). Nothing in the default path caps or summarises the growing memory — summary_mode=True is used only for planning messages (agents.py:684, agents.py:886), not for the main action loop.

The math (evaluate it on your own numbers)

Let:

P = tokens in the system prompt (fixed, sent every step)
s = tokens each completed step adds to memory — the model's code/thought plus the tool observation it produced

At step k (1-indexed) the input the model receives is P + (k-1)·s — the prompt plus everything the previous k-1 steps left behind. Summed over an n-step run, cumulative input tokens are:

Σ (k=1..n) [ P + (k-1)·s ]  =  n·P  +  s · n(n-1)/2

The s · n(n-1)/2 term is quadratic in n. Compare it to the intuition most people price with — "n steps ≈ n × one step" = n·(P + s). The history you re-pay for is n(n-1)/2 · s instead of n · s:

Steps n	History replays (× `s`), naive	History replays (× `s`), actual	Overpay factor
4	4	6	1.5×
8	8	28	3.5×
12	12	66	5.5×
20 (max)	20	190	9.5×

(Table is illustrative of the formula above — it is the closed form n(n-1)/2 vs n, not a measured run. Plug in your own P and s to get dollars.)

The observation size s is where it bites hardest. If a tool returns a chunk of a web page, a file, or a dataframe, that payload is now re-sent on every subsequent step for the rest of the run. Long, tool-heavy tasks are exactly the ones that hit max_steps, so the worst tasks pay the worst multiplier.

Good news: smolagents already measures this for you. Every ActionStep carries a token_usage field with input/output token counts (memory.py:63). After a run, sum step.token_usage.input_tokens across agent.memory.steps and you will see the curve directly. The problem is that by the time you read it, you have already paid.

The fix: prune the replayed history with a step callback

smolagents gives you the exact hook you need. The agent accepts step_callbacks — callables invoked at the end of each step, and you can register them per step-type (agents.py:282, agents.py:304, wired in _setup_step_callbacks, agents.py:416-425). Because memory.steps is just a list you own, a callback can cap how much history survives into the next write_memory_to_messages call:

from smolagents import CodeAgent, ActionStep

KEEP_LAST = 6  # replay only the most recent N action steps

def trim_memory(step, agent):
    action_steps = [s for s in agent.memory.steps if isinstance(s, ActionStep)]
    for stale in action_steps[:-KEEP_LAST]:
        # collapse the bulky observation; keep a short marker so the model
        # still knows the step happened
        stale.observations = "[trimmed to control context cost]"

agent = CodeAgent(tools=[...], model=..., step_callbacks=[trim_memory])

(Illustrative usage of the real step_callbacks API — tune KEEP_LAST and what you collapse to your task. The point is that the hook is first-class, not that these exact lines ship in the library.)

This turns the input curve from quadratic back toward linear: a fixed window of history instead of an ever-growing one. You trade some long-range recall for a bounded bill — for most tool-loop tasks that is the right trade, and you make it deliberately instead of discovering it on an invoice.

Other levers, in order of bluntness: lower max_steps from the default 20 so a wandering run can't rack up 190× history replays; truncate large tool return values before they enter memory; and use the planning/summary path smolagents already has for long-horizon tasks.

See the bill before you run it

The pattern in this series is always the same: the framework is fine, the default is expensive, and the cost is invisible until it shows up on the invoice. smolagents is the most honest case yet — it even hands you token_usage — but you still have to run the task, at full quadratic cost, to see it.

That is the gap tokenscope closes — it shows what a run actually cost, and estimates a source tree's token footprint before you spend it. See a real cost breakdown in five seconds, no setup or logs required:

npx @wartzar-bee/tokenscope --demo

Then run it on your own most-recent Claude Code session (just npx @wartzar-bee/tokenscope), or estimate a directory's token cost before a run — the static check that powers the guardrail:

npx @wartzar-bee/tokenscope scan --dir .

If you want that check enforced automatically — a bot that comments the predicted token-cost delta on the responsible files in every pull request and can block a regression — that is what we build the ci-guardrail GitHub Action for. It's one copy-paste step — no account, no proxy:

# .github/workflows/agent-cost.yml
- uses: wartzar-bee/ci-guardrail@v1
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    threshold-pct: 20   # fail the PR if token cost grows >20% vs the base branch

Next in the series: we turn the audits into a checklist — the five context-cost anti-patterns that show up in almost every agent framework, and the one-line review question that catches each. Follow @wartzarbee so you don't miss it.

Found an error in this audit? The whole point is that every number is reproducible — reply with the line and I'll fix it in public.

LangGraph isn't cheaper than LangChain — unless you opt out of its defaults

wartzar-bee — Tue, 28 Jul 2026 09:56:41 +0000

LangGraph isn't cheaper than LangChain — unless you opt out of its defaults

Cost-audit series, episode 4. This series began with an AI agent that burned 136M tokens overnight →.

When LangChain deprecated ConversationBufferMemory (the subject of episode 1 in this series), the official migration path was LangGraph. The pitch: explicit state management, you control exactly what flows where. More expressive, more controllable.

It is — but only if you reach for the controls. The default state model in LangGraph has the same unbounded-growth problem as the memory it replaced. Teams migrating to escape ConversationBufferMemory's cost curve often land on an identical curve, with new graph complexity on top.

This audit shows exactly where the default grows, what it costs, and what opt-outs exist.

The default: `MessagesState` + `add_messages`

The quickstart in LangGraph's own docs uses this pattern:

from langgraph.graph import StateGraph, MessagesState

def my_node(state: MessagesState):
    messages = state["messages"]
    response = llm.invoke(messages)   # sends ALL messages to the LLM
    return {"messages": [response]}

graph = StateGraph(MessagesState)
graph.add_node("agent", my_node)

MessagesState is a TypedDict with a single key, messages, backed by the add_messages reducer. Here's what that reducer does:

# langgraph/graph/message.py — add_messages (def at line 18; merge loop below)
def add_messages(left: Messages, right: Messages) -> Messages:
    # ... (coerces left/right to lists of BaseMessage) ...
    left_idx_by_id = {m.id: i for i, m in enumerate(left)}
    merged = left.copy()
    ids_to_remove = set()
    for m in right:
        if (existing_idx := left_idx_by_id.get(m.id)) is not None:
            if isinstance(m, RemoveMessage):
                ids_to_remove.add(m.id)
            else:
                merged[existing_idx] = m      # same id → update in place
        else:
            merged.append(m)                  # new id → APPEND (the list grows)
    merged = [m for m in merged if m.id not in ids_to_remove]
    return merged

Source: langgraph/graph/message.py

This is not a summarizer, not a window, not a trimmer. It is an append-only list. Every message ever added to state stays in state — and every node that reads state["messages"] sees the full list.

This is ConversationBufferMemory with a graph wrapper.

The cost math

Assume a conversational agent: 10 turns, 150 tokens per user message, 200 tokens per assistant reply (modest — a short answer each time).

After 10 turns, state["messages"] contains 20 messages = (10 × 150) + (10 × 200) = 3,500 tokens of accumulated history.

For the 11th call, the node sends all 3,500 tokens of prior history as context, then generates a new reply. Each further turn adds another 350 tokens (150 user + 200 assistant), so the 12th call sends 3,850, the 13th 4,200, and so on.

Total input tokens for a 20-turn conversation:

Turn	Messages in state	Input tokens (messages + system)
1	0 prior	150 + 400 (system)
5	4 prior turns	1,550 + 400
10	9 prior turns	3,300 + 400
15	14 prior turns	5,050 + 400
20	19 prior turns	6,800 + 400
Total		~77,500 tokens input

(Each call = 400 system + 150 current user + (turn−1) × 350 accumulated history.)

A naive estimate (flat 550 tokens/call × 20 calls) = 11,000 tokens.

Actual with add_messages default = ~77,500 tokens. 7× over.

With claude-haiku-4-5 ($0.80/M input, $4/M output) for a chatbot doing 500 conversations/day:

Naive estimate: 11,000 × 500 × 30 × $0.80/M = $132/month
Actual: 77,500 × 500 × 30 × $0.80/M = $930/month

That's $798/month of silent overspend on input tokens alone, just from the default accumulation — before you add nodes, tools, or memory.

Multiplier 1: multi-node graphs (each node pays the full state)

LangGraph's value over a simple chat loop is composing multiple nodes — a router, a tool-caller, a summarizer, a responder. Each node that reads state["messages"] pays the full token cost of the accumulated message list.

graph = StateGraph(MessagesState)
graph.add_node("router", route_node)      # reads state["messages"]
graph.add_node("tool_caller", tool_node)  # reads state["messages"]
graph.add_node("responder", respond_node) # reads state["messages"]

For a 3-node graph where each node reads messages, a single user turn that passes through all three nodes costs 3× the message-list tokens. After 10 turns with 3,500 accumulated tokens, one user message costs: 3 × 3,500 = 10,500 tokens just for message history, before any node-specific prompts.

Multiplier 2: `interrupt_before` / `interrupt_after` (human-in-the-loop)

LangGraph's human-in-the-loop feature pauses graph execution at a node boundary. When the graph resumes, it deserializes the full checkpointed state and re-injects it into the next node:

# langgraph/pregel/__init__.py — Pregel.astream (v0.2.60), the entrypoint
# that drives interruptible execution. Verbatim signature:
async def astream(
    self,
    input: Union[dict[str, Any], Any],
    config: Optional[RunnableConfig] = None,
    *,
    stream_mode: Optional[Union[StreamMode, list[StreamMode]]] = None,
    output_keys: Optional[Union[str, Sequence[str]]] = None,
    interrupt_before: Optional[Union[All, Sequence[str]]] = None,
    interrupt_after: Optional[Union[All, Sequence[str]]] = None,
    debug: Optional[bool] = None,
    subgraphs: bool = False,
) -> AsyncIterator[Union[dict[str, Any], Any]]:
    ...

Source: langgraph/pregel/__init__.py — search the file for async def astream (defined ~line 1683; interrupt_before/interrupt_after are the pause controls). When a run resumes after an interrupt, Pregel reloads the pending state from the checkpointer (the aget_tuple/aget_state path returns the full checkpoint blob — every message included) before continuing at the next node.

The cost: full state deserialization on every resume. If a workflow interrupts 3 times before completion (a common approval flow), and the state has 5,000 tokens of messages, the resumption overhead alone is 3 × 5,000 = 15,000 extra tokens — paid every time, even if the approval is just a "yes."

Multiplier 3: parallel fan-out (`Send` API)

LangGraph's Send API dispatches parallel subgraph invocations, each receiving a copy of state:

from langgraph.types import Send

def fanout_node(state: MessagesState):
    return [
        Send("worker_a", {"messages": state["messages"], "task": "summarize"}),
        Send("worker_b", {"messages": state["messages"], "task": "critique"}),
        Send("worker_c", {"messages": state["messages"], "task": "expand"}),
    ]

Source: langgraph/types.py

Each Send carries the full state["messages"] to the worker node. With 3 workers and 5,000 tokens of history: 15,000 tokens dispatched in the fan-out alone. If those workers themselves call an LLM, each call pays the full 5,000-token history again. Compare to the CrewAI quadratic problem from episode 3 — this is the same failure mode, different API.

The opt-outs (LangGraph actually provides them)

Unlike ConversationBufferMemory (which had no good trim story), LangGraph ships built-in tools to fix this. Teams just don't use them by default.

Trim messages before every LLM call

from langchain_core.messages import trim_messages

def my_node(state: MessagesState):
    trimmed = trim_messages(
        state["messages"],
        max_tokens=2000,          # hard cap
        strategy="last",          # keep the most recent
        token_counter=llm,        # use the model's tokenizer
        include_system=True,      # always keep the system message
        allow_partial=False,
    )
    response = llm.invoke(trimmed)   # sends trimmed history, not full list
    return {"messages": [response]}

Source: langchain_core/messages/utils.py

This keeps the full history in state (for checkpointing, human inspection) while capping what the LLM actually sees. Applying a 2,000-token cap on a 20-turn conversation reduces input tokens from ~77,500 to ~40,000 (2,000 tokens × 20 calls). ~48% cost reduction, one line change.

Pass only what the node needs

Instead of giving every node the full state["messages"], scope what each node receives:

class MyState(TypedDict):
    messages: Annotated[list[BaseMessage], add_messages]
    last_tool_result: str        # structured, compact
    task_description: str        # set once, doesn't grow

def tool_caller_node(state: MyState):
    # This node only needs the task + last result — not the full conversation
    prompt = f"Task: {state['task_description']}\nContext: {state['last_tool_result']}"
    response = llm.invoke(prompt)
    return {"last_tool_result": response.content}

The tool_caller_node never touches state["messages"] — it pays zero for message accumulation. Only the nodes that genuinely need conversational context receive it.

Summarize periodically (the right way)

def maybe_summarize(state: MessagesState):
    messages = state["messages"]
    if len(messages) > 20:                # threshold: tune to your cost tolerance
        summary = llm.invoke([
            SystemMessage("Summarize this conversation in 3 sentences."),
            *messages,
        ])
        return {
            "messages": [
                SystemMessage(f"Conversation summary: {summary.content}"),
                messages[-2],    # keep the last human message
                messages[-1],    # keep the last assistant message
            ]
        }
    return {}   # no change needed

This collapses the accumulated history into a single system message at the summarization trigger. After summarization, the effective context is ~200 tokens (summary + last exchange) instead of 3,500+. Insert maybe_summarize as an always-on node before expensive LLM calls.

Measuring your own graph

LangGraph's built-in tracing (via LangSmith) shows per-node token usage, but it's behind a paid tier for production volumes. For a free alternative that works with any JSONL export:

npm install -g @wartzar-bee/tokenscope
npx @wartzar-bee/tokenscope --demo   # instant sample, no setup
npx @wartzar-bee/tokenscope langgraph-session.jsonl

Output shows the session total, how much of each call is re-sent accumulated context versus new work, and where the growth is steepest — the same view that surfaced the 136M-token burn in episode 1.

Summary

LangGraph's MessagesState + add_messages default is ConversationBufferMemory under a new name. The graph model gives you explicit controls that the old memory API lacked — but the controls are opt-in. Without trim_messages, selective state reads, or periodic summarization, migration from LangChain to LangGraph buys you graph expressiveness at the same (or higher, for multi-node graphs) token cost.

Pattern	Cost impact	Fix
`MessagesState` default	7× over naive estimate at 20 turns	`trim_messages` before every LLM call
Multi-node graph, all nodes read messages	N× multiplier (N = node count)	Scope state: only pass what each node needs
`interrupt`/resume	Full state re-injected on every resume	Summarize before checkpointing at long sessions
`Send` fan-out	Parallel full-state copies	Pass minimal substate to each worker

The migration from LangChain to LangGraph is worth it — but only once you understand and explicitly opt out of these defaults. Otherwise you're paying for the complexity without the savings.

Once you've opted out of the expensive defaults, the way to keep them out is to measure cost on every change. tokenscope gives you the per-run number locally (npx @wartzar-bee/tokenscope --demo), and our ci-guardrail GitHub Action fails a pull request when a change pushes token cost past a budget you set — so a silent 7× regression like this one gets caught in review, not on the invoice.

It's one copy-paste step in a workflow — no account, no proxy:

# .github/workflows/agent-cost.yml
- uses: wartzar-bee/ci-guardrail@v1
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    threshold-pct: 20   # fail the PR if token cost grows >20% vs the base branch

wartzar-bee builds tools for operating cost-efficient autonomous agents: tokenscope for token-cost measurement, ci-guardrail to gate cost regressions in CI, and enclave — the open-source sandboxed runtime we run our own agent fleet on. All free and open-source. Follow on dev.to →

CrewAI's quadratic context problem: why a 5-agent crew costs 6 more than you expect

wartzar-bee — Sun, 26 Jul 2026 03:55:49 +0000

CrewAI's quadratic context problem: why a 5-agent crew costs 6× more than you expect

Cost-audit series, episode 3. This series began with an AI agent that burned 136M tokens overnight →.

CrewAI is one of the most-starred agent orchestration frameworks on GitHub. Its pitch is intuitive: define a crew of role-playing agents, assign tasks, watch them collaborate. What the README doesn't tell you is that the default context-passing model has a quadratic token cost curve. A 5-agent crew doesn't cost 5× a solo agent — on input tokens it costs closer to 6×, and once you stack memory, delegation, and verbose mode it climbs to 8–15×. The multiplier grows with every agent or task you add.

This audit shows you exactly where the tokens go, with line numbers.

The architecture in one paragraph

CrewAI executes tasks sequentially (by default). Each task has an optional context list — a list of other tasks whose outputs should be available to the executing agent. When you don't specify context explicitly, CrewAI's default behavior is to make all previously completed tasks available to each subsequent agent. The output of Task 1 goes into Task 2's prompt. Task 1 + Task 2 outputs go into Task 3's prompt. And so on.

This is linear accumulation — and it produces a quadratic total token count.

The code that does it

src/crewai/crew.py — the Crew._get_context method (line 792 at tag 0.80.0):

def _get_context(self, task: Task, task_outputs: List[TaskOutput]):
    context = (
        aggregate_raw_outputs_from_tasks(task.context)
        if task.context
        else aggregate_raw_outputs_from_task_outputs(task_outputs)
    )
    return context

Source: src/crewai/crew.py#L792

The decisive branch is the else: when a task sets no explicit context, CrewAI falls back to aggregate_raw_outputs_from_task_outputs(task_outputs) — the outputs of all prior tasks. That aggregation helper (in crewai/utilities) serializes each prior task's full raw output into the string that gets injected into the current task's prompt. There is no summarization, no truncation, no deduplication. The string grows with every task.

src/crewai/crew.py — the task execution loop lives in Crew._execute_tasks (defined at line 635 at tag 0.80.0; the context-injection call is at line 696). Simplified to the two lines that matter:

# inside _execute_tasks, iterating over the crew's tasks:
context = self._get_context(task, task_outputs)   # task_outputs = every prior task's output
task_output = task.execute_sync(
    agent=agent_to_use, context=context, tools=agent_to_use.tools
)

Source: src/crewai/crew.py (method _execute_tasks)

task_outputs is the running list of every prior task's output, so it grows as the crew progresses. Each call to execute_sync constructs a full prompt that includes that accumulated context string, and it is sent with every LLM API request — not cached between tasks (CrewAI doesn't use prompt caching by default).

The math

Assume:

N = 5 tasks (a typical research + writing crew)
T_task = 500 tokens per task output (modest: a paragraph or a JSON object)
T_system = 800 tokens per agent's system prompt + task description (role, goal, backstory, task instructions)

Token count for Task k = T_system + (k-1) × T_task (accumulated context from all prior tasks)

Task	Context tokens	System + task	Total input tokens
1	0	800	800
2	500	800	1,300
3	1,000	800	1,800
4	1,500	800	2,300
5	2,000	800	2,800
Total	5,000	4,000	9,000

A naive model would predict 5 × 800 = 4,000 input tokens. The actual bill is 9,000 — 2.25× for just five tasks with modest outputs. Now scale up:

T_task = 2,000 tokens (a realistic research output, a code block, a structured list):

Task	Context tokens	System + task	Total input tokens
1	0	800	800
2	2,000	800	2,800
3	4,000	800	4,800
4	6,000	800	6,800
5	8,000	800	8,800
Total	20,000	4,000	24,000

Now you're at 6× the naive estimate — and that's input tokens only. Add output tokens from each task (another ~2,000 × 5 = 10,000) and the total API cost for one crew run is 34,000 tokens instead of the ~14,000 you'd expect.

With claude-sonnet-4-6 ($3/M input, $15/M output):

Naive estimate: (4,000 × $3 + 10,000 × $15) / 1,000,000 = $0.162
Actual: (24,000 × $3 + 10,000 × $15) / 1,000,000 = $0.222 — 37% over

Now run 50 crew executions per day:

Naive: $0.162 × 50 × 30 = $243/month
Actual: $0.222 × 50 × 30 = $333/month

For a SaaS product with hundreds of daily crew runs, this gap becomes tens of thousands of dollars per month — and it gets worse as you add agents or as task outputs grow.

Three multipliers on top of the base cost

1. Memory layers (each one adds tokens)

CrewAI's memory=True flag (off by default, but heavily promoted) activates four memory systems:

# src/crewai/memory/short_term/short_term_memory.py
class ShortTermMemory(Memory):
    def search(self, query: str, score_threshold: float = 0.35):
        return self.storage.search(query=query, score_threshold=score_threshold)

Source: src/crewai/memory/short_term/short_term_memory.py

Short-term memory retrieves every stored item scoring above a similarity threshold (default 0.35) and appends them to the prompt — the count is unbounded, so a longer run history injects more retrieved context per task. Long-term memory hits an SQLite database. Entity memory maintains structured entity descriptions. Semantic memory uses ChromaDB embeddings — each retrieval costs an embedding API call plus the tokens from the retrieved chunks injected into the prompt.

With memory=True on a 10-run crew history, expect +1,500–3,000 tokens per task in retrieval overhead.

2. `verbose=True` (the default in most tutorials)

Most CrewAI tutorials set verbose=True or verbose=2. This surfaces the agent's intermediate reasoning — but the real cost driver isn't the logging, it's the ReAct loop underneath it. CrewAI runs its own CrewAgentExecutor (not LangChain's), and each agent step accumulates the full "Thought / Action / Observation" trace back into the LLM prompt for the next iteration. Each tool call adds another round of Thought + Action + Observation tokens before the final answer. For an agent that calls 3 tools, this can add 800–2,000 tokens per task.

3. Human delegation (`allow_delegation=True`)

When an agent can delegate to another, it can route subtasks to specialist agents mid-task. This is CrewAI's "hierarchical" feature. The cost: each delegation creates a new complete LLM call with the delegating agent's accumulated context inherited into the delegatee's prompt. A single task can spawn 2–3 delegation chains, each carrying the full context blob from above.

What a real run looks like

Let's measure a concrete 3-agent research crew using @wartzar-bee/tokenscope:

npx @wartzar-bee/tokenscope --demo   # instant sample, no setup
npx @wartzar-bee/tokenscope crew-session.jsonl

For a crew with:

Researcher agent: finds + summarizes 3 sources (outputs ~1,800 tokens)
Analyst agent: interprets the research (outputs ~1,200 tokens)
Writer agent: produces a structured report (outputs ~2,000 tokens)

The per-agent context accumulation (illustrative — tokenscope reports the session total you can check this against, not a per-agent split):

Per-agent context accumulation:
  researcher  →  input:  1,100  output:  1,800   (system + task only)
  analyst     →  input:  2,900  output:  1,200   (+1,800 context from researcher)
  writer      →  input:  4,100  output:  2,000   (+1,800 + 1,200 context from prior two)

  Total input:    8,100
  Total output:   5,000
  Session total: 13,100 tokens

  Naive estimate (no context accumulation): 8,300 tokens
  Actual multiplier: 1.58×

This is a modest crew. Bump to 5 agents with research-heavy outputs and the multiplier reaches 4–6×. Add memory and delegation: 8–15×.

The fix: explicit context scoping

CrewAI lets you control which tasks feed context to which. Use the context parameter explicitly:

from crewai import Task

research_task = Task(
    description="Find the top 3 cloud cost optimization techniques.",
    agent=researcher,
    # no context — this is the first task
)

analysis_task = Task(
    description="Analyze the techniques and rank by ROI.",
    agent=analyst,
    context=[research_task],  # only research output, not all prior tasks
)

writing_task = Task(
    description="Write a 500-word summary of the #1 technique.",
    agent=writer,
    context=[analysis_task],  # only the analysis — researcher output not needed here
)

Result: the writer agent sees only the analyst's output (~1,200 tokens), not the researcher's raw output + the analyst's output (3,000 tokens). Context tokens halved on the most expensive task.

For longer pipelines, consider a summary task: a cheap, short-output task that condenses prior results, and only its output flows forward. The cost of the summarization step is far less than sending raw outputs through N subsequent agents.

Measuring your own crew

If you're running CrewAI in production, the default LLM logging doesn't surface how much of each call is re-sent accumulated context. Point tokenscope at a session transcript to see it:

npm install -g @wartzar-bee/tokenscope

Export your run as a session JSONL (a Claude Code session under ~/.claude/projects, or any transcript in that format), then:

npx @wartzar-bee/tokenscope crew-session.jsonl
# machine-readable:
npx @wartzar-bee/tokenscope crew-session.jsonl --json

You'll see the session total, how much is new work versus re-sent accumulated context, and where the growth is steepest — the accumulation this whole post is about.

Summary

CrewAI's default context model accumulates all prior task outputs into every subsequent agent's prompt. This produces a quadratic total token count as N grows — not linear. The practical impact at modest scale (5 agents, 2,000-token outputs): 4–6× the token count you'd expect. Add memory layers, delegation, and verbose mode: 8–15×.

The fix is explicit context scoping: pass only the task outputs that each agent actually needs. It's a one-line change per task, and it can halve your API bill immediately.

Next in the series: we'll look at LangGraph's token footprint — the stateful graph model has a different cost shape, and it's worth understanding before you migrate from LangChain to LangGraph chasing efficiency gains.

wartzar-bee builds tools for operating cost-efficient autonomous agents. tokenscope is free and open-source. Follow on dev.to →

Put a token-cost gate on your AI-agent PRs in 5 minutes

wartzar-bee — Sat, 25 Jul 2026 08:41:46 +0000

If you run an AI agent, your token bill lives in your source code — in prompts, tool
descriptions, and how much context you re-send every turn. And source code changes in pull
requests. So the natural place to catch a cost regression is the same place you catch a bug:
in CI, on the PR, before it merges.

I learned this the expensive way. We once
put an agent on a timer and it burned 136M tokens overnight doing almost nothing.
That was the dramatic end. The everyday end is quieter: a system prompt that grew by 200
tokens, a context window that stopped being trimmed, a new tool whose description is 800 words
long. None of it shows up in a code review — the diff looks fine. The tokens are invisible.

This is a 5-minute walkthrough to make them visible: a GitHub Action that estimates the
token-cost delta of a PR, comments the responsible files on the PR, and can fail the build if
the cost jumps past a threshold you set.

The one file you add

Drop this into .github/workflows/cost-guardrail.yml:

name: Cost Guardrail
on: [pull_request]

jobs:
  cost-check:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write   # needed to post the comment
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0       # the action compares HEAD vs the base branch

      - uses: wartzar-bee/ci-guardrail@v1
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          threshold-pct: 20        # block if tokens grow >20% vs base
          working-directory: .     # where your agent code lives

That's the whole setup. No account, no API key beyond the GITHUB_TOKEN your repo already
has, no service to sign up for. fetch-depth: 0 matters: the action checks out both branches
to diff them, so it needs the git history, not just the tip commit.

What you get on the next PR

Here's a comment from a real run of the action — a contributor added a few-shot block to an
agent's system_prompt.txt, which re-sends on every turn. These are genuine tokenscope
numbers, not a mock-up; the action updates the same comment on each push, so it never spams
the thread:

## 🚨 wartzar-bee Cost Guardrail

| Metric             | Value      |
|--------------------|------------|
| Base branch tokens | 978        |
| This PR tokens     | 1,512      |
| Delta              | **54.6%** (**+$0.0016**) |
| Threshold          | 20%        |

💵 Cost estimated at $3.00/1M tokens — set `price-per-1m-tokens` to your model's price.

### Biggest cost increases (responsible files)
| File                     | Base | Head  | Δ        |
|--------------------------|-----:|------:|---------:|
| agent/system_prompt.txt  | 962  | 1,496 | **+534** |

> ⛔ Build blocked — cost regression exceeds the 20% threshold.
> Reduce prompt size, add caching, or raise the threshold if intentional.

The value isn't the top number — it's the responsible-files table. It turns "the bill went
up" into "the bill went up because system_prompt.txt grew 534 tokens." That's a comment a
reviewer can act on in the PR, not a surprise on the invoice. (The per-PR dollar figure is
small; the point is it re-sends every turn, every day, across your fleet — the percentage is
what gates the build.)

Start in report-only mode

Failing builds on day one is a good way to get an Action deleted. Start by measuring, not
blocking. Set the threshold to 0 and it always comments, never fails:

- uses: wartzar-bee/ci-guardrail@v1
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    threshold-pct: 0   # report only, never block

Let it run on real PRs for a week. You'll see what a "normal" delta looks like for your repo —
maybe +5% is routine and +40% is the one worth stopping. Then set threshold-pct to a number
that reflects that, and turn on blocking with intent instead of guessing.

Make the dollar figure yours

The comment shows an estimated dollar delta so a non-engineer reading the PR gets it. By default
it uses an approximate blended input-token price of $3.00 per 1M tokens. That's a placeholder —
override it to your model so the number is real:

- uses: wartzar-bee/ci-guardrail@v1
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    threshold-pct: 20
    price-per-1m-tokens: 0.80   # e.g. a cheaper model's input price

The percentage is what gates the build; the dollar figure is there to make the percentage land
with whoever approves the merge.

What it's actually doing

No magic, and nothing sent anywhere:

Runs tokenscope on your HEAD branch to estimate its total token footprint.
Fetches the base branch and runs the same scan.
Computes the delta as a percentage.
Posts (or updates) the PR comment with the per-file breakdown — and writes the same table to the Actions run summary, so you also see it on push or scheduled runs where there's no PR to comment on.
Exits non-zero if the delta exceeds your threshold-pct (and the threshold is above 0).

The engine is tokenscope, an open-source token-cost analyzer; the Action is a thin, auditable
composite wrapper around it. Both are MIT.

Try it

The Action: github.com/wartzar-bee/ci-guardrail (uses: wartzar-bee/ci-guardrail@v1)
On the Marketplace: CI Cost Guardrail
The engine, standalone: npx @wartzar-bee/tokenscope <your-session.jsonl>

Add the one file, open a PR, and watch the token delta show up next to the diff. If it catches
even one 800-word tool description before it merges, it's paid for itself.

We open-sourced the runtime our agent fleet runs on: Enclave

wartzar-bee — Thu, 23 Jul 2026 17:23:48 +0000

We open-sourced the runtime our agent fleet runs on

We run a small fleet of autonomous agents. Not demos — long-running operators that wake up on a timer, read their own memory, pick the next step, and do it, tick after tick, without a human in the loop. Doing that safely and without setting fire to your model bill turns out to be most of the work.

The agent logic is the easy part. The runtime around it — the sandbox, the credential scoping, the memory that survives a restart, the cost governor that keeps a frontier model from burning your whole cap on a heartbeat — is where the weeks go.

Today we're open-sourcing that runtime. It's called Enclave, it's Apache-2.0, and it's the same code our own fleet runs on right now.

github.com/wartzar-bee/enclave — public alpha, Apache-2.0.

git clone https://github.com/wartzar-bee/enclave.git enclave && cd enclave
./bin/enclave init      # wizard: name, brain, model, port, paste your credential
./bin/enclave run       # build + start, opens a browser chat at 127.0.0.1:8888

What it is

Enclave runs one autonomous agent in a hardened container with scoped credentials and a local web chat. docker compose up, and you talk to your agent in the browser — a real Claude-Code conversation, resumable, multi-thread, at the agent's own model.

It's brain-agnostic: one env var, BRAIN=claude | api | local | optimize. Same container, same security guard, whether you run it on a Claude subscription, any OpenAI-compatible key, or a local model server. Switching brains keeps the agent's memory intact.

What "constrained" actually means (read this before you trust it)

We're allergic to security theatre, so here's the precise boundary — the part most "agent framework" READMEs hand-wave:

The container boundary is enforced by the kernel, always. The agent runs --cap-drop=ALL --security-opt=no-new-privileges with no inbound ports. It sees exactly the mounts you gave it and a read-only secrets/ — nothing else on your disk. A prompt injection does not change that; it's architectural, not a policy the model can be talked out of.
The network boundary is enforced only when you turn it on. The egress allowlist ships in report-only mode by default — it logs disallowed hosts rather than blocking them, so a first run doesn't fail in a way you can't diagnose. For anything real, set GUARD_EGRESS_ENFORCE=1. We'd rather tell you that up front than have you discover it in an audit.

A PreToolUse guard (platform/agentd/hooks/guard.py) fires even under --dangerously-skip-permissions and blocks git, foreign-secret reads, and — via opt-in profiles — cloud writes and production mutations. You can verify all of this by reading the code, which is the point.

The part we care about most: cost discipline

A persistent fleet on a frontier model burns your subscription or API cap fast. At fleet scale, that's the binding constraint — not latency, not quality. Enclave ships two layers to keep judgment quality while cutting spend, both toggled by an env flag:

Model-tier routing (ROUTER=on) — routine heartbeats and mechanical directives (post / measure / narrate / commit) run on a cheaper model, reserving the top model for actual judgment (decide / design / review). It's safe-by-default: anything ambiguous, or any upstream error, resolves up to the top model. You never silently downgrade a decision that mattered.
Manager→worker delegation — when BRAIN=claude, a capable manager is forced to hand bulk code-writing to a cheap or local worker instead of spending frontier tokens on keystrokes. The manager plans and reviews; the worker does the labor under a verify-gate. The guard self-disables for api/local brains, which already are the cheap worker.

This is the same thesis behind our other tools — tokenscope (measure where the tokens actually go) and our CI cost-regression guardrail (block a PR that spikes token cost). Enclave is where those patterns run in production.

Memory that survives a machine wipe

The agent's memory is one linked, markdown, git-trackable vault — an LLM-maintained wiki plus operational memory and skills, navigable as a graph, no DB or GPU required. The runtime auto-snapshots after every tick, and every snapshot is scan-gated and fail-closed: a credential pasted into memory blocks the commit, because git history is forever. Opt-in qmd semantic search and a codegraph symbol/call graph layer on top when you want them.

What it deliberately isn't (the honest gaps)

Open-sourcing a thing you use daily means publishing its rough edges too:

macOS + Linux only. Developed on Apple silicon and Linux; Windows is untested (WSL2 is the likely path, unverified). If you run it elsewhere, reports are genuinely welcome — that's a big reason to open it up.
Host capabilities (bridges) aren't bundled. Browser automation, transcription, TTS live outside the container as host services. Enclave ships the pattern (docs/BRIDGES.md + a working tools/bridge-template/), not the services. Out of the box an agent can think, read/write files, and call APIs — it can't drive a browser until you stand a bridge up.
Public alpha. The API and layout still move. It runs a live fleet daily, but pin your version.

The most useful contribution is a bridge — a new host capability behind a narrow, audited surface. If you've wanted an agent runtime you can actually read end-to-end before you trust it with a credential, this is that.

⭐ github.com/wartzar-bee/enclave (Apache-2.0). Kick the tyres, file an issue with your enclave status output, and tell us what breaks on your OS.

DEV Community: wartzar-bee

Gate your CI on a dollar ceiling, not a percentage — the number your finance team actually asks for

Gate your CI on a dollar ceiling, not a percentage — the number your finance team actually asks for

I scanned the code of 25 MCP servers — what they can do to your machine before you sandbox them

Method (static source scan — honest about its limits)

The capability surface, across 25 servers

Why this matters more for agents than for humans

You can see your cloud bill. Can you see what your AI agent's context costs?

Why this is the number that bites

Three ways to use it

Get it

I scanned 25 popular MCP servers — the hidden context-tax spans 544

The context-tax, measured

While I had the packages open, I also scanned what they do

Measure your own

The cheapest way to try a CI cost gate: run it in warn mode for a week

The cheapest way to try a CI cost gate: run it in warn mode for a week

AutoGen's hidden token tax: why a 3-agent chat costs 15 what you expect

AutoGen's hidden token tax: why a 3-agent chat costs 15× what you expect

The setup: a 3-agent RoundRobin chat

The default context: unbounded, per-agent

The handoff tax: every agent sees every message

The math: O(N × T²) total tokens

Worked example: 10 turns, 3 agents, 300 tokens/message

Why per-call logs hide this

The fix: cap the context

Option 1: BufferedChatCompletionContext (sliding window)

Option 2: TokenLimitedChatCompletionContext (token budget)

Which to use?

Detecting this in CI before it hits production

Summary

Gate your AI agents' token cost in CI — before the bill, not after

1. Budget the prompt before the run, not after

2. Attribute tokens per source, not per message

3. Gate the cost in CI the way you'd gate build time

smolagents replays its whole memory every step: the O(n ) token bill nobody mentions

smolagents replays its whole memory every step: the O(n²) token bill nobody mentions

Where the tokens go

The math (evaluate it on your own numbers)

The fix: prune the replayed history with a step callback

See the bill before you run it

LangGraph isn't cheaper than LangChain — unless you opt out of its defaults

LangGraph isn't cheaper than LangChain — unless you opt out of its defaults

The default: MessagesState + add_messages

The cost math

Multiplier 1: multi-node graphs (each node pays the full state)

Multiplier 2: interrupt_before / interrupt_after (human-in-the-loop)

Multiplier 3: parallel fan-out (Send API)

The opt-outs (LangGraph actually provides them)

Trim messages before every LLM call

Pass only what the node needs

Summarize periodically (the right way)

Measuring your own graph

Summary

CrewAI's quadratic context problem: why a 5-agent crew costs 6 more than you expect

CrewAI's quadratic context problem: why a 5-agent crew costs 6× more than you expect

The architecture in one paragraph

The code that does it

The math

Three multipliers on top of the base cost

1. Memory layers (each one adds tokens)

2. verbose=True (the default in most tutorials)

3. Human delegation (allow_delegation=True)

What a real run looks like

The fix: explicit context scoping

Measuring your own crew

Summary

Put a token-cost gate on your AI-agent PRs in 5 minutes

The one file you add

What you get on the next PR

Start in report-only mode

Make the dollar figure yours

What it's actually doing

Try it

We open-sourced the runtime our agent fleet runs on: Enclave

We open-sourced the runtime our agent fleet runs on

What it is

What "constrained" actually means (read this before you trust it)

The part we care about most: cost discipline

Memory that survives a machine wipe

Option 1: `BufferedChatCompletionContext` (sliding window)

Option 2: `TokenLimitedChatCompletionContext` (token budget)

The default: `MessagesState` + `add_messages`

Multiplier 2: `interrupt_before` / `interrupt_after` (human-in-the-loop)

Multiplier 3: parallel fan-out (`Send` API)

2. `verbose=True` (the default in most tutorials)

3. Human delegation (`allow_delegation=True`)