DEV Community: Assili Salim

A 256-Token Prompt Just Became a $0.58 Bomb. Here's the Fix.

Assili Salim — Tue, 18 Aug 2026 04:33:50 +0000

I was reading through the CCS 2026 accepted papers when one stopped me cold.

The paper is called ReasoningBomb.

The premise: craft a prompt short enough to look completely harmless — under 256 tokens — and it drives a reasoning model into a pathologically long chain of thought. Not occasionally. Consistently. Across commercial models. With a 286.7× input-to-output amplification ratio and a 98.4% bypass rate against dual-stage detection systems built specifically to catch it.

The security community is calling this a DoS attack. That framing is correct.

But there's a second attack happening at the same time that nobody's writing about: the financial one.

Run the math. It's worse than you think.

GPT-5.6 Sol pricing: $5/M input, $30/M output. Reasoning tokens bill as output.

One triggered call:

Input:   256 tokens  × $5/M   =  $0.00128
Output: 19,263 tokens × $30/M =  $0.57789
Total:                          $0.579 per call

452× cost amplification. On a single request. Not because the model failed — because it succeeded at generating the longest possible reasoning trace.

On Claude Opus 4.8 ($15/$75 per million): $1.44 per triggered call.

Now think about what your agent processes every day. Customer support tickets. Document ingestion. Code review tools reading PR descriptions. Web browsing agents fetching arbitrary URLs. Any external content flowing through a reasoning model on metered API is a potential trigger surface.

The attack doesn't need a malicious user at a keyboard. A crafted document in your RAG pipeline, a poisoned page your agent scrapes, a tampered issue in your ticketing system — all of them can carry it.

Why your existing controls won't catch this

Rate limits cap requests per minute, not cost per request. ReasoningBomb firing once a minute is fully within rate limits — billing $0.578 per call, continuously.

Spending alerts fire after the cost is incurred. By the time your 80% budget alert triggers, the session is already over budget. The alert is accurate and completely useless.

Monthly caps — you find out next month.

No max_completion_tokens set — most frameworks don't set this by default on reasoning models. The model decides how long to think. The attack exploits exactly this gap.

The only intervention point that matters: before the call

Two defenses. They work best together.

First: always set max_completion_tokens on reasoning model calls.

This is the minimum viable defense. Set a ceiling that covers your legitimate use cases — with room to spare, but not 19,000 tokens of room:

const response = await openai.chat.completions.create({
  model: 'gpt-5.6-sol',
  messages: [{ role: 'user', content: userInput }],
  max_completion_tokens: 2000, // hard ceiling — tune for your task
});

A task that legitimately needs 800 output tokens isn't affected by a 2,000-token ceiling. A ReasoningBomb trying to generate 19,263 tokens gets cut off at 2,000 — and your bill reflects that, not the attack's intent.

Second: make the ceiling dynamic based on remaining session budget.

A static ceiling protects individual calls. A budget-aware ceiling protects the entire session:

const MODEL_PRICES: Record<string, { inputPerM: number; outputPerM: number }> = {
  'gpt-5.6-sol':     { inputPerM: 5.00,  outputPerM: 30.00 },
  'gpt-5.6-terra':   { inputPerM: 2.00,  outputPerM: 12.00 },
  'gpt-5.6-luna':    { inputPerM: 0.20,  outputPerM:  1.20 },
  'claude-opus-4-8': { inputPerM: 15.00, outputPerM: 75.00 },
};

function guardWithOutputCeiling(
  model: string,
  session: { spentCents: number; reservedCents: number; limitCents: number }
): number {
  const price = MODEL_PRICES[model];
  if (!price) throw new Error(`Unregistered model: "${model}"`);

  const remainingCents =
    session.limitCents - session.spentCents - session.reservedCents;

  if (remainingCents <= 0) {
    throw new Error(`Session budget exhausted`);
  }

  const affordableTokens = Math.floor(
    (remainingCents / 100 / price.outputPerM) * 1_000_000
  );

  const TASK_MAX_OUTPUT = 4_000;
  return Math.min(affordableTokens, TASK_MAX_OUTPUT);
}

// Before every reasoning model call:
const maxTokens = guardWithOutputCeiling('gpt-5.6-sol', session);

const response = await openai.chat.completions.create({
  model: 'gpt-5.6-sol',
  messages,
  max_completion_tokens: maxTokens,
});

What this does: a ReasoningBomb targeting a session with $0.10 remaining gets capped at 3,333 output tokens — the maximum $0.10 buys at Sol pricing.

The attack still fires. But its cost is bounded by what the session can actually afford — not by what the attack is trying to spend.

That's the distinction that matters: "attack succeeded" versus "attack controlled your bill." The ceiling doesn't prevent the trigger. It prevents the trigger from determining the cost.

The threat model to carry into production

ReasoningBomb isn't a theoretical edge case. It's accepted at a top-tier security conference. The code is published on GitHub. It achieves near-perfect detection bypass on commercial models including GPT-5.6.

Any agent processing external content through a reasoning model on metered API is currently exposed.

The fix costs you one parameter per API call. The dynamic version costs you one function. Neither requires infrastructure changes.

@salimassili/ai-costguard implements this as maxOutputTokens enforcement in the session guard — the ceiling is calculated from remaining budget before each call and set automatically. The underlying pattern is always the same: compute the maximum output tokens your remaining budget can afford, set it explicitly, let the provider enforce it at the API level.

The bill for a ReasoningBomb call arrives before you can stop it.

The ceiling is the only defense positioned in the right place.

Repo: github.com/salimassili62-afk/ai-costguard

Two Billing Clocks, One Session Budget: How to Guard Costs When Your Agent Runs in a Metered Sandbox

Assili Salim — Mon, 17 Aug 2026 04:40:44 +0000

Docker Sandboxes launched this week — purpose-built microVMs for coding agents running Claude Code, Codex, and Gemini. Isolated daemon, filesystem, network. Metered compute.

Right architecture for agent isolation. Wrong assumption for most cost guards.

The model most guards assume — and when it breaks

Local agent execution has a simple cost model:

Session cost = tokens × model rate

The sandbox is your machine.
Compute is free.

Move to Docker Sandboxes, E2B, Daytona, or Modal — and you now have two meters running simultaneously:

Token cost:    tokens × model rate       ← your guard probably tracks this
Compute cost:  uptime × compute rate     ← your guard probably doesn't

A retry loop that runs 45 minutes doesn't just burn token budget.
It keeps a metered microVM alive for 45 minutes.
At E2B's standard rate of ~$0.083/hour, that loop adds $0.062 in compute on top of whatever tokens it consumed.

Small in isolation.
Fatal to a session budget calibrated on token costs alone.

What a real overrun looks like

A production coding agent, $0.50 session budget, GPT-5.6 Terra pricing ($2/$12 per million):

Expected:
  5 agent turns × ~3,000 tokens
  Token cost:    ~$0.050
  Sandbox uptime: ~8 minutes
  Compute cost:  ~$0.011
  Projected:     ~$0.061 — well within budget

A tool call fails. 
Agent retries. 
Gets confused. 
Loops.

Actual:
  80+ tool calls, ~50,000 tokens
  Token cost:    ~$0.710
  Compute cost:  ~$0.065  (47 min × $0.083/hr)
  Actual total:  ~$0.775

The token guard would have eventually blocked on token spend.
The compute cost was accruing from the moment the sandbox spawned , no token-level guard can touch it.
Only a compute budget check or a lifetime limit can stop it.

The unified session model

interface SandboxSession {
  sandboxId: string;
  startedAt: Date;
  computeRatePerHour: number;

  spentTokenCents: number;
  reservedTokenCents: number;
  totalLimitCents: number;
}

function currentComputeCost(session: SandboxSession): number {
  const uptimeHours = (Date.now() - session.startedAt.getTime()) / 3_600_000;
  return uptimeHours * session.computeRatePerHour * 100; // cents
}

function totalCurrentCost(session: SandboxSession): number {
  return session.spentTokenCents + currentComputeCost(session);
}

Pre-call guard checks the combined total before every model call:

function guardCall(
  model: string,
  estimatedInputTokens: number,
  estimatedOutputTokens: number,
  estimatedCallMinutes: number,
  session: SandboxSession
): void {
  const price = MODEL_PRICES[model];
  if (!price) throw new Error(`Unregistered model: "${model}"`);

  const tokenCost =
    (estimatedInputTokens  / 1_000_000) * price.inputPerM  * 100 +
    (estimatedOutputTokens / 1_000_000) * price.outputPerM * 100;

  const additionalCompute =
    (estimatedCallMinutes / 60) * session.computeRatePerHour * 100;

  const projectedTotal =
    totalCurrentCost(session)  +
    session.reservedTokenCents +
    tokenCost                  +
    additionalCompute;

  if (projectedTotal > session.totalLimitCents) {
    throw new BudgetExceededError({
      sandboxId: session.sandboxId,
      projectedTotal,
      limitCents: session.totalLimitCents,
      breakdown: {
        currentTokenSpend:  session.spentTokenCents,
        currentComputeCost: currentComputeCost(session),
        thisCallTokens:     tokenCost,
        thisCallCompute:    additionalCompute,
      },
    });
  }

  session.spentTokenCents += tokenCost;
}

projectedTotal includes already-accrued compute.
If the session has been running 20 minutes, currentComputeCost reflects that before the guard evaluates the next call.
The guard sees the real cost trajectory , not just forward token spend.

The simpler version: just cap sandbox lifetime

Per-call compute modeling too granular for your workload? A lifetime limit gets you most of the protection with a fraction of the complexity:

function guardSandboxLifetime(
  session: SandboxSession,
  maxLifetimeMinutes: number
): void {
  const uptimeMinutes = (Date.now() - session.startedAt.getTime()) / 60_000;

  if (uptimeMinutes >= maxLifetimeMinutes) {
    throw new SandboxLifetimeExceededError({
      sandboxId:    session.sandboxId,
      uptimeMinutes,
      limitMinutes: maxLifetimeMinutes,
    });
  }
}

// Call at the start of every agent turn — before any tool calls
guardSandboxLifetime(session, 30);

A 30-minute hard cap is a compute ceiling without exact rate modeling.
It also directly kills the loop scenario , the agent can't run for 47 minutes if the session terminates at 30.

One important detail: call this at the start of every agent turn, not just on LLM calls.
Compute billing accrues during tool execution, file I/O, and idle waiting , not just during inference.

Before you move to metered sandboxes

If your agents run locally today and you're planning the move to Docker Sandboxes or any metered provider, expand the budget model before you flip the switch.

Token-only cost modeling that was accurate locally becomes structurally incomplete on metered infrastructure.
A session you budgeted at $0.06 in token costs becomes $0.12 once compute time is included , before any overruns.
That gap isn't an edge case.
It's the baseline.

The two invoices are real. The guard needs to see both.

GPT-5.6 Puts Reasoning in Every Tier. Your Cost Estimator Isn't Ready.

Assili Salim — Sun, 16 Aug 2026 03:54:24 +0000

OpenAI's GPT-5.6 ships reasoning across the entire production family. Sol, Terra, Luna — all of them reason before responding.

That's a capability upgrade. It's also a billing problem most teams haven't hit yet — but will.

The invisible tokens eating your budget

Reasoning tokens are billed as output tokens. They just never appear in the response.

So the model returns a short answer while consuming substantially more billed tokens internally. Your estimator sees the visible output. The invoice sees everything.

Concrete example — a Sol request returning 200 visible tokens with 600 reasoning tokens underneath:

Visible output:    200 tokens
Reasoning:         600 tokens
Billed output:     800 tokens

At $30 per million output tokens:

Estimated (visible only):   200 / 1M × $30 = $0.006
Actual:                     800 / 1M × $30 = $0.024

4× off. Not because the math was wrong. Because the estimator measured the wrong thing.

Why a point estimate breaks here

Most pre-call cost guards look like this:

const estimatedCost =
  (inputTokens / 1_000_000) * inputPrice +
  (expectedOutputTokens / 1_000_000) * outputPrice;

That's fine when visible output length is a reasonable proxy for billed output. For reasoning models, it isn't.

A simple formatting task and a complex architectural problem can return similarly-sized responses while generating completely different reasoning-token counts internally. Same output length, wildly different cost.

The guard needs to account for that uncertainty — not pretend it doesn't exist.

Use a range, not a number

Instead of estimating a single cost figure, estimate a range:

interface ModelPricing {
  inputPerM: number;
  outputPerM: number;
  thinkingMultiplierRange?: [number, number];
}

Shift the question from:

"What will this call cost?"

to:

"What's the plausible cost ceiling — and can the session afford it?"

If the worst-case estimate exceeds the remaining budget, block the call before it leaves your process. That's the only point where you still have control.

Reconcile after execution

Once the provider responds, replace the reservation with reality:

Before call:
  estimate range
  reserve against ceiling
  block if ceiling is unaffordable

After call:
  read actual token usage
  release reservation
  record actual spend

This is a standard reservation pattern. The key property is unchanged: the budget decision happens before the request goes out. Actual usage just closes the loop.

Let your own workload calibrate the range

A generic multiplier is a starting point, not an answer.

Once you have production data, track the ratio of reasoning tokens to visible output tokens per model per workload type. Over time, your registry moves from:

"reasoning might add 0.5–4×"

to:

"for this workload, observed range is 1.2–1.8×"

The guard gets sharper without pretending reasoning cost is predictable before execution.

Caching creates the opposite failure

Reasoning inflates actual cost above estimate. Caching deflates it.

A simplistic guard fails in both directions:

Reasoning: actual cost > estimate → you underreserved, overspent
Caching: actual cost < estimate → you overreserved, blocked calls unnecessarily

The fix is the same either way: model the provider's actual billing mechanics, not just input × price + output × price.

The bigger shift

As models get more sophisticated, token count alone stops being a sufficient cost model.

A production cost guard needs to account for:

Model-specific pricing tiers
Cached input discounts
Reasoning output ranges
Retries and partial failures
Session-level budgets across multiple agents

Some of those variables are only fully known after execution. That's not a limitation to work around — it's the architecture to design for.

Estimate and reserve before the call. Reconcile against actual usage afterward.

The estimate isn't wrong. It's just not the whole bill anymore.

Multi-Agent Systems Share Context. They Should Share Budgets Too.

Assili Salim — Wed, 12 Aug 2026 04:23:48 +0000

This week, researchers introduced AgentRadio — an async messaging layer that lets coding agents exchange information between execution steps without burning an extra turn.

The headline was performance: agent teams using AgentRadio nearly doubled results on long-horizon software engineering tasks versus isolated agents.

The part I found more interesting was cost.

Better coordination already reduces waste

Traditional multi-agent systems sync at fixed checkpoints. So if Agent A figures out halfway through that the team is going in the wrong direction, Agents B, C, and D keep running until the next sync point.

Every one of those calls is still billed.

AgentRadio cuts that waste. Agents receive updates asynchronously and adjust at their next step — no waiting for a scheduled checkpoint. Less duplicated work means fewer wasted model calls.

Better coordination isn't just faster. It's cheaper.

But that's the easy part.

The real problem: per-agent budgets don't compose

The harder problem shows up once multiple agents share the same objective.

Most budget guards look like this:

class AgentBudgetGuard {
  private spent = 0;

  check(cost: number) {
    if (this.spent + cost > LIMIT) {
      throw new Error("Budget exceeded");
    }
    this.spent += cost;
  }
}

Perfect for one agent. Completely blind to the others.

Now imagine four agents sharing the same task:

Agent	Spent
A	$0.42
B	$0.39
C	$0.47
D	$0.44

Every individual guard says: still within budget.

Combined, the task has nearly blown its limit.

Nobody knows. Nothing stops.

Fix: budget belongs to the task, not the agent

Stop tracking spend per agent. Track it once — at the task level — and make every agent check the same pool before each call.

// Every agent checks the shared task budget before calling a model
await taskBudget.check(agentId, estimatedCost);

One line change. Every agent now knows what the team has left, not just what it has spent.

The implementation is straightforward. The critical detail is atomicity.

If two agents simultaneously read the same remaining balance and both decide to proceed, they'll both overspend — together, instantly. Whether you use Redis, PostgreSQL, or DynamoDB, the check-and-update must be a single atomic operation. Read-then-write doesn't cut it here.

Shared budgets unlock team-level controls

Once your agents share a budget, threshold-based policies actually make sense:

const THRESHOLDS = {
  NOTIFY:      0.60,  // alert the orchestrator
  DOWNGRADE:   0.80,  // switch all agents to a cheaper model
  HALT:        1.00,  // stop all new model calls
};

Notice what changes: the decision applies to the entire team. Not whichever agent happened to cross the threshold first. Everyone degrades together, everyone stops together, everyone operates under the same budget policy.

That's coherent cost governance. Per-agent guards can't give you that.

The principle AgentRadio proves, applied to money

AgentRadio's core insight is that agents perform better when they share information.

The same is true for financial state.

Sharing context without sharing budgets creates a strange situation: every agent makes a locally correct decision while the team collectively burns through money nobody is watching.

As multi-agent architectures get more common, shared budgets need to be treated as seriously as shared memory.

Capability scales through coordination. Cost control has to scale with it.

https://github.com/salimassili62-afk/ai-costguard

Article 50 Is Live. Your Compliance Logs and Your Debug Logs Should Be the Same Thing.

Assili Salim — Tue, 11 Aug 2026 03:32:47 +0000

Most teams responded to Article 50 of the EU AI Act with a disclosure banner.

That's fine. Do that. But it's not enough.

Because if a regulator ever investigates a specific interaction, "we always show a disclosure" won't cut it. You need evidence tied to that exact session — a record you can point to and say: here's what happened, when it happened, and to whom.

That's a logging problem. And if you're already building AI agents, you're probably 80% of the way there — you just haven't connected the dots yet.

What compliance actually requires

Article 50 doesn't define a logging schema. But in practice, answering a regulatory inquiry means being able to reconstruct:

When did this session start?
Which model generated the response?
Was the AI disclosure shown — and how?
What output did the user receive?
Which user or tenant does this belong to?

Policy documents don't answer those questions. Session records do.

The overlap with what engineering already needs

Here's the part worth paying attention to: the data compliance needs is mostly the same data your engineers already want for debugging, routing analysis, and cost attribution.

ts
interface AgentSessionRecord {
sessionId: string;
userId?: string;
startedAt: Date;

calls: {
timestamp: Date;
model: string;
estimatedInputTokens: number;
estimatedOutputTokens: number;
estimatedCost: number;
}[];

disclosures: {
timestamp: Date;
method: "message" | "ui";
}[];
}

Same structure. Different consumers.

Engineering asks: why was this session expensive? why did model routing change?
Compliance asks: was a disclosure shown? which system generated this output?

You don't need two separate systems for those questions. You need one well-designed session record.

What to build first

If you're shipping AI agents today, start here:

Generate a unique session ID for every interaction.
Don't reconstruct sessions from logs after the fact. Assign the ID upfront and thread it through everything.
Record which model handled each request.
Not just the provider — the exact model. Models change, routing changes, costs change. You need the audit trail.
Log disclosure events explicitly.
Don't assume you can infer them later from UI behavior. An explicit disclosures[] array is a first-class field, not an afterthought.
Associate records with users or tenants where appropriate.
This matters for both cost attribution and regulatory traceability.
Record enough metadata to trace outputs without storing every response.
You don't necessarily need to store full completions — but you need enough to reconstruct what happened.

The real takeaway

The interesting thing about Article 50 isn't the transparency requirement itself.

It's that compliance now depends on operational evidence — not policy documents, not promises, not architecture diagrams.

Teams that have already invested in structured session-level observability will find themselves ahead. Not just for cost control and debugging — but for being able to demonstrate, concretely, how their AI systems behaved.

Session records aren't a compliance feature bolted on top of your system.

They're just good engineering — that also happens to keep you accountable.
https://github.com/salimassili62-afk/ai-costguard

Alert-Based vs. Threshold-Based: Why Monitoring Isn't the Same as Cost Control

Assili Salim — Sun, 09 Aug 2026 03:33:38 +0000

A recent OpenAI update caught my attention—not because of the model itself, but because of the control architecture behind it.

According to OpenAI, parts of Astra's development were paused after internal evaluations crossed a predefined threshold under its Preparedness Framework.

The important engineering pattern isn't the safety policy.

It's that crossing a threshold triggered an automatic response.

No one had to notice a dashboard first.

That distinction applies far beyond AI safety.

Monitoring tells you what happened

Most AI cost systems look something like this:

const response = await callProvider(...);

const cost = calculateCost(response);

session.spent += cost;

if (session.spent > session.limit) {
  notifySlack();
}

This is useful.

You know exactly when the budget was exceeded.

The problem is obvious:

The expensive request has already happened.

For interactive applications, that's often acceptable.

For autonomous agents running overnight, it usually isn't.

Control happens before the request

A runtime guard works differently.

Instead of reacting to completed requests, it evaluates the next request.

const estimatedCost = estimateCost(request);

if (session.spent + estimatedCost > session.limit) {
  throw new BudgetExceededError();
}

await callProvider(...);

The key difference isn't the calculation.

It's when the calculation happens.

One architecture observes.

The other controls.

Thresholds can trigger different actions

Blocking isn't the only possible response.

A production system can define multiple thresholds with predefined behavior:

Budget usage	Response
50%	Log or notify
80%	Route to a cheaper model
100%	Block additional requests

That creates deterministic behavior instead of relying on humans to intervene.

The system already knows what to do before the threshold is reached.

Why this matters for agents

Traditional chat applications naturally include a human feedback loop.

Agents don't.

A coding agent, research workflow, or CI automation might execute hundreds of requests without anyone watching.

In those systems, an alert that arrives after spending has already occurred is useful for reporting—but it cannot prevent additional spend.

That's why autonomous systems benefit from runtime guardrails rather than monitoring alone.

Monitoring and control solve different problems

This isn't an argument against dashboards, telemetry, or alerts.

Those remain essential.

But they're answering a different question.

Monitoring asks:

What happened?

Runtime control asks:

Should the next request be allowed?

Those are complementary systems—not interchangeable ones.

As more software becomes agent-driven, I expect this distinction to become increasingly important. The most effective cost-control architectures won't simply report threshold crossings—they'll define, in advance, exactly how the system responds when those thresholds are reached.
https://github.com/salimassili62-afk/ai-costguard

Spot-priced LLM inference changes how runtime budget guards should work

Assili Salim — Thu, 06 Aug 2026 05:30:46 +0000

Most AI cost guards assume one thing:

You know the price before you make the request.

That's been true for most LLM APIs.

It's becoming less true as inference marketplaces introduce dynamic pricing, where providers compete for requests and the final price is determined at runtime.

That creates an interesting engineering problem.

Traditional budget guards

A typical pre-call budget check is simple:

Look up the model price.
Estimate token usage.
Estimate request cost.
Compare against the remaining session budget.
Allow or block the request.

That works well when prices are fixed.

It doesn't work as well when the final price isn't known until the request completes.

The wrong solutions

There are two obvious approaches.

Ignore pricing uncertainty

This removes one of the main reasons the guard exists.

Always assume list price

Safe, but often overly conservative. You end up rejecting requests that would have cleared well within budget.

Neither is ideal.

Reserve a maximum, settle the actual

A pattern borrowed from cloud infrastructure works much better.

Instead of estimating the exact cost, reserve the maximum amount you're willing to spend.

interface SpotRequest {
  maxBidPerMillion: number;
  estimatedInputTokens: number;
  estimatedOutputTokens: number;
}

Before the request:

Reserve the worst-case spend.
Reject the request if that reservation exceeds the session budget.

After the request:

Replace the reservation with the actual amount charged.
Release any unused budget back to the session.

The important property is that your budget is never temporarily overstated.

Budget enforcement becomes deterministic

With this approach, the guard doesn't ask:

How much will this request cost?

Instead, it asks:

Can this session afford the maximum amount we're willing to pay?

That's a question you can answer before the request leaves your process.

Bid ceilings become another optimization tool

Once you introduce a maximum acceptable price, that ceiling can become part of your routing strategy.

For example:

low-value classification → low bid ceiling
code review → medium ceiling
architecture tasks → higher ceiling

As the session budget decreases, those ceilings can become more conservative.

Instead of simply rejecting requests when money runs low, your system can automatically become more selective about what it's willing to pay for.

Why this matters

Today it's spot-priced inference.

Tomorrow it may be:

dynamic provider marketplaces,
region-specific pricing,
time-based pricing,
SLA-based pricing,
or real-time routing across multiple providers.

The common pattern is the same:

The exact price is no longer guaranteed before the request is sent.

That means runtime cost controls need to evolve as well.

Rather than assuming every request has a fixed price, they should be designed to operate under uncertainty—enforcing budgets using maximum acceptable cost, then reconciling against the actual price afterward.

That's a much more flexible architecture for the direction AI infrastructure appears to be heading.
https://github.com/salimassili62-afk/ai-costguard

Model selection isn't just an AI decision. It's a cost decision.

Assili Salim — Wed, 05 Aug 2026 04:25:50 +0000

A lot of AI teams optimize prompts, workflows, and infrastructure.

Far fewer treat model selection as a runtime cost-control problem.

That's becoming a mistake.

A recent example is Cursor. Switching to a more capable model can dramatically increase API usage costs, even though the only thing a developer changed was a dropdown in their editor.

Nothing else changed.

No deployment.

No architecture change.

Just a different model.

The real problem

Most agent systems treat model choice as a preference:

const model = process.env.DEFAULT_MODEL;

In reality, it should be treated as a decision constrained by:

task complexity,
remaining session budget,
latency requirements,
and expected ROI.

Instead of asking "What's the best model?", production systems should ask:

"What's the cheapest model that can successfully complete this task?"

1. Route models by task

Not every request needs your most capable—and most expensive—model.

A simple router is often enough:

function selectModel(task: Task) {
  switch (task.complexity) {
    case "low":
      return "cheap";

    case "medium":
      return "standard";

    case "high":
      return "frontier";
  }
}

Classification, formatting, and summarization rarely need frontier models.

Save those for the problems that actually benefit from them.

2. Make budget part of routing

Routing shouldn't depend only on the task.

It should also depend on how much budget remains.

if (remainingBudget < threshold) {
    return "cheap";
}

That allows an agent to degrade gracefully instead of exhausting its budget on the last few requests.

3. Validate before the call

Every provider request should pass through a budget check.

That check should answer questions like:

Is this model registered?
Do we know its pricing?
Can this session afford another request?

If the answer is no, the request shouldn't leave your process.

Failing early is much cheaper than discovering the problem in next month's invoice.

4. Don't let context grow forever

Long-running agents become expensive because every request carries more history.

A sliding context window plus pinned instructions often reduces token usage without affecting output quality.

Keeping all history is usually the easiest implementation—not the cheapest one.

The bigger shift

The industry talks a lot about observability.

Observability tells you what happened.

Cost governance decides what is allowed to happen.

That's an important distinction.

As models become more numerous—and pricing changes more frequently—the most effective cost optimization won't come from better dashboards.

It will come from making model selection a guarded runtime decision instead of a static configuration value.

That's the architectural direction I'm exploring with AI CostGuard, but the principle applies regardless of the framework or provider you're using.
https://github.com/salimassili62-afk/ai-costguard

Migrating from Bedrock Agents Classic to AgentCore? Audit your cost model first.

Assili Salim — Tue, 04 Aug 2026 04:43:45 +0000

Amazon has closed Bedrock Agents Classic to new customers. Existing agents continue to run, and there's no announced end-of-life—but if you're planning a move to AgentCore, there's one migration step that deserves more attention:

Review your cost model before you migrate your code.

Your billing model has changed

With Bedrock Agents Classic, most teams only needed to think about:

Model inference
Lambda
Knowledge Base queries
Other AWS resources

The orchestration layer itself wasn't a separate billable service.

AgentCore changes that.

In addition to inference costs, you now have platform-level charges such as agent invocations and gateway requests. That means your total cost is no longer driven only by tokens.

Token tracking is no longer enough

Many internal cost dashboards look something like this:

interface SessionCost {
  inputTokens: number;
  outputTokens: number;
  estimatedCost: number;
}

That worked well when inference represented almost the entire bill.

After migrating, your total session cost may also include platform activity that token-based tracking never sees.

The result isn't an incorrect calculation—it's an incomplete one.

Think in two cost layers

Instead of treating "AI cost" as a single number, separate it into two categories:

Inference costs

Input tokens
Output tokens
Model pricing

Platform costs

Agent invocations
Gateway requests
Other orchestration charges

Tracking both independently makes it much easier to understand why costs change over time.

A prompt optimization affects inference.

An orchestration redesign might barely change tokens while significantly increasing platform activity.

Those are different engineering problems.

Before migrating

A migration is a good opportunity to validate your assumptions.

I'd recommend comparing:

Estimated inference cost
Platform charges
Combined session cost

for the same workload before rolling changes into production.

That baseline makes future regressions much easier to spot.

The broader trend

AgentCore isn't unique here.

Across the industry, agent platforms are becoming more opinionated and more capable—and that often means more of the bill moves into the orchestration layer instead of the model layer.

For years, tracking tokens was enough.

Going forward, teams will increasingly need to treat model costs and platform costs as separate budgets.

The sooner your observability reflects that split, the easier future migrations will be.
https://github.com/salimassili62-afk/ai-costguard

GPT-5.6 Luna dropped 80% in three weeks. Here's the pricing bug it can introduce into your agent stack.

Assili Salim — Mon, 03 Aug 2026 05:11:54 +0000

OpenAI recently cut GPT-5.6 Luna from $1.00 → $0.20 per million input tokens—an 80% price reduction just three weeks after launch.

Cheaper models are great.

But they expose a problem many agent systems quietly have: pricing logic that assumes model costs never change.

The stale-pricing bug

A lot of cost controls look something like this:

const LUNA_INPUT_PER_M = 1.00;

function estimateCost(tokens: number) {
  return (tokens / 1_000_000) * LUNA_INPUT_PER_M;
}

That worked when Luna launched.

After the price cut, it doesn't.

If your session budget is based on those constants, your guardrails are now enforcing assumptions that no longer match reality.

Nothing crashes.

Nothing throws an error.

Your budgeting logic simply becomes stale.

The fix isn't just updating one constant—it's avoiding hardcoded pricing throughout the codebase.

Unknown model variants are another hidden risk

Price changes often arrive alongside new model variants.

If your routing layer starts sending traffic to a model that isn't registered in your pricing table, you have another problem:

const MODEL_REGISTRY = {
  "gpt-5.6-luna": {...},
  "gpt-5.6-terra": {...},
  "gpt-5.6-sol": {...},
};

If a request targets an unknown variant, the safest behavior is to fail before the request is sent.

Unknown pricing should never silently become zero pricing.

A centralized model registry makes this much easier to manage than scattered constants across multiple services.

Treat pricing as configuration, not code

Provider pricing changes more often than most engineering teams expect.

That means model prices should be treated like any other operational configuration:

Keep pricing in one registry.
Validate model IDs before requests.
Update pricing independently of application logic.
Fail loudly when pricing is missing.

That approach makes repricing events predictable instead of surprising.

The bigger lesson

The real issue isn't that GPT-5.6 Luna became cheaper.

It's that pricing is no longer static enough to hardcode.

As providers continue adding models, variants, and promotional pricing, cost controls need to adapt just as quickly.

If your pricing lives in code, every provider announcement becomes a deployment.

If your pricing lives in a validated registry, it's just another configuration update.
https://github.com/salimassili62-afk/ai-costguard

Supabase's new agent benchmark reveals 3 lessons production AI agents still ignore

Assili Salim — Sun, 02 Aug 2026 04:25:59 +0000

Supabase just open-sourced supabase/evals, a benchmark that runs Claude Code, Codex, and OpenCode against real-world Supabase tasks using containerized environments, MCP servers, and live CLI tools.

The scores are interesting.

The engineering decisions behind the benchmark are even more valuable.

Here are three choices that stood out—and why they matter if you're building AI agents in production.

1. They capped retries

Supabase allows one retry before grading a task.

The reason is simple: more retries reduce false negatives, but they also make evaluations more expensive. At some point, the benchmark stops being practical.

Production agents usually don't have this discipline.

Most frameworks let you cap total iterations, but that's different from limiting retries of the same failed action. An agent can fail several times, eventually succeed, and still burn far more tokens than expected.

Success doesn't always mean efficiency.

If you're running agents at scale, retry budgets deserve to be treated as first-class limits—not just an implementation detail.

2. Tool usage matters as much as accuracy

One benchmark result caught my attention:

Claude Code averaged roughly 2 documentation reads per scenario.
Codex averaged roughly 8.

Both may solve the task.

One simply consumes much more context along the way.

Accuracy leaderboards rarely show this, but production bills certainly do.

When comparing models, measuring tool calls, token usage, and cost per completed task is often more useful than looking at completion rate alone.

3. Better descriptions changed agent behavior dramatically

Supabase noticed one internal skill was only activating about 10% of the time.

They rewrote its description.

Activation jumped to around 60%.

Nothing about the underlying capability changed—only how the agent understood when to use it.

That's an important reminder that prompts, skill descriptions, and routing logic are part of your system design. If you only measure task success, you'll miss whether your agents are actually using the components you built.

The bigger lesson

None of these decisions were made because they're academically interesting.

They were made because Supabase runs these evaluations every day, and daily benchmarks need predictable cost and reliability.

Production agents face the same constraints.

The difference is that benchmark designers build those limits in from day one, while production systems often discover them only after expensive failures.

If I had to copy one idea immediately, it would be the retry cap.

https://github.com/salimassili62-afk/ai-costguard

Two Claude network failures this week. Here's the retry-logic bug they exposed.

Assili Salim — Fri, 31 Jul 2026 11:34:29 +0000

Over about 36 hours spanning July 29 to 31, Claude's serving infrastructure had four distinct incidents. Not one outage — four, each a different shape, each hitting a different scope.

Per independent status tracking: a 40-minute drop across all models the evening of the 29th, a four-hour-plus window of elevated errors the next morning, a 40-minute "degraded performance" period on Claude Opus 4.8 that afternoon, and a shorter 9-minute degraded-performance blip on Claude Sonnet 5 the morning of the 31st. Anthropic's own statement named the mechanism for the first two: multiple separate network failures cut into serving capacity, and the reroute response itself caused some requests to fail.

That last detail is the one worth building around. The fix caused failures of its own. If your circuit breaker only knows how to detect "down," this entire window looks like something else: flaky.

Two failure shapes, two very different bills

Hard failure. The provider returns a fast, clean error: connection refused, a 503, a timeout. Every reasonable retry library detects this on the first or second attempt and either backs off hard or opens a circuit. This is the failure mode most agent frameworks are built around, and it's genuinely the easy case.

Capacity-shaped failure. This is what "elevated errors" and "reduced availability" actually mean in practice: some calls succeed, some time out, some fail, mixed together over hours, because the provider is serving from reduced capacity rather than serving nothing. A naive consecutive-failure breaker was never designed to catch this:

typescript
// Trips only on N consecutive failures.
// A capacity-shaped incident rarely produces
// five failures in a row — it produces five failures
// spread across forty successful calls.
class NaiveBreaker {
private consecutiveFailures = 0;
private readonly threshold = 5;

recordResult(ok: boolean) {
this.consecutiveFailures = ok ? 0 : this.consecutiveFailures + 1;
}

isOpen() {
return this.consecutiveFailures >= this.threshold;
}
}

Every interspersed success resets the counter. The breaker never opens. Your retry logic keeps sampling a flaky distribution for four hours, and every sample is a billed call.

The fix is to stop counting streaks and start tracking a rate over a window:

typescript
// Trips on error RATE over a rolling window,
// regardless of whether failures are consecutive.
class RollingRateBreaker {
private window: boolean[] = []; // true = success
private readonly windowSize = 20;
private readonly errorRateThreshold = 0.3;

recordResult(ok: boolean) {
this.window.push(ok);
if (this.window.length > this.windowSize) this.window.shift();
}

isOpen() {
if (this.window.length < this.windowSize) return false;
const errorRate = this.window.filter((r) => !r).length / this.window.length;
return errorRate >= this.errorRateThreshold;
}
}

Twenty calls, six or more failures anywhere in the window, and it trips, whether those failures are clustered or scattered. That matches how capacity-shaped incidents actually behave.

The bill waiting on the other side of recovery

Here's the second cost, and it only shows up if your agent has a fallback provider configured. Say your harness fell back to a second model mid-incident. The task keeps running there. Then the primary recovers, and if nothing tracked which provider actually finished the job, you risk redoing the same work on both — paying twice for one logical task.

This isn't hypothetical. It's specific enough that incident write-ups from this week explicitly warned people not to duplicate work across harnesses once the primary provider came back.

The fix is a small reservation ledger keyed on the logical task, not the provider:

typescript
interface TaskAttempt {
taskId: string;
provider: string;
status: 'in_flight' | 'completed';
}

const attempts = new Map();

function beforeCall(taskId: string, provider: string) {
const existing = attempts.get(taskId) ?? [];
const doneElsewhere = existing.find(
(a) => a.status === 'completed' && a.provider !== provider
);

if (doneElsewhere) {
throw new Error(
Task ${taskId} already completed on ${doneElsewhere.provider}
);
}

existing.push({ taskId, provider, status: 'in_flight' });
attempts.set(taskId, existing);
}

One map. One check before the call goes out. It's the difference between "the outage cost us four hours of degraded service" and "the outage cost us four hours of degraded service, plus a duplicate invoice from the fallback provider we forgot was still running."

What to actually change this week
Replace consecutive-failure breakers with rolling-rate breakers, at least for provider calls.
If you run a fallback provider, key task completion by task ID, not by provider, and check it before every call, not just the first one.
Treat "degraded performance" status entries as a distinct alert category from "down." They're not lower priority. They're the ones your retry logic is least equipped to handle.

None of this prevents a network failure. It's the layer that decides what a network failure costs you afterward.

This is the exact class of failure AI CostGuard checks for before a call executes: not whether the provider is up, but whether the call about to go out is one your session has already paid for somewhere else.