DEV Community: Assili Salim

MCP 2026-07-28 shipped today. Here's what it means for agent cost tracking.

Assili Salim — Wed, 29 Jul 2026 03:28:19 +0000

The MCP 2026-07-28 specification went final today.
The maintainers call it the largest revision since launch.
Most coverage is focusing on the stateless core and the new Extensions framework.

Two changes in this release affect how you track agent costs — and both push responsibility into your application layer.

The session primitive is gone

Mcp-Session-Id and the initialize/initialized handshake are removed.
Every request is now self-describing.
Correct call for scalability: MCP servers can run behind a plain round-robin load balancer, no sticky sessions, no shared session store.

The side effect: there's no longer a session scope at the protocol level to group calls under.

Before today, teams anchoring budget tracking to the session ID had a natural grouping unit:

typescript
// Before 2026-07-28: session ID was a protocol primitive
const sessionId = request.headers['mcp-session-id'];
const budget = await store.get(budget:${sessionId});

That's gone.
The spec's own guidance is direct: "If your server needs to carry state across calls, mint an explicit handle from a tool and have the model pass it back as an argument."

Which means your budget scope is now something you define, not something the protocol gives you:

typescript
// After 2026-07-28: you own the scope
const agentRunId = context.runId; // your construct, not the protocol's
const budget = await store.get(budget:${agentRunId});

This is a cleaner model.
It's also opt-in work.
If you weren't tracking session-scoped cost before, you definitely don't get it automatically now.

Tasks shift cost accumulation async

The Tasks extension is now first-class under io.modelcontextprotocol/tasks.
A tools/call can return a task handle instead of a synchronous result.
The client drives the task forward with tasks/get, tasks/update, and tasks/cancel.

typescript
const result = await mcp.callTool({ name: 'deep-research', params });

if (result.type === 'task') {
// Pre-call budget check has already run and exited.
// LLM work is happening async from here.
let taskResult;
do {
taskResult = await mcp.tasks.get(result.taskId);
} while (taskResult.status !== 'complete');
}

The cost tracking problem: your pre-call guard ran before tools/call went out.
It saw the call, checked the session budget, let it through.
The server converted it to a Task.
The actual LLM work — potentially many internal calls, potentially expensive models — runs async while your polling loop waits.

The guard did its job.
It just had no visibility into what happened inside the Task.

Tasks are the right primitive for long-running agent work.
But they move cost accumulation to a point that call-level-only guards can't reach.

What to do about it

Define session scope explicitly — don't assume the protocol gives it to you.

Use your agent run ID, a user session ID, or a handle the model threads between tool calls.
Make the budget scope an explicit construct in your application:

typescript
const session = {
id: crypto.randomUUID(),
budgetCents: 50,
spentCents: 0,
reservedCents: 0,
};

Carry budget tracking through the Task polling lifecycle.

Check on each poll.
If the session is over budget, cancel the task — tasks/cancel is part of the spec:

typescript
while (taskResult.status !== 'complete') {
await updateSpend(session, taskResult.progressTokens);

if (session.spentCents + session.reservedCents >= session.budgetCents) {
await mcp.tasks.cancel(taskResult.taskId);
throw new BudgetExceededError(session.id);
}

taskResult = await mcp.tasks.get(taskResult.taskId);
}

Reserve budget at task start, not just at task check.

Estimate the maximum cost of the task before you let the tools/call through. Reserve that amount.
If the reservation would exceed the session limit, block it before the task starts:

typescript
const estimatedMaxCost = estimateTaskCost(toolName, params);

if (session.spentCents + estimatedMaxCost > session.budgetCents) {
throw new BudgetExceededError(Task would exceed session budget);
}

session.reservedCents += estimatedMaxCost;
// Now let the tools/call through

Estimation will be imperfect.
A reasonable ceiling is still better than no guard against a task that runs to completion at full cost.

The broader pattern

Supabase's note in the spec blog is worth reading: they're using the new MRTR (Multi Round-Trip Requests) to ask the user for confirmation before executing a tool that has a real cost — like creating a new project or running a destructive query.
The tool pauses mid-execution and gets explicit sign-off.

That's the right instinct. The same confirmation pattern applies to budget: a tool that's about to start an expensive Task should check whether the session can absorb the cost before it commits, not trust that the pre-call check was sufficient.

This is the architectural pattern AI CostGuard is built around — session-scoped budget with a pre-call decision point, not just a per-call token counter.
The MCP spec today made that distinction explicit at the protocol level by removing the session primitive and making async Task execution first-class at the same time.

If you were relying on Mcp-Session-Id for budget scope, migrate now.
If you weren't tracking session-level cost at all, this release is a good moment to start.
https://github.com/salimassili62-afk/ai-costguard

GhostApproval: Why Agent Permissions Need Resolved Runtime State

Assili Salim — Mon, 13 Jul 2026 03:47:49 +0000

A malicious repo makes an AI coding assistant appear to edit project_settings.json while the actual write lands on ~/.ssh/authorized_keys. The approval UI shows the harmless path. The filesystem follows the symlink.
The bug isn't symlinks. It's this:
An approval prompt is only strong if it shows the resolved operation, not the requested one.

The approval showed a lie
Common agent mistake: approve the visible request instead of what the runtime actually does.
Visible request: Edit project_settings.json
Resolved operation: Write to ~/.ssh/authorized_keys
If the human sees only the first, approval is theater.

Fix: resolve before you approve
Before showing an approval prompt, the runtime should resolve the actual target:
typescriptimport path from "node:path";
import fs from "node:fs/promises";

async function resolveWriteTarget(workspaceRoot: string, requestedPath: string) {
const workspace = await fs.realpath(workspaceRoot);
const absolute = path.resolve(workspace, requestedPath);
const parent = await fs.realpath(path.dirname(absolute));
const resolved = path.join(parent, path.basename(absolute));

return {
requested: absolute,
resolved,
safe: resolved.startsWith(workspace + path.sep),
};
}
Don't approve the string. Approve the canonical target.
Then add a gate:
typescriptasync function beforeFileWrite(workspace, path, content) {
const target = await resolveWriteTarget(workspace, path);

if (!target.safe) {
return { allowed: false, reason: "write_outside_workspace", target };
}

return { allowed: true, target };
}
The check happens before execution. Not after. Not in an audit log.

Same pattern for provider calls
GhostApproval is about files. The design applies to every agent operation.
Before a provider call executes, resolve:

Which model?
Known price?
Which run?
Retry count?
Step count?
Prompt loop?
Budget remaining?
Making progress?

Naive:
typescriptconst result = await provider.call({ model, messages });
Safer:
typescriptconst decision = guard.beforeCall({
runId, model, messages, stepCount, retryCount, budgetRemaining, progressState,
});

if (!decision.allowed) return { status: "stopped", reason: decision.reason };

const result = await provider.call({ model, messages });
The key is placement. Decide before.

Human approval alone isn't enough
GhostApproval is a reminder that formal approval can be practically weak.
If the human sees a simplified story, approval is not informed. If the operation already executed, approval is not a gate. If the runtime hides resolved details, the human approved the wrong thing.
A real approval prompt shows what matters:
typescripttype ApprovalPrompt = {
action: "file_write" | "shell_command" | "provider_call";
requested?: string;
resolved?: string; // The actual target
model?: string;
estimatedCost?: number;
budgetRemaining?: number;
};
Users shouldn't infer the real operation. The runtime should expose it.

The pattern
Resolve the real operation.
Check the policy.
Then decide.
Then execute.
This is what AI CostGuard does for provider calls—pre-call guards that catch retry storms, prompt loops, budget overruns, and step explosions before they happen. Not after. Not in a dashboard. Before.

Takeaway
GhostApproval isn't just a symlink story. It's a runtime design story.
Don't ask humans to approve abstractions. Show them what's actually happening. Make the decision before execution. Then run.
https://github.com/salimassili62-afk/ai-costguard

Agent Telemetry Is Not Agent Control

Assili Salim — Sun, 12 Jul 2026 20:51:59 +0000

GitHub's Copilot updates added session streaming, cost tracking, OpenTelemetry exports. Useful. But here's the thing: seeing what went wrong and stopping it before it happens are not the same layer.
You can log that an agent retried 14 times. You can't prevent call 15 from your dashboard.
This is the gap.
Telemetry = evidence. Control = prevention.
Telemetry answers: "What happened?"
Runtime control answers: "Should this next call happen?"
Both matter. They just happen at different times.

Why agents break this worse than normal LLM calls
A normal model call is done in one roundtrip. Request → response → log it → move on.
An agent runs in a loop. It calls the model, reads results, retries, delegates, switches strategies, keeps going. One bad call might look fine. The whole sequence wastes money.
Telemetry will show you this beautifully after it happens. A runtime guard stops it before.

The basic agent loop (vulnerable)
javascriptwhile (!task.done) {
const response = await provider.call({
model: task.model,
messages: task.messages,
});

telemetry.record({ runId: task.id, model, response });
task = await applyAgentStep(task, response);
}
You get evidence. Not protection.
Add a decision before the call (safer)
javascriptconst decision = guard.beforeCall({
runId: task.id,
model: task.model,
stepCount: task.steps.length,
retryCount: task.retryCount,
budgetRemaining: task.budgetRemaining,
progressState: task.progress,
});

if (!decision.allowed) {
telemetry.record({ runId: task.id, status: "stopped", reason: decision.reason });
return { status: "stopped", reason: decision.reason };
}

const response = await provider.call({ model, messages });
Placement matters. The guard decides before. Telemetry records after.

Six checks that belong before every provider call

Known model pricing If you can't price it, you can't budget it. Model names matter—aliases and rewrites break cost assumptions.
Task-level budget User is under monthly limit? Great. This agent run can still be wasting work. Both levels matter.
Max-step limits Agents run until something stops them. A step limit isn't sophisticated. That's why it works.
Retry storms Retries are good. Repeated failure loops are not. Stop similar errors after N retries.
Prompt loops Agents get stuck asking nearly the same question again. The text changes. The task doesn't. Detect it.
No-progress detection Track: tests passing, errors changing, tool results adding info, checklist items completing. If none move after several steps, stop.

Better telemetry includes stop reasons
Pre-call control doesn't replace telemetry. It makes telemetry better.
Record blocked calls, not just executed ones:
javascripttype AgentDecisionEvent = {
runId: string;
model: string;
stepCount: number;
allowed: boolean;
reason?: "budget_exceeded" | "retry_storm" | "prompt_loop" | "no_progress";
};
Now you can ask: Which workflows trigger prompt loops? Which teams need tighter limits? Which models have unknown pricing? That's more useful than total spend.

The takeaway
Agent observability is table stakes now. GitHub's got you there.
The next layer is admission control. Let the runtime say no before execution, not in the dashboard after.
Record what happened. Before the next call, decide if it should happen at all.

This is what AI CostGuard does—pre-call guards for production agent applications. Catches retry storms, prompt loops, budget overruns before they execute.
https://github.com/salimassili62-afk/ai-costguard

AI Agents Need Runtime State Checks, Not Just Better Prompts

Assili Salim — Sat, 11 Jul 2026 04:46:23 +0000

Claude Code’s July 8 changelog is a useful reminder of what production agent engineering actually looks like.
The interesting parts are not model benchmarks.
They are state-management fixes.
Claude Code 2.1.205 fixed a message sent while Claude was working being silently lost when the turn ended at the --max-turns limit. It also fixed background agents staying shown as “failed” or “completed” after being resumed, background jobs flipping from “needs input” back to “working” with no readable text, and stale “Running” status in web and mobile Remote Control panels. �
Claude
The same changelog added a safety detail: background task notifications now explicitly state that no human input occurred, preventing fabricated in-transcript approvals from being acted on. �
Claude
These are not flashy issues.
They are the issues that show up when agents become real runtime systems.
Agents are state machines
A coding agent is not only a model call.
It has state:
current step
current task status
previous messages
tool results
approvals
turn limits
retry count
background job state
user input state
stop reason
If that state is wrong, the agent can behave incorrectly even if the model response is good.
A stale “Running” state can mislead the user.
A lost message can cause the agent to continue without new context.
A fake approval inside a transcript can be dangerous if treated as real input.
A max-turn limit can stop the model while leaving the surrounding runtime ambiguous.
This is why agent reliability is not just prompt quality.
It is runtime correctness.
The dangerous pattern: activity without permission
A common agent failure is not a crash.
It is continuation.
The agent keeps moving because nothing clearly told it to stop.
That can happen when:
the task status is stale
the stop condition is unclear
the agent hits a max-turn limit but the runtime still continues
a retry policy ignores the reason for failure
generated text is mistaken for user approval
background state is out of sync
For cost-sensitive agents, this matters.
Every unnecessary continuation can become another provider call.
The invoice will show the usage later.
The runtime should prevent the obviously wrong continuation before it happens.
Treat every provider call as an admission decision
A safer pattern is to check runtime state before every provider call.
type AgentRuntimeState = {
runId: string;
status: "working" | "needs_input" | "stopped" | "failed" | "completed";
stepCount: number;
maxSteps: number;
retryCount: number;
budgetRemaining: number;
model: string;
modelPriceKnown: boolean;
hasRealUserApproval: boolean;
lastStopReason?: string;
recentProgress: boolean;
};
Then make a decision before calling the provider.
function beforeProviderCall(state: AgentRuntimeState) {
if (state.status === "needs_input") {
return { allowed: false, reason: "needs_user_input" };
}

if (!state.hasRealUserApproval) {
return { allowed: false, reason: "missing_real_approval" };
}

if (state.stepCount >= state.maxSteps) {
return { allowed: false, reason: "max_steps_exceeded" };
}

if (!state.modelPriceKnown) {
return { allowed: false, reason: "unknown_model_pricing" };
}

if (state.budgetRemaining <= 0) {
return { allowed: false, reason: "budget_exceeded" };
}

if (!state.recentProgress && state.retryCount > 0) {
return { allowed: false, reason: "no_progress_retry" };
}

return { allowed: true };
}
The exact API is not important.
The placement is important.
The check happens before the provider call.
Generated approval is not approval
The approval issue is especially important.
An agent transcript may contain text that looks like permission.
That does not mean a human gave permission.
For production systems, approval should be an explicit runtime event.
Not a string inside generated text.
A safer shape:
type ApprovalEvent = {
runId: string;
approvedBy: "human" | "policy";
approvedAt: number;
scope: "tool_call" | "provider_call" | "file_edit";
};
Then the runtime checks the approval event, not the transcript.
function hasValidApproval(events: ApprovalEvent[], scope: ApprovalEvent["scope"]) {
return events.some(
event =>
event.scope === scope &&
(event.approvedBy === "human" || event.approvedBy === "policy")
);
}
This prevents a model-generated sentence from becoming operational permission.
Max-turns should produce a stop reason
A turn limit should not be a vague failure.
It should create a structured stop reason.
if (turnCount >= maxTurns) {
return {
status: "stopped",
reason: "max_turns_exceeded",
nextAction: "requires_human_review",
};
}
This makes the runtime easier to debug.
It also prevents accidental continuation.
If the agent stopped because it hit a limit, the next provider call should not happen automatically.
Status should control execution
Background agents make status accuracy critical.
If the UI says “Running,” but the task is actually blocked, the user may assume progress is happening.
If the runtime says “working,” but the agent needs input, the system may continue incorrectly.
A simple rule helps:
The runtime status should be the source of truth for execution.
if (task.status !== "working") {
return {
allowed: false,
reason: task_status_${task.status},
};
}
Do not let the provider call path ignore the task state machine.
Where AI CostGuard fits
This is the kind of failure mode I’m building AI CostGuard around.
AI CostGuard is a local-first TypeScript/Node.js pre-call runtime guard for AI-agent applications.
It focuses on risky provider calls before execution:
retry storms
prompt loops
max-step explosions
runaway agent execution
unknown model pricing
budget overruns
uncontrolled provider calls
It is not a billing ledger.
It is not a hard security boundary.
It does not replace provider dashboards.
The goal is narrower: give the runtime a way to say “this next call should not execute.”
Takeaway
Agent failures are becoming runtime failures.
Not just bad prompts.
Not just weak models.
State bugs.
Approval bugs.
Stale status.
Lost messages.
Unclear stop reasons.
If your agent can run in the background, it needs a real state machine.
And before every provider call, that state machine should decide whether the call is still allowed.

GPT-5.6 Ultra Mode Means Agent Budgets Need to Handle Subagents

Assili Salim — Fri, 10 Jul 2026 08:13:53 +0000

OpenAI released GPT-5.6 across ChatGPT, Codex, and the API on July 9.

The model family has three tiers: Sol, Terra, and Luna. OpenAI lists API pricing at $5 input / $30 output per million tokens for Sol, $2.50 / $15 for Terra, and $1 / $6 for Luna.

The pricing matters.

But if you build AI agents, the bigger detail is this:

GPT-5.6 adds max reasoning effort and an ultra mode that uses subagents to accelerate complex work. In the API, OpenAI says multi-agent support can run concurrent subagents and synthesize their work in a single request.

That changes how runtime budgets should work.

A single-call budget is not enough when the task can fan out.

The naive budget

A simple budget guard might look like this:

const estimatedCost =
inputTokens * inputPrice +
maxOutputTokens * outputPrice;

if (estimatedCost > budgetRemaining) {
throw new Error("Budget exceeded");
}

This is a decent start.

But it assumes the next call is the whole unit of risk.

That assumption breaks down when an agent can create subagents.

A parent agent may start several branches.

Each branch may call a model.

Each branch may use tools.

Each branch may retry.

The parent may then call the model again to synthesize results.

The real budget is not only the next call.

It is the execution tree.

Think in execution trees

A multi-agent workflow can be modeled like this:

type AgentNode = {
id: string;
parentId?: string;
model: string;
effort: "low" | "medium" | "high" | "max" | "ultra";
stepCount: number;
retryCount: number;
budgetRemaining: number;
};

The parent task needs a shared budget.

Each subagent needs a local budget.

type WorkflowBudget = {
workflowId: string;
totalRemaining: number;
perAgentLimit: number;
perCallLimit: number;
};

This gives you three useful boundaries:

per-call budget

per-subagent budget

whole-workflow budget

You need all three.

A call can be safe locally while the subagent is wasting work.

A subagent can be safe locally while the whole workflow is burning too much in parallel.

Add a pre-call decision

Before every provider call, make the runtime decide whether the call should happen.

type BeforeCallInput = {
workflowId: string;
agentId: string;
parentAgentId?: string;
model: string;
effort: string;
prompt: string;
estimatedInputTokens: number;
maxOutputTokens: number;
stepCount: number;
retryCount: number;
budgetRemainingForAgent: number;
budgetRemainingForWorkflow: number;
previousPrompts: string[];
progressState: {
filesChanged?: boolean;
testsImproved?: boolean;
errorsChanged?: boolean;
objectiveCompleted?: boolean;
};
};

type GuardDecision =
| { allowed: true }
| {
allowed: false;
reason:
| "unknown_model_pricing"
| "call_budget_exceeded"
| "agent_budget_exceeded"
| "workflow_budget_exceeded"
| "max_steps_exceeded"
| "retry_storm"
| "prompt_loop"
| "no_progress";
};

Then use it before the provider call:

const decision = guard.beforeCall({
workflowId,
agentId,
parentAgentId,
model,
effort,
prompt,
estimatedInputTokens,
maxOutputTokens,
stepCount,
retryCount,
budgetRemainingForAgent,
budgetRemainingForWorkflow,
previousPrompts,
progressState,
});

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
};
}

const result = await provider.call({
model,
prompt,
});

The exact API is not the point.

The placement is the point.

The decision happens before the provider call.

Check model pricing

GPT-5.6 has multiple model tiers with different prices.

That means the runtime should not treat model names as harmless strings.

if (!pricingCatalog.has(model)) {
return {
allowed: false,
reason: "unknown_model_pricing",
};
}

Do not guess.

Unknown pricing should fail closed.

This matters even more when agents can escalate, fallback, or fan out.

Check effort level

Effort level is not only a quality setting.

It is an execution-policy setting.

if (effort === "ultra" && !taskPolicy.allowUltra) {
return {
allowed: false,
reason: "effort_not_allowed",
};
}

A small task should not silently use the most expensive execution shape.

A high-value task might deserve it.

The runtime should know the difference.

Check workflow budget

For subagents, local checks are not enough.

if (estimatedNextCallCost > budgetRemainingForWorkflow) {
return {
allowed: false,
reason: "workflow_budget_exceeded",
};
}

This prevents many individually reasonable branches from consuming the shared ceiling.

Parallel waste is still waste.

Check subagent budget

Each branch also needs a local ceiling.

if (estimatedNextCallCost > budgetRemainingForAgent) {
return {
allowed: false,
reason: "agent_budget_exceeded",
};
}

If one subagent is stuck, it should stop without consuming the parent workflow.

Check max steps

Subagents need step limits too.

if (stepCount >= maxSteps) {
return {
allowed: false,
reason: "max_steps_exceeded",
};
}

A branch that cannot produce useful work in a reasonable number of steps should stop cleanly.

Check prompt loops

If a subagent keeps asking nearly the same thing, the runtime should treat that as a loop.

if (similarToRecentPrompt(prompt, previousPrompts)) {
return {
allowed: false,
reason: "prompt_loop",
};
}

This is especially useful in coding agents.

A stuck agent often rephrases the same failing approach.

Check no progress

A subagent can be active without being useful.

Track signals like:

tests improved
error changed
relevant files changed
objective completed
retrieved context changed
final answer got closer

If none of those move, stop.

if (!madeProgress(progressState, recentSteps)) {
return {
allowed: false,
reason: "no_progress",
};
}

A stopped branch is better than a branch that looks busy while wasting calls.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript/Node.js runtime safety layer I’m building for AI-agent projects.

It focuses on pre-call checks for:

retry storms
prompt loops
max-step explosions
runaway agent execution
unknown model pricing
budget overruns
uncontrolled provider calls
invisible AI-agent cost risk

It is not a billing ledger.

It is not a hard security boundary.

It does not replace provider dashboards.

The goal is narrow:

help the runtime decide whether the next provider call should execute.

Takeaway

Multi-agent execution changes the cost-control problem.

You are no longer guarding one prompt.

You are guarding an execution tree.

That means budgets need to exist at three levels:

per call

per subagent

per workflow

Once agents can fan out, the invoice is even later than it used to be.
https://github.com/salimassili62-afk/ai-costguard

Always-Running AI Agents Need Pre-Call Stop Conditions

Assili Salim — Wed, 08 Jul 2026 21:02:58 +0000

Claude Cowork can now keep working after you close your laptop.

WIRED reports that Anthropic expanded Cowork beyond the desktop app, so users do not need an active desktop session to keep tasks running. Users can interact with limited versions through the Claude smartphone app or web browser, and Cowork can continue tasks after the user clocks out.

That is a useful feature.

It also creates a runtime engineering problem.

If agents can run unattended, they need stop conditions before provider calls execute.

A chatbot waits. An agent continues.

A chatbot is mostly reactive.

The user sends a message.

The model responds.

The interaction pauses.

An agent is different.

It may:

call a model
call tools
inspect results
add context
retry
call another tool
generate a document
send another provider request
keep going while the user is away

That means the agent runtime needs controls.

Not only logs.

Not only dashboards.

Controls before execution.

The naive loop

A simple agent loop might look like this:

while (!task.done) {
const response = await provider.call({
model: task.model,
messages: task.messages,
});

task = await applyAgentStep(task, response);
}

This is easy to write.

It is also unsafe for unattended workflows.

There is no max-step limit.

No budget check.

No retry-storm detection.

No prompt-loop detection.

No known-pricing check.

No no-progress stop.

If the agent gets stuck, it keeps creating provider calls until something external stops it.

That “something” might be a provider error, user intervention, an account limit, or a bill.

None of those are ideal runtime controls.

Add a pre-call decision

A safer loop puts a guard before every provider call.

const decision = guard.beforeCall({
runId: task.id,
model: task.model,
messages: task.messages,
stepCount: task.steps.length,
retryCount: task.retryCount,
budgetRemaining: task.budgetRemaining,
previousPrompts: task.previousPrompts,
progressState: task.progress,
});

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
error: decision.error,
};
}

const response = await provider.call({
model: task.model,
messages: task.messages,
});

The exact API does not matter.

The placement matters.

The runtime checks the call before the provider sees it.

What should the runtime check?

Known model pricing

If the runtime cannot price the model, it cannot enforce a reliable budget.

if (!pricingCatalog.has(model)) {
return {
allowed: false,
reason: "unknown_model_pricing",
};
}

Do not guess inside an unattended loop.

A typo, fallback, gateway rewrite, or model alias can break cost assumptions.

Task budget

Every unattended run should have a task-level budget.

if (estimatedNextCallCost > budgetRemaining) {
return {
allowed: false,
reason: "budget_exceeded",
};
}

Monthly dashboards are useful, but they are late.

A task budget stops the next call before spend is created.

Max steps

Agents need explicit step limits.

if (stepCount >= maxSteps) {
return {
allowed: false,
reason: "max_steps_exceeded",
};
}

A run that cannot finish inside a reasonable number of steps should stop cleanly.

Retry storms

Retries are normal.

Retry storms are not.

if (retryCount >= maxRetries && recentErrorsAreSimilar(errors)) {
return {
allowed: false,
reason: "retry_storm_detected",
};
}

The goal is not to remove retries.

The goal is to prevent repeated failure from becoming the workload.

Prompt loops

Agents often get stuck by asking almost the same thing again.

if (similarToRecentPrompt(currentPrompt, previousPrompts)) {
return {
allowed: false,
reason: "similar_prompt_loop",
};
}

This catches a common pattern:

the agent looks active, but it is not exploring a new path.

No progress

A run can consume steps while producing no useful movement.

Track progress signals:

errors decreasing
files changing meaningfully
tests improving
checklist items completing
retrieved information changing
user-defined success criteria improving

If progress does not change after several steps, stop.

if (!madeProgress(recentSteps)) {
return {
allowed: false,
reason: "no_progress",
};
}
Why this matters more for always-running agents

When the user is watching, some failures are caught manually.

When the agent runs overnight, the runtime becomes the first line of control.

That changes the design requirement.

An unattended agent needs:

local budgets
max-step policies
retry budgets
prompt-loop detection
known model pricing
structured stop reasons

These are not advanced features.

They are basic operating rules.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript/Node.js runtime safety layer I’m building for AI-agent projects.

It focuses on pre-call protection for:

retry storms
prompt loops
max-step explosions
runaway agent execution
unknown model pricing
budget overruns
uncontrolled provider calls
invisible AI-agent cost risk

It is not a billing ledger.

It is not a hard security boundary.

It does not replace provider dashboards.

The goal is to help the runtime decide whether the next provider call should execute.

Takeaway

Always-running agents are useful because they remove friction.

That same lack of friction is the risk.

If an agent can keep working after the laptop closes, it needs runtime rules for when to stop.
https://github.com/salimassili62-afk/ai-costguard

Tokenizer Changes Can Break AI-Agent Budget Assumptions

Assili Salim — Sat, 04 Jul 2026 08:41:45 +0000

Claude Sonnet 5 is a good reminder that AI-agent cost control is not only about model pricing.

Anthropic says Sonnet 5 launches at $2 per million input tokens and $10 per million output tokens through August 31, 2026, then moves to $3 per million input tokens and $15 per million output tokens.

Vercel’s AI Gateway changelog adds an important implementation detail: Sonnet 5 uses an updated tokenizer, and the same input can map to more tokens.

That matters if you build agents.

A model can have attractive pricing and still change your runtime cost assumptions.

The problem

Many agent systems estimate cost like this:

const estimatedCost =
inputTokens * inputPrice +
maxOutputTokens * outputPrice;

That is fine as a start.

But it assumes the runtime knows:

which model is being used
how the input is tokenized
the correct input price
the correct output price
how many retries are allowed
how many steps are allowed
whether the agent is making progress

If any of those assumptions drift, the estimate becomes weaker.

A tokenizer update is one way this happens.

The prompt text may be the same.

The token count may not be.

Agents amplify small estimation errors

For a single request, a token-count surprise is usually manageable.

For an agent, it can compound.

An agent may:

call the model
inspect files
add context
run tools
retry
ask a similar prompt
switch models
continue for more steps

Small cost drift per call becomes bigger across a run.

Now add parallel agents, fallback paths, or long-context workflows.

The problem is no longer “What does this model cost?”

The problem is:

Should this next provider call be allowed?

Add a pre-call decision

Before the provider call, add a guard decision.

type BeforeCallInput = {
runId: string;
model: string;
prompt: string;
estimatedInputTokens: number;
maxOutputTokens: number;
stepCount: number;
retryCount: number;
budgetRemaining: number;
previousPrompts: string[];
};

type GuardDecision =
| { allowed: true }
| {
allowed: false;
reason:
| "unknown_model_pricing"
| "budget_exceeded"
| "max_steps_exceeded"
| "retry_storm"
| "prompt_loop";
};

Then use it before execution:

const decision = guard.beforeCall({
runId,
model,
prompt,
estimatedInputTokens,
maxOutputTokens,
stepCount,
retryCount,
budgetRemaining,
previousPrompts,
});

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
};
}

const result = await provider.call({
model,
prompt,
});

The important part is placement.

The check happens before the provider call.

Not after the bill.

Fail closed on unknown pricing

If the runtime does not know the model price, it cannot enforce a reliable budget.

if (!pricingCatalog.has(model)) {
return {
allowed: false,
reason: "unknown_model_pricing",
};
}

Do not guess.

A model alias, fallback, gateway rewrite, or typo can break your assumptions.

Unknown pricing should stop the run before the call executes.

Recalculate tokens after model migration

When changing models, do not assume the same prompt has the same token count.

A basic migration checklist:

run representative prompts through the new tokenizer
compare input token counts
compare output length behavior
update pricing metadata
retest max-step limits
retest retry behavior
check fallback paths
measure cost per successful task

Cost per token is useful.

Cost per successful task is more useful.

Watch retry and prompt-loop behavior

A smarter model may finish tasks in fewer steps.

It may also continue longer because it can pursue more complex plans.

Your runtime should still stop obvious waste.

if (stepCount >= maxSteps) {
return { allowed: false, reason: "max_steps_exceeded" };
}

if (retryCount >= maxRetries) {
return { allowed: false, reason: "retry_storm" };
}

if (similarToRecentPrompt(prompt, previousPrompts)) {
return { allowed: false, reason: "prompt_loop" };
}

These checks do not make the model better.

They make the system safer to operate.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript/Node.js runtime safety layer I’m building for AI-agent projects.

It focuses on pre-call safety checks for:

retry storms
prompt loops
max-step explosions
runaway execution
unknown model pricing
budget overruns
uncontrolled provider calls

It is not a billing ledger.

It is not a hard security boundary.

It does not replace provider dashboards.

The goal is to catch obviously risky calls before they execute.

Takeaway

Model pricing pages tell you the unit price.

They do not tell you whether your agent should make the next call.

Tokenizer changes, fallback models, retries, and prompt loops all affect real runtime cost.

For AI agents, cost safety belongs before the provider call.

Gateway Routing Helps AI Apps. Agent Runtimes Still Need Pre-Call Guards.

Assili Salim — Fri, 03 Jul 2026 07:22:32 +0000

Vercel added routing rules to AI Gateway on July 2.

Routing rules let teams rewrite or deny model requests at the gateway level. A rewrite rule serves a request for one model using another model. A deny rule blocks a model and returns a 403. Vercel lists use cases like rerouting when a model is down or retired, standardizing on one model, routing an expensive model to a cheaper one, or keeping a team off unapproved models.

That is useful infrastructure.

But if you are building AI agents, it is not the whole safety layer.

A gateway can decide where a request goes.

Your runtime still needs to decide whether the request should happen.

The difference

Gateway routing answers:

"Which model should serve this request?"

Runtime guarding answers:

"Should this request be allowed at all?"

Those are different problems.

A gateway may rewrite:

anthropic/claude-opus-4.8 -> anthropic/claude-haiku-4.5

That can keep traffic moving.

But the gateway may not know:

the agent already retried 12 times
the current prompt is nearly identical to previous failed prompts
the run exceeded its task budget
the agent passed its max-step limit
tool calls are happening without progress
the fallback model keeps the loop alive but does not improve the task

That context usually lives inside the agent runtime.

A naive agent loop

Many agent loops start like this:

while (!task.done) {
const response = await provider.call({
model: task.model,
messages: task.messages,
});

task = await applyAgentStep(task, response);
}

This is simple.

It is also missing the controls that matter in production.

There is no budget check.

No max-step check.

No retry-storm detection.

No prompt-loop detection.

No unknown-pricing block.

No no-progress stop.

If a gateway rewrites the model, this loop may keep running.

That is not always what you want.

Add a pre-call guard

A safer pattern puts a local decision before the provider call:

const decision = guard.beforeCall({
runId: task.id,
model: task.model,
messages: task.messages,
stepCount: task.steps.length,
retryCount: task.retryCount,
previousPrompts: task.previousPrompts,
budgetRemaining: task.budgetRemaining,
progressState: task.progress,
});

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
error: decision.error,
};
}

const response = await provider.call({
model: task.model,
messages: task.messages,
});

The exact API does not matter.

The placement matters.

The guard runs before the provider call.

What should the runtime check?

Known model pricing

If the runtime cannot price the model, it cannot enforce a reliable budget.

if (!pricingCatalog.has(model)) {
return {
allowed: false,
reason: "unknown_model_pricing",
};
}

This matters even more when routing and fallback rules exist.

A rewritten model still has a cost profile.

The runtime should know it.

Budget remaining

Task-level budgets are different from account-level limits.

if (estimatedNextCallCost > budgetRemaining) {
return {
allowed: false,
reason: "budget_exceeded",
};
}

A monthly dashboard can show spend later.

A runtime budget can stop the next call now.

Max steps

Agents should have explicit stopping rules.

if (stepCount >= maxSteps) {
return {
allowed: false,
reason: "max_steps_exceeded",
};
}

A model route can change.

The step limit should still apply.

Retry storms

Retries are useful.

Blind retries are not.

if (retryCount > maxRetries && recentErrorsAreSimilar(errors)) {
return {
allowed: false,
reason: "retry_storm_detected",
};
}

A fallback model can hide a retry storm by keeping the run alive.

The runtime should detect the pattern.

Prompt loops

Agents sometimes ask almost the same thing repeatedly.

if (similarToRecentPrompt(currentPrompt, previousPrompts)) {
return {
allowed: false,
reason: "similar_prompt_loop",
};
}

If the prompt is not changing meaningfully, the model route may not be the main issue.

The agent may be stuck.

No progress

A run can be active without improving.

Useful progress signals include:

tests passing
errors decreasing
files changing meaningfully
task checklist items completing
user-defined success criteria improving

If the agent consumes steps without progress, stop.

Layer the controls

A good AI-agent architecture can use both gateway policy and runtime guards.

One possible flow:

agent wants next call
↓
local runtime guard checks run state
↓
gateway applies model routing or deny rules
↓
provider executes request
↓
logs and dashboards record result

The order matters.

The runtime has run context.

The gateway has team-level model policy.

The provider executes.

The dashboard explains.

Do not ask one layer to do all four jobs.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript / Node.js runtime safety layer I’m building for this exact class of problem.

It focuses on pre-call checks for AI-agent projects:

retry storms
prompt loops
max-step explosions
runaway agent execution
unknown model pricing
budget overruns
uncontrolled provider calls

It is not a billing ledger.

It is not a hard security boundary.

It does not replace provider dashboards or gateway routing.

The goal is narrower:

help the agent runtime decide whether the next provider call should execute.

Takeaway

Gateway routing is useful.

It centralizes model policy.

It helps teams move traffic when models are down, retired, too expensive, or not approved.

But routing does not replace runtime safety.

A cheaper fallback can still waste money.

A policy-approved model can still be part of a prompt loop.

A valid request can still exceed the task budget.

For AI agents, the critical question happens before the call:

Should this request exist?
https://github.com/salimassili62-afk/ai-costguard

Reusable Agent Skills Need Pre-Call Runtime Checks

Assili Salim — Sun, 28 Jun 2026 14:10:34 +0000

OpenAI’s recent Codex research includes one detail that matters for developers building agents:

26.6% of users use skills to share instructions for complex workflows, and more than 10% manage three or more concurrent Codex agents at some point each week.

That means agent usage is moving from one-off prompts toward reusable workflows.

That is good.

It also means failures can become reusable.

The problem

A bad prompt can waste one model call.

A bad agent skill can waste many runs.

A skill might encode:

how to retry
how to call tools
how to inspect files
how to recover from errors
how much context to add
when to continue
when to stop

If those rules are loose, every run inherits the looseness.

This is the part developers need to treat carefully.

Reusable agent behavior needs reusable runtime boundaries.

A naive agent skill

Imagine a coding-agent skill for fixing failing tests.

The instruction might be:

const skill = {
name: "fix-failing-tests",
instructions: Inspect the failing test. Find the relevant files. Apply a fix. Run the tests again. Repeat until the tests pass.,
};

This sounds fine.

But “repeat until the tests pass” is dangerous without runtime limits.

What if the test failure is environmental?

What if the agent keeps editing unrelated files?

What if the prompt barely changes across attempts?

What if each retry adds more context?

What if the fallback model has unknown pricing?

The skill is useful.

The runtime is under-specified.

Add a pre-call decision

Before every provider call, the runtime should decide whether the call is still allowed.

type BeforeCallInput = {
runId: string;
workflowId?: string;
model: string;
prompt: string;
stepCount: number;
retryCount: number;
budgetRemaining: number;
previousPrompts: string[];
progressState: {
testsImproved?: boolean;
errorsChanged?: boolean;
filesChanged?: boolean;
};
};

type GuardDecision =
| { allowed: true }
| {
allowed: false;
reason:
| "unknown_model_pricing"
| "budget_exceeded"
| "max_steps_exceeded"
| "retry_storm_detected"
| "similar_prompt_loop"
| "no_progress";
error: Error;
};

Then use it before the provider call:

const decision = guard.beforeCall({
runId,
workflowId,
model,
prompt,
stepCount,
retryCount,
budgetRemaining,
previousPrompts,
progressState,
});

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
error: decision.error,
};
}

const response = await provider.call({
model,
prompt,
});

The exact API does not matter.

The placement matters.

The check happens before the provider call.

What should the guard check?

Known model pricing

If the runtime cannot price the model, it cannot enforce a budget.

if (!pricingCatalog.has(model)) {
return {
allowed: false,
reason: "unknown_model_pricing",
error: new Error(Unknown pricing for model: ${model}),
};
}

Do not guess.

Fail closed.

Budget remaining

Agent workflows should have task-level budgets.

if (estimatedNextCallCost > budgetRemaining) {
return {
allowed: false,
reason: "budget_exceeded",
error: new Error("Agent run budget exceeded"),
};
}

A small bug fix and a long migration should not share the same budget.

Max steps

Agents need step limits.

if (stepCount >= maxSteps) {
return {
allowed: false,
reason: "max_steps_exceeded",
error: new Error("Maximum agent steps exceeded"),
};
}

This is basic production hygiene.

If a workflow cannot complete inside a reasonable number of steps, it should stop.

Retry storms

Retries are useful.

Blind retries are expensive.

if (retryCount >= maxRetries && recentErrorsAreSimilar(errors)) {
return {
allowed: false,
reason: "retry_storm_detected",
error: new Error("Retry storm detected"),
};
}

The goal is not to ban retries.

The goal is to stop repeated failure.

Prompt loops

A prompt loop happens when the agent keeps asking nearly the same thing.

if (similarToRecentPrompt(prompt, previousPrompts)) {
return {
allowed: false,
reason: "similar_prompt_loop",
error: new Error("Similar prompt loop detected"),
};
}

Even a simple similarity check can catch obvious loops.

No progress

A run can be active and still not improve.

Track progress signals:

did tests improve?
did the error change?
did files change meaningfully?
did a checklist item complete?
did the agent reduce uncertainty?

If several steps pass without progress, stop.

if (!madeProgress(progressState, recentSteps)) {
return {
allowed: false,
reason: "no_progress",
error: new Error("Agent run is not making progress"),
};
}
Why concurrency makes this more important

OpenAI’s Codex research says more than 10% of users manage three or more concurrent agents at some point each week.

That changes the risk.

One agent wasting a few calls is visible.

Several agents each wasting a few calls can look normal.

The local loop becomes a global budget problem.

For parallel workflows, add shared budget checks:

if (estimatedNextCallCost > workflowBudgetRemaining) {
return {
allowed: false,
reason: "workflow_budget_exceeded",
error: new Error("Workflow budget exceeded"),
};
}

Each agent needs its own limit.

The workflow needs a shared limit.

Both matter.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript / Node.js runtime safety layer I’m building for this problem.

It focuses on pre-call protection for AI-agent projects:

retry storms
prompt loops
max-step explosions
runaway execution
unknown model pricing
budget overruns
uncontrolled provider calls

It is not a billing ledger.

It is not a hard security boundary.

It does not replace provider dashboards.

The goal is to stop obviously risky calls before they execute.

Takeaway

Reusable agent skills are a good abstraction.

But they should not only package instructions.

They should also inherit runtime policy.

Before every provider call, ask:

Should this call still happen?

That one question catches many expensive agent failures before they become API usage.

Tags: ai, agents, typescript, devtools
https://github.com/salimassili62-afk/ai-costguard

If AI Agents Run in Parallel, Budget Checks Need to Happen Before Every Provider Call

Assili Salim — Sat, 27 Jun 2026 10:39:23 +0000

OpenAI’s latest Codex usage data shows a clear shift from short assistant interactions to longer, delegated agent work.

By May 2026, 80.6% of sampled individual Codex users had made at least one request estimated to exceed 30 minutes of human work. 70.2% had made one estimated to exceed one hour.

The more interesting detail:

By June 2026, the 99th percentile of daily active OpenAI users regularly generated more than 60 hours of Codex agent turns per day, distributed across multiple parallel agents.

That is the engineering lesson.

A parallel agent workflow is not a prompt.

It is a runtime system.

Runtime systems need budgets.

The naive version

A simple agent loop often looks like this:

while (!task.done) {
const result = await provider.call({
model: task.model,
messages: task.messages,
});

task = await applyAgentStep(task, result);
}

This is easy to write.

It is also missing the controls that matter in production.

No max-step limit.

No budget check.

No retry-storm detection.

No prompt-loop detection.

No unknown-pricing block.

No no-progress stop.

Now imagine running many of these in parallel.

await Promise.all(tasks.map(runAgent));

This is where the failure mode changes.

One agent overspending is visible.

Ten agents each overspending slightly can look like normal usage until the bill or queue pressure shows up.

The better shape

Before every provider call, the runtime should make a decision.

const decision = guard.beforeCall({
runId: task.id,
workflowId: task.workflowId,
model: task.model,
messages: task.messages,
stepCount: task.steps.length,
retryCount: task.retryCount,
budgetRemaining: task.budgetRemaining,
sharedBudgetRemaining: workflow.budgetRemaining,
previousPrompts: task.previousPrompts,
progressState: task.progress,
});

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
error: decision.error,
};
}

const result = await provider.call({
model: task.model,
messages: task.messages,
});

The API shape does not matter.

The placement matters.

The guard runs before the provider call.

That gives the runtime a chance to stop the next unit of spend before it exists.

What should be checked?
Known model pricing

If the runtime does not know the model price, it cannot enforce a reliable budget.

if (!pricingCatalog.has(model)) {
return {
allowed: false,
reason: "unknown_model_pricing",
};
}

Do not guess.

Fail closed.

Per-run budget

Each agent run needs its own budget.

if (estimatedNextCallCost > runBudgetRemaining) {
return {
allowed: false,
reason: "run_budget_exceeded",
};
}

This stops one confused task from consuming more than it should.

Shared workflow budget

Parallel agents also need a shared ceiling.

if (estimatedNextCallCost > workflowBudgetRemaining) {
return {
allowed: false,
reason: "workflow_budget_exceeded",
};
}

This matters because parallel waste is harder to notice.

Each worker may look reasonable locally while the workflow burns too much globally.

Max-step limit

Agents should not run forever.

if (stepCount >= maxSteps) {
return {
allowed: false,
reason: "max_steps_exceeded",
};
}

Simple controls are often the most valuable.

Retry-storm detection

Retries are useful until they become the workload.

if (retryCount > maxRetries && recentErrorsAreSimilar(errors)) {
return {
allowed: false,
reason: "retry_storm_detected",
};
}

The goal is not to ban retries.

The goal is to prevent blind retries.

Prompt-loop detection

If the current prompt is too similar to earlier failed prompts, the agent may be stuck.

if (similarToRecentPrompt(currentPrompt, previousPrompts)) {
return {
allowed: false,
reason: "similar_prompt_loop",
};
}

This catches a common agent failure:

the system looks active, but it is asking the same question again.

No-progress detection

A run can consume steps without improving the outcome.

Track progress signals:

tests passing
errors decreasing
files changing meaningfully
plan items completing
user-defined success criteria improving

If nothing improves after several steps, stop.

Why this matters

OpenAI’s post says agents change the unit of knowledge work from single interactions to delegated, long-horizon tasks. Agents can operate independently for minutes or hours while using tools, interacting with environments, and iterating toward solutions.

That is exactly why runtime control matters.

A chatbot can fail and wait for the next user message.

An agent can fail and continue.

That continuation is the risk.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript / Node.js runtime safety layer I’m building for this problem.

It is designed to stop agent failure modes before provider calls execute:

retry storms
prompt loops
max-step explosions
no-progress runs
budget overruns
unknown model pricing
runaway agent behavior

The core question is:

Should this next provider call be allowed?

If no, the runtime should stop with a structured reason.

Not after the invoice.

Before the call.

When agents run for minutes or hours, cost control becomes runtime control.

When agents run in parallel, cost control becomes coordination.

Start with one practical rule:

Never call the provider before asking whether the next call is still allowed.

Add a pre-call decision object to your agent loop before adding another dashboard.
https://github.com/salimassili62-afk/ai-costguard

AI Coding Agents Need Runtime Telemetry Before Commit Telemetry

Assili Salim — Fri, 26 Jun 2026 13:52:33 +0000

A new arXiv paper published on June 23, 2026 scanned more than 180 million Git repositories to detect traces of AI coding agents in open source. The authors used multiple signals, including configuration-file scanning, commit-message analysis, author-identity matching, and bot-signature lookup.

The most useful result for developers is the visibility gap.

In one snapshot, multi-method detection found 850,157 Claude Code commits.

Bot-account lookup found only 28,154.

That is 3.3%, or a 30x relative recall gap.

The paper also reports more than 320,000 commit-attributed agent commits per month across snapshots from December 2024 to April 2026.

The immediate takeaway:

AI coding agents are being used heavily.

The engineering takeaway:

Single-signal observability is weak.

Commit telemetry is too late

A commit is the end of an agent run.

It does not tell you enough about the run itself.

A commit may not show:

how many model calls happened
how many retries happened
whether prompts repeated
whether tools failed
whether the model price was known
whether the run exceeded budget
whether the agent made progress
whether fallback models were used
whether the agent stopped safely

If you only inspect the repository after the fact, you are observing the artifact.

You are not observing the execution.

For agent systems, execution is where many failures happen.

Agents are loops

A coding agent is usually some version of this:

while (!task.done) {
const response = await model.call(task.context);

const action = parseAction(response);

const result = await runTool(action);

task = updateTask(task, result);
}

This is useful.

It is also incomplete.

There is no budget.

No max-step limit.

No retry control.

No prompt-loop detection.

No known-pricing check.

No no-progress stop.

A safer runtime shape puts a decision before the provider call.

const decision = guard.beforeCall({
runId: task.id,
model: task.model,
prompt: task.currentPrompt,
stepCount: task.steps.length,
retryCount: task.retryCount,
previousPrompts: task.previousPrompts,
budgetRemaining: task.budgetRemaining,
progressState: task.progress,
});

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
error: decision.error,
};
}

const response = await model.call(task.context);

The important part is not the exact API.

The important part is timing.

The check happens before the provider call.

That means the runtime can stop unsafe execution before more cost is created.

What to log before the call

A useful agent runtime should log decision inputs, not only final outputs.

For each provider call, consider recording:

type AgentCallDecision = {
runId: string;
model: string;
modelPriceKnown: boolean;
stepCount: number;
maxSteps: number;
retryCount: number;
budgetRemaining: number;
estimatedNextCallCost: number;
promptSimilarityScore?: number;
progressScore?: number;
allowed: boolean;
stopReason?: string;
};

This gives you data that a commit cannot provide.

You can now ask:

Which tasks hit max steps?

Which runs stopped because pricing was unknown?

Which prompts repeated?

Which models caused budget pressure?

Which agent workflows produced commits only after many failed attempts?

Which agents consumed budget without progress?

That is runtime telemetry.

Guardrails to implement first

Max-step limits

Agents should not run forever.

if (stepCount >= maxSteps) {
return {
allowed: false,
reason: "max_steps_exceeded",
};
}

This is basic.

It is also one of the highest-value controls.

Unknown pricing blocks

If the runtime cannot price the model, it cannot enforce a budget.

if (!pricingCatalog[model]) {
return {
allowed: false,
reason: "unknown_model_pricing",
};
}

Do not guess.

Fail closed.

Budget guards

Budgets should exist at the task level, not only at the account level.

if (estimatedNextCallCost > budgetRemaining) {
return {
allowed: false,
reason: "budget_exceeded",
};
}

A small refactor and a multi-hour migration should not share the same ceiling.

Retry-storm detection

Retries are normal.

Retry storms are not.

if (retryCount > maxRetries && recentErrorsAreSimilar(errors)) {
return {
allowed: false,
reason: "retry_storm_detected",
};
}

The goal is not to ban retries.

The goal is to stop blind repetition.

Prompt-loop detection

If the current prompt is almost the same as previous failed prompts, the agent may be stuck.

if (similarToRecentPrompt(currentPrompt, previousPrompts)) {
return {
allowed: false,
reason: "similar_prompt_loop",
};
}

Even a simple similarity check can catch obvious waste.

No-progress detection

A run can be active and still not moving.

Track progress signals:

tests passing
errors decreasing
files changing meaningfully
checklist items completing
user-defined success criteria improving

If those signals do not change after several steps, stop.

Why this matters now

GitHub has already said Copilot moved to usage-based billing on June 1, 2026, with usage calculated from token consumption including input, output, and cached tokens. GitHub also described Copilot as moving from an in-editor assistant into an agentic platform capable of long, multi-step coding sessions across repositories.

That means agent runtime behavior increasingly has direct cost impact.

A loop is no longer just a UX problem.

It is a billing problem.

A retry storm is not just noisy.

It is spend.

A prompt loop is not just inefficient.

It is measurable waste.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript / Node.js runtime safety layer I’m building for this problem.

It focuses on stopping agent failures before provider calls execute:

retry storms
prompt loops
max-step explosions
no-progress runs
budget overruns
unknown model pricing
runaway agent behavior

The key design question is simple:

Should this next provider call be allowed?

If the answer is no, the runtime should return a structured stop reason before the call happens.

Takeaway

The new arXiv paper shows that even detecting AI coding-agent activity in repositories requires multiple signals.

That lesson applies directly to runtime engineering.

Do not wait for the commit.

Do not wait for the dashboard.

Do not wait for the invoice.

Instrument the loop.
Add one pre-call decision log to your agent runtime before adding another dashboard.
https://github.com/salimassili62-afk/ai-costguard

Usage-Based AI Coding Needs Runtime Budgets, Not Just Billing Dashboards

Assili Salim — Thu, 25 Jun 2026 08:29:05 +0000

The signal

GitHub reportedly had its “best month ever” in June because demand for AI coding kept growing after Copilot moved to usage-based billing. Business Insider also reported that increased usage has contributed to major outages in 2026 and capacity pressure.

GitHub’s own billing announcement explains the underlying shift: Copilot moved to AI Credits on June 1, and usage is calculated from token consumption, including input, output, and cached tokens. GitHub also described Copilot as evolving into an agentic platform capable of long, multi-step coding sessions across repositories.

That matters for engineering.

A long, multi-step coding session is not a chat message.

It is a loop.

And loops need runtime budgets.

Why this is not just a pricing issue

Usage-based billing makes one thing very clear:

runtime behavior has financial consequences.

A coding agent can spend because it is useful.

It can also spend because it is stuck.

The failure mode usually does not look dramatic.

It looks like this:

inspect files
call a model
edit code
run tests
fail
add context
call the model again
retry with a similar prompt
switch strategy
call another model
run tests again
keep going

Every step can look reasonable.

The whole run can still be waste.

That is why a billing dashboard is not enough.

A dashboard tells you what happened after usage exists.

A runtime budget decides whether the next call should happen.

A naive agent loop

A simple coding agent loop might look like this:

while (!task.done) {
const response = await provider.call({
model: task.model,
messages: task.messages,
});

task = await applyAgentStep(task, response);
}

This is easy to understand.

It is also dangerous.

There is no budget.

No max-step limit.

No retry-storm detection.

No prompt-loop detection.

No known-pricing check.

No no-progress detection.

If this loop gets stuck, it keeps spending until something else stops it.

That “something else” might be a provider limit, a user interruption, an admin cap, or the bill.

None of those are ideal runtime controls.

Add a pre-call decision

A safer pattern puts a guard before the provider call:

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
error: decision.error,
};
}

const response = await provider.call({
model: task.model,
messages: task.messages,
});

The exact API does not matter.

The placement matters.

The check happens before the provider call.

That means the runtime can stop unsafe execution before token usage is created.

What should the runtime check?

Known model pricing

If the runtime does not know the model price, it cannot enforce a reliable budget.

Do not guess.

Fail closed.

if (!pricingCatalog.has(model)) {
throw new Error(Unknown pricing for model: ${model});
}

In usage-based billing, model identity is part of the cost contract.

A typo, alias, fallback, or wrapper mismatch can break assumptions.

Task-level budget

Monthly limits are useful.

But agent runs also need task-level budgets.

A code review task should not have the same spend ceiling as a multi-hour migration.

if (estimatedNextCallCost > budgetRemaining) {
return {
allowed: false,
reason: "budget_exceeded",
};
}

This lets the agent stop before the next call.

Not after.

Max-step protection

Agent loops need step limits.

if (stepCount >= maxSteps) {
return {
allowed: false,
reason: "max_steps_exceeded",
};
}

This is basic, but important.

An agent that cannot finish within a reasonable number of steps may be stuck.

Retry-storm detection

Retries are useful.

Retry storms are not.

if (retryCount >= maxRetries && lastErrorsAreSimilar(task.errors)) {
return {
allowed: false,
reason: "retry_storm_detected",
};
}

The goal is not to remove retries.

The goal is to prevent blind retries.

Prompt-loop detection

Agents sometimes send nearly the same prompt repeatedly.

Small wording changes can hide the fact that the run is not moving.

if (isSimilarToRecentPrompt(currentPrompt, previousPrompts)) {
return {
allowed: false,
reason: "similar_prompt_loop",
};
}

Prompt-loop detection is not perfect.

But even a simple version can catch obvious waste.

No-progress detection

A run can be active and still useless.

The agent may be editing files, calling tools, and producing logs.

But the task may not be converging.

A runtime should track progress signals:

tests passing
errors decreasing
files changing meaningfully
plan steps completing
final answer getting closer
user-defined success criteria

If the run consumes steps without progress, stop.

Why admin caps are not enough

GitHub’s announcement says admins can set budgets at enterprise, cost center, and user levels, and decide whether to allow additional usage once included credits are exhausted.

That is useful.

But admin caps operate at a broad level.

They do not know why a single agent run is stuck.

They do not know whether a prompt is repeating.

They do not know whether this specific task is worth another call.

That decision belongs closer to the runtime.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript runtime layer I’m building for this problem.

It is designed to stop expensive agent failure modes before provider calls execute:

retry storms
prompt loops
max-step explosions
no-progress runs
budget overruns
unknown model pricing
runaway agent behavior

It is not a billing dashboard.

It is not a cloud control plane.

It is not a hard security boundary.

It is a pre-call kill switch for agent cost and loop failures.

The takeaway

Usage-based AI coding changes the engineering model.

You cannot only ask:

“How much does this tool cost per month?”

You have to ask:

“What does my agent do when it gets stuck?”

For agentic coding, cost control belongs in the runtime.

Before the provider call.

Before the bill.

Before the loop becomes waste.
https://github.com/salimassili62-afk/ai-costguard