DEV Community: Assili Salim

Always-Running AI Agents Need Pre-Call Stop Conditions

Assili Salim — Wed, 08 Jul 2026 21:02:58 +0000

Claude Cowork can now keep working after you close your laptop.

WIRED reports that Anthropic expanded Cowork beyond the desktop app, so users do not need an active desktop session to keep tasks running. Users can interact with limited versions through the Claude smartphone app or web browser, and Cowork can continue tasks after the user clocks out.

That is a useful feature.

It also creates a runtime engineering problem.

If agents can run unattended, they need stop conditions before provider calls execute.

A chatbot waits. An agent continues.

A chatbot is mostly reactive.

The user sends a message.

The model responds.

The interaction pauses.

An agent is different.

It may:

call a model
call tools
inspect results
add context
retry
call another tool
generate a document
send another provider request
keep going while the user is away

That means the agent runtime needs controls.

Not only logs.

Not only dashboards.

Controls before execution.

The naive loop

A simple agent loop might look like this:

while (!task.done) {
const response = await provider.call({
model: task.model,
messages: task.messages,
});

task = await applyAgentStep(task, response);
}

This is easy to write.

It is also unsafe for unattended workflows.

There is no max-step limit.

No budget check.

No retry-storm detection.

No prompt-loop detection.

No known-pricing check.

No no-progress stop.

If the agent gets stuck, it keeps creating provider calls until something external stops it.

That “something” might be a provider error, user intervention, an account limit, or a bill.

None of those are ideal runtime controls.

Add a pre-call decision

A safer loop puts a guard before every provider call.

const decision = guard.beforeCall({
runId: task.id,
model: task.model,
messages: task.messages,
stepCount: task.steps.length,
retryCount: task.retryCount,
budgetRemaining: task.budgetRemaining,
previousPrompts: task.previousPrompts,
progressState: task.progress,
});

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
error: decision.error,
};
}

const response = await provider.call({
model: task.model,
messages: task.messages,
});

The exact API does not matter.

The placement matters.

The runtime checks the call before the provider sees it.

What should the runtime check?

Known model pricing

If the runtime cannot price the model, it cannot enforce a reliable budget.

if (!pricingCatalog.has(model)) {
return {
allowed: false,
reason: "unknown_model_pricing",
};
}

Do not guess inside an unattended loop.

A typo, fallback, gateway rewrite, or model alias can break cost assumptions.

Task budget

Every unattended run should have a task-level budget.

if (estimatedNextCallCost > budgetRemaining) {
return {
allowed: false,
reason: "budget_exceeded",
};
}

Monthly dashboards are useful, but they are late.

A task budget stops the next call before spend is created.

Max steps

Agents need explicit step limits.

if (stepCount >= maxSteps) {
return {
allowed: false,
reason: "max_steps_exceeded",
};
}

A run that cannot finish inside a reasonable number of steps should stop cleanly.

Retry storms

Retries are normal.

Retry storms are not.

if (retryCount >= maxRetries && recentErrorsAreSimilar(errors)) {
return {
allowed: false,
reason: "retry_storm_detected",
};
}

The goal is not to remove retries.

The goal is to prevent repeated failure from becoming the workload.

Prompt loops

Agents often get stuck by asking almost the same thing again.

if (similarToRecentPrompt(currentPrompt, previousPrompts)) {
return {
allowed: false,
reason: "similar_prompt_loop",
};
}

This catches a common pattern:

the agent looks active, but it is not exploring a new path.

No progress

A run can consume steps while producing no useful movement.

Track progress signals:

errors decreasing
files changing meaningfully
tests improving
checklist items completing
retrieved information changing
user-defined success criteria improving

If progress does not change after several steps, stop.

if (!madeProgress(recentSteps)) {
return {
allowed: false,
reason: "no_progress",
};
}
Why this matters more for always-running agents

When the user is watching, some failures are caught manually.

When the agent runs overnight, the runtime becomes the first line of control.

That changes the design requirement.

An unattended agent needs:

local budgets
max-step policies
retry budgets
prompt-loop detection
known model pricing
structured stop reasons

These are not advanced features.

They are basic operating rules.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript/Node.js runtime safety layer I’m building for AI-agent projects.

It focuses on pre-call protection for:

retry storms
prompt loops
max-step explosions
runaway agent execution
unknown model pricing
budget overruns
uncontrolled provider calls
invisible AI-agent cost risk

It is not a billing ledger.

It is not a hard security boundary.

It does not replace provider dashboards.

The goal is to help the runtime decide whether the next provider call should execute.

Takeaway

Always-running agents are useful because they remove friction.

That same lack of friction is the risk.

If an agent can keep working after the laptop closes, it needs runtime rules for when to stop.
https://github.com/salimassili62-afk/ai-costguard

Tokenizer Changes Can Break AI-Agent Budget Assumptions

Assili Salim — Sat, 04 Jul 2026 08:41:45 +0000

Claude Sonnet 5 is a good reminder that AI-agent cost control is not only about model pricing.

Anthropic says Sonnet 5 launches at $2 per million input tokens and $10 per million output tokens through August 31, 2026, then moves to $3 per million input tokens and $15 per million output tokens.

Vercel’s AI Gateway changelog adds an important implementation detail: Sonnet 5 uses an updated tokenizer, and the same input can map to more tokens.

That matters if you build agents.

A model can have attractive pricing and still change your runtime cost assumptions.

The problem

Many agent systems estimate cost like this:

const estimatedCost =
inputTokens * inputPrice +
maxOutputTokens * outputPrice;

That is fine as a start.

But it assumes the runtime knows:

which model is being used
how the input is tokenized
the correct input price
the correct output price
how many retries are allowed
how many steps are allowed
whether the agent is making progress

If any of those assumptions drift, the estimate becomes weaker.

A tokenizer update is one way this happens.

The prompt text may be the same.

The token count may not be.

Agents amplify small estimation errors

For a single request, a token-count surprise is usually manageable.

For an agent, it can compound.

An agent may:

call the model
inspect files
add context
run tools
retry
ask a similar prompt
switch models
continue for more steps

Small cost drift per call becomes bigger across a run.

Now add parallel agents, fallback paths, or long-context workflows.

The problem is no longer “What does this model cost?”

The problem is:

Should this next provider call be allowed?

Add a pre-call decision

Before the provider call, add a guard decision.

type BeforeCallInput = {
runId: string;
model: string;
prompt: string;
estimatedInputTokens: number;
maxOutputTokens: number;
stepCount: number;
retryCount: number;
budgetRemaining: number;
previousPrompts: string[];
};

type GuardDecision =
| { allowed: true }
| {
allowed: false;
reason:
| "unknown_model_pricing"
| "budget_exceeded"
| "max_steps_exceeded"
| "retry_storm"
| "prompt_loop";
};

Then use it before execution:

const decision = guard.beforeCall({
runId,
model,
prompt,
estimatedInputTokens,
maxOutputTokens,
stepCount,
retryCount,
budgetRemaining,
previousPrompts,
});

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
};
}

const result = await provider.call({
model,
prompt,
});

The important part is placement.

The check happens before the provider call.

Not after the bill.

Fail closed on unknown pricing

If the runtime does not know the model price, it cannot enforce a reliable budget.

if (!pricingCatalog.has(model)) {
return {
allowed: false,
reason: "unknown_model_pricing",
};
}

Do not guess.

A model alias, fallback, gateway rewrite, or typo can break your assumptions.

Unknown pricing should stop the run before the call executes.

Recalculate tokens after model migration

When changing models, do not assume the same prompt has the same token count.

A basic migration checklist:

run representative prompts through the new tokenizer
compare input token counts
compare output length behavior
update pricing metadata
retest max-step limits
retest retry behavior
check fallback paths
measure cost per successful task

Cost per token is useful.

Cost per successful task is more useful.

Watch retry and prompt-loop behavior

A smarter model may finish tasks in fewer steps.

It may also continue longer because it can pursue more complex plans.

Your runtime should still stop obvious waste.

if (stepCount >= maxSteps) {
return { allowed: false, reason: "max_steps_exceeded" };
}

if (retryCount >= maxRetries) {
return { allowed: false, reason: "retry_storm" };
}

if (similarToRecentPrompt(prompt, previousPrompts)) {
return { allowed: false, reason: "prompt_loop" };
}

These checks do not make the model better.

They make the system safer to operate.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript/Node.js runtime safety layer I’m building for AI-agent projects.

It focuses on pre-call safety checks for:

retry storms
prompt loops
max-step explosions
runaway execution
unknown model pricing
budget overruns
uncontrolled provider calls

It is not a billing ledger.

It is not a hard security boundary.

It does not replace provider dashboards.

The goal is to catch obviously risky calls before they execute.

Takeaway

Model pricing pages tell you the unit price.

They do not tell you whether your agent should make the next call.

Tokenizer changes, fallback models, retries, and prompt loops all affect real runtime cost.

For AI agents, cost safety belongs before the provider call.

Gateway Routing Helps AI Apps. Agent Runtimes Still Need Pre-Call Guards.

Assili Salim — Fri, 03 Jul 2026 07:22:32 +0000

Vercel added routing rules to AI Gateway on July 2.

Routing rules let teams rewrite or deny model requests at the gateway level. A rewrite rule serves a request for one model using another model. A deny rule blocks a model and returns a 403. Vercel lists use cases like rerouting when a model is down or retired, standardizing on one model, routing an expensive model to a cheaper one, or keeping a team off unapproved models.

That is useful infrastructure.

But if you are building AI agents, it is not the whole safety layer.

A gateway can decide where a request goes.

Your runtime still needs to decide whether the request should happen.

The difference

Gateway routing answers:

"Which model should serve this request?"

Runtime guarding answers:

"Should this request be allowed at all?"

Those are different problems.

A gateway may rewrite:

anthropic/claude-opus-4.8 -> anthropic/claude-haiku-4.5

That can keep traffic moving.

But the gateway may not know:

the agent already retried 12 times
the current prompt is nearly identical to previous failed prompts
the run exceeded its task budget
the agent passed its max-step limit
tool calls are happening without progress
the fallback model keeps the loop alive but does not improve the task

That context usually lives inside the agent runtime.

A naive agent loop

Many agent loops start like this:

while (!task.done) {
const response = await provider.call({
model: task.model,
messages: task.messages,
});

task = await applyAgentStep(task, response);
}

This is simple.

It is also missing the controls that matter in production.

There is no budget check.

No max-step check.

No retry-storm detection.

No prompt-loop detection.

No unknown-pricing block.

No no-progress stop.

If a gateway rewrites the model, this loop may keep running.

That is not always what you want.

Add a pre-call guard

A safer pattern puts a local decision before the provider call:

const decision = guard.beforeCall({
runId: task.id,
model: task.model,
messages: task.messages,
stepCount: task.steps.length,
retryCount: task.retryCount,
previousPrompts: task.previousPrompts,
budgetRemaining: task.budgetRemaining,
progressState: task.progress,
});

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
error: decision.error,
};
}

const response = await provider.call({
model: task.model,
messages: task.messages,
});

The exact API does not matter.

The placement matters.

The guard runs before the provider call.

What should the runtime check?

Known model pricing

If the runtime cannot price the model, it cannot enforce a reliable budget.

if (!pricingCatalog.has(model)) {
return {
allowed: false,
reason: "unknown_model_pricing",
};
}

This matters even more when routing and fallback rules exist.

A rewritten model still has a cost profile.

The runtime should know it.

Budget remaining

Task-level budgets are different from account-level limits.

if (estimatedNextCallCost > budgetRemaining) {
return {
allowed: false,
reason: "budget_exceeded",
};
}

A monthly dashboard can show spend later.

A runtime budget can stop the next call now.

Max steps

Agents should have explicit stopping rules.

if (stepCount >= maxSteps) {
return {
allowed: false,
reason: "max_steps_exceeded",
};
}

A model route can change.

The step limit should still apply.

Retry storms

Retries are useful.

Blind retries are not.

if (retryCount > maxRetries && recentErrorsAreSimilar(errors)) {
return {
allowed: false,
reason: "retry_storm_detected",
};
}

A fallback model can hide a retry storm by keeping the run alive.

The runtime should detect the pattern.

Prompt loops

Agents sometimes ask almost the same thing repeatedly.

if (similarToRecentPrompt(currentPrompt, previousPrompts)) {
return {
allowed: false,
reason: "similar_prompt_loop",
};
}

If the prompt is not changing meaningfully, the model route may not be the main issue.

The agent may be stuck.

No progress

A run can be active without improving.

Useful progress signals include:

tests passing
errors decreasing
files changing meaningfully
task checklist items completing
user-defined success criteria improving

If the agent consumes steps without progress, stop.

Layer the controls

A good AI-agent architecture can use both gateway policy and runtime guards.

One possible flow:

agent wants next call
↓
local runtime guard checks run state
↓
gateway applies model routing or deny rules
↓
provider executes request
↓
logs and dashboards record result

The order matters.

The runtime has run context.

The gateway has team-level model policy.

The provider executes.

The dashboard explains.

Do not ask one layer to do all four jobs.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript / Node.js runtime safety layer I’m building for this exact class of problem.

It focuses on pre-call checks for AI-agent projects:

retry storms
prompt loops
max-step explosions
runaway agent execution
unknown model pricing
budget overruns
uncontrolled provider calls

It is not a billing ledger.

It is not a hard security boundary.

It does not replace provider dashboards or gateway routing.

The goal is narrower:

help the agent runtime decide whether the next provider call should execute.

Takeaway

Gateway routing is useful.

It centralizes model policy.

It helps teams move traffic when models are down, retired, too expensive, or not approved.

But routing does not replace runtime safety.

A cheaper fallback can still waste money.

A policy-approved model can still be part of a prompt loop.

A valid request can still exceed the task budget.

For AI agents, the critical question happens before the call:

Should this request exist?
https://github.com/salimassili62-afk/ai-costguard

Reusable Agent Skills Need Pre-Call Runtime Checks

Assili Salim — Sun, 28 Jun 2026 14:10:34 +0000

OpenAI’s recent Codex research includes one detail that matters for developers building agents:

26.6% of users use skills to share instructions for complex workflows, and more than 10% manage three or more concurrent Codex agents at some point each week.

That means agent usage is moving from one-off prompts toward reusable workflows.

That is good.

It also means failures can become reusable.

The problem

A bad prompt can waste one model call.

A bad agent skill can waste many runs.

A skill might encode:

how to retry
how to call tools
how to inspect files
how to recover from errors
how much context to add
when to continue
when to stop

If those rules are loose, every run inherits the looseness.

This is the part developers need to treat carefully.

Reusable agent behavior needs reusable runtime boundaries.

A naive agent skill

Imagine a coding-agent skill for fixing failing tests.

The instruction might be:

const skill = {
name: "fix-failing-tests",
instructions: Inspect the failing test. Find the relevant files. Apply a fix. Run the tests again. Repeat until the tests pass.,
};

This sounds fine.

But “repeat until the tests pass” is dangerous without runtime limits.

What if the test failure is environmental?

What if the agent keeps editing unrelated files?

What if the prompt barely changes across attempts?

What if each retry adds more context?

What if the fallback model has unknown pricing?

The skill is useful.

The runtime is under-specified.

Add a pre-call decision

Before every provider call, the runtime should decide whether the call is still allowed.

type BeforeCallInput = {
runId: string;
workflowId?: string;
model: string;
prompt: string;
stepCount: number;
retryCount: number;
budgetRemaining: number;
previousPrompts: string[];
progressState: {
testsImproved?: boolean;
errorsChanged?: boolean;
filesChanged?: boolean;
};
};

type GuardDecision =
| { allowed: true }
| {
allowed: false;
reason:
| "unknown_model_pricing"
| "budget_exceeded"
| "max_steps_exceeded"
| "retry_storm_detected"
| "similar_prompt_loop"
| "no_progress";
error: Error;
};

Then use it before the provider call:

const decision = guard.beforeCall({
runId,
workflowId,
model,
prompt,
stepCount,
retryCount,
budgetRemaining,
previousPrompts,
progressState,
});

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
error: decision.error,
};
}

const response = await provider.call({
model,
prompt,
});

The exact API does not matter.

The placement matters.

The check happens before the provider call.

What should the guard check?

Known model pricing

If the runtime cannot price the model, it cannot enforce a budget.

if (!pricingCatalog.has(model)) {
return {
allowed: false,
reason: "unknown_model_pricing",
error: new Error(Unknown pricing for model: ${model}),
};
}

Do not guess.

Fail closed.

Budget remaining

Agent workflows should have task-level budgets.

if (estimatedNextCallCost > budgetRemaining) {
return {
allowed: false,
reason: "budget_exceeded",
error: new Error("Agent run budget exceeded"),
};
}

A small bug fix and a long migration should not share the same budget.

Max steps

Agents need step limits.

if (stepCount >= maxSteps) {
return {
allowed: false,
reason: "max_steps_exceeded",
error: new Error("Maximum agent steps exceeded"),
};
}

This is basic production hygiene.

If a workflow cannot complete inside a reasonable number of steps, it should stop.

Retry storms

Retries are useful.

Blind retries are expensive.

if (retryCount >= maxRetries && recentErrorsAreSimilar(errors)) {
return {
allowed: false,
reason: "retry_storm_detected",
error: new Error("Retry storm detected"),
};
}

The goal is not to ban retries.

The goal is to stop repeated failure.

Prompt loops

A prompt loop happens when the agent keeps asking nearly the same thing.

if (similarToRecentPrompt(prompt, previousPrompts)) {
return {
allowed: false,
reason: "similar_prompt_loop",
error: new Error("Similar prompt loop detected"),
};
}

Even a simple similarity check can catch obvious loops.

No progress

A run can be active and still not improve.

Track progress signals:

did tests improve?
did the error change?
did files change meaningfully?
did a checklist item complete?
did the agent reduce uncertainty?

If several steps pass without progress, stop.

if (!madeProgress(progressState, recentSteps)) {
return {
allowed: false,
reason: "no_progress",
error: new Error("Agent run is not making progress"),
};
}
Why concurrency makes this more important

OpenAI’s Codex research says more than 10% of users manage three or more concurrent agents at some point each week.

That changes the risk.

One agent wasting a few calls is visible.

Several agents each wasting a few calls can look normal.

The local loop becomes a global budget problem.

For parallel workflows, add shared budget checks:

if (estimatedNextCallCost > workflowBudgetRemaining) {
return {
allowed: false,
reason: "workflow_budget_exceeded",
error: new Error("Workflow budget exceeded"),
};
}

Each agent needs its own limit.

The workflow needs a shared limit.

Both matter.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript / Node.js runtime safety layer I’m building for this problem.

It focuses on pre-call protection for AI-agent projects:

retry storms
prompt loops
max-step explosions
runaway execution
unknown model pricing
budget overruns
uncontrolled provider calls

It is not a billing ledger.

It is not a hard security boundary.

It does not replace provider dashboards.

The goal is to stop obviously risky calls before they execute.

Takeaway

Reusable agent skills are a good abstraction.

But they should not only package instructions.

They should also inherit runtime policy.

Before every provider call, ask:

Should this call still happen?

That one question catches many expensive agent failures before they become API usage.

Tags: ai, agents, typescript, devtools
https://github.com/salimassili62-afk/ai-costguard

If AI Agents Run in Parallel, Budget Checks Need to Happen Before Every Provider Call

Assili Salim — Sat, 27 Jun 2026 10:39:23 +0000

OpenAI’s latest Codex usage data shows a clear shift from short assistant interactions to longer, delegated agent work.

By May 2026, 80.6% of sampled individual Codex users had made at least one request estimated to exceed 30 minutes of human work. 70.2% had made one estimated to exceed one hour.

The more interesting detail:

By June 2026, the 99th percentile of daily active OpenAI users regularly generated more than 60 hours of Codex agent turns per day, distributed across multiple parallel agents.

That is the engineering lesson.

A parallel agent workflow is not a prompt.

It is a runtime system.

Runtime systems need budgets.

The naive version

A simple agent loop often looks like this:

while (!task.done) {
const result = await provider.call({
model: task.model,
messages: task.messages,
});

task = await applyAgentStep(task, result);
}

This is easy to write.

It is also missing the controls that matter in production.

No max-step limit.

No budget check.

No retry-storm detection.

No prompt-loop detection.

No unknown-pricing block.

No no-progress stop.

Now imagine running many of these in parallel.

await Promise.all(tasks.map(runAgent));

This is where the failure mode changes.

One agent overspending is visible.

Ten agents each overspending slightly can look like normal usage until the bill or queue pressure shows up.

The better shape

Before every provider call, the runtime should make a decision.

const decision = guard.beforeCall({
runId: task.id,
workflowId: task.workflowId,
model: task.model,
messages: task.messages,
stepCount: task.steps.length,
retryCount: task.retryCount,
budgetRemaining: task.budgetRemaining,
sharedBudgetRemaining: workflow.budgetRemaining,
previousPrompts: task.previousPrompts,
progressState: task.progress,
});

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
error: decision.error,
};
}

const result = await provider.call({
model: task.model,
messages: task.messages,
});

The API shape does not matter.

The placement matters.

The guard runs before the provider call.

That gives the runtime a chance to stop the next unit of spend before it exists.

What should be checked?
Known model pricing

If the runtime does not know the model price, it cannot enforce a reliable budget.

if (!pricingCatalog.has(model)) {
return {
allowed: false,
reason: "unknown_model_pricing",
};
}

Do not guess.

Fail closed.

Per-run budget

Each agent run needs its own budget.

if (estimatedNextCallCost > runBudgetRemaining) {
return {
allowed: false,
reason: "run_budget_exceeded",
};
}

This stops one confused task from consuming more than it should.

Shared workflow budget

Parallel agents also need a shared ceiling.

if (estimatedNextCallCost > workflowBudgetRemaining) {
return {
allowed: false,
reason: "workflow_budget_exceeded",
};
}

This matters because parallel waste is harder to notice.

Each worker may look reasonable locally while the workflow burns too much globally.

Max-step limit

Agents should not run forever.

if (stepCount >= maxSteps) {
return {
allowed: false,
reason: "max_steps_exceeded",
};
}

Simple controls are often the most valuable.

Retry-storm detection

Retries are useful until they become the workload.

if (retryCount > maxRetries && recentErrorsAreSimilar(errors)) {
return {
allowed: false,
reason: "retry_storm_detected",
};
}

The goal is not to ban retries.

The goal is to prevent blind retries.

Prompt-loop detection

If the current prompt is too similar to earlier failed prompts, the agent may be stuck.

if (similarToRecentPrompt(currentPrompt, previousPrompts)) {
return {
allowed: false,
reason: "similar_prompt_loop",
};
}

This catches a common agent failure:

the system looks active, but it is asking the same question again.

No-progress detection

A run can consume steps without improving the outcome.

Track progress signals:

tests passing
errors decreasing
files changing meaningfully
plan items completing
user-defined success criteria improving

If nothing improves after several steps, stop.

Why this matters

OpenAI’s post says agents change the unit of knowledge work from single interactions to delegated, long-horizon tasks. Agents can operate independently for minutes or hours while using tools, interacting with environments, and iterating toward solutions.

That is exactly why runtime control matters.

A chatbot can fail and wait for the next user message.

An agent can fail and continue.

That continuation is the risk.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript / Node.js runtime safety layer I’m building for this problem.

It is designed to stop agent failure modes before provider calls execute:

retry storms
prompt loops
max-step explosions
no-progress runs
budget overruns
unknown model pricing
runaway agent behavior

The core question is:

Should this next provider call be allowed?

If no, the runtime should stop with a structured reason.

Not after the invoice.

Before the call.

When agents run for minutes or hours, cost control becomes runtime control.

When agents run in parallel, cost control becomes coordination.

Start with one practical rule:

Never call the provider before asking whether the next call is still allowed.

Add a pre-call decision object to your agent loop before adding another dashboard.
https://github.com/salimassili62-afk/ai-costguard

AI Coding Agents Need Runtime Telemetry Before Commit Telemetry

Assili Salim — Fri, 26 Jun 2026 13:52:33 +0000

A new arXiv paper published on June 23, 2026 scanned more than 180 million Git repositories to detect traces of AI coding agents in open source. The authors used multiple signals, including configuration-file scanning, commit-message analysis, author-identity matching, and bot-signature lookup.

The most useful result for developers is the visibility gap.

In one snapshot, multi-method detection found 850,157 Claude Code commits.

Bot-account lookup found only 28,154.

That is 3.3%, or a 30x relative recall gap.

The paper also reports more than 320,000 commit-attributed agent commits per month across snapshots from December 2024 to April 2026.

The immediate takeaway:

AI coding agents are being used heavily.

The engineering takeaway:

Single-signal observability is weak.

Commit telemetry is too late

A commit is the end of an agent run.

It does not tell you enough about the run itself.

A commit may not show:

how many model calls happened
how many retries happened
whether prompts repeated
whether tools failed
whether the model price was known
whether the run exceeded budget
whether the agent made progress
whether fallback models were used
whether the agent stopped safely

If you only inspect the repository after the fact, you are observing the artifact.

You are not observing the execution.

For agent systems, execution is where many failures happen.

Agents are loops

A coding agent is usually some version of this:

while (!task.done) {
const response = await model.call(task.context);

const action = parseAction(response);

const result = await runTool(action);

task = updateTask(task, result);
}

This is useful.

It is also incomplete.

There is no budget.

No max-step limit.

No retry control.

No prompt-loop detection.

No known-pricing check.

No no-progress stop.

A safer runtime shape puts a decision before the provider call.

const decision = guard.beforeCall({
runId: task.id,
model: task.model,
prompt: task.currentPrompt,
stepCount: task.steps.length,
retryCount: task.retryCount,
previousPrompts: task.previousPrompts,
budgetRemaining: task.budgetRemaining,
progressState: task.progress,
});

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
error: decision.error,
};
}

const response = await model.call(task.context);

The important part is not the exact API.

The important part is timing.

The check happens before the provider call.

That means the runtime can stop unsafe execution before more cost is created.

What to log before the call

A useful agent runtime should log decision inputs, not only final outputs.

For each provider call, consider recording:

type AgentCallDecision = {
runId: string;
model: string;
modelPriceKnown: boolean;
stepCount: number;
maxSteps: number;
retryCount: number;
budgetRemaining: number;
estimatedNextCallCost: number;
promptSimilarityScore?: number;
progressScore?: number;
allowed: boolean;
stopReason?: string;
};

This gives you data that a commit cannot provide.

You can now ask:

Which tasks hit max steps?

Which runs stopped because pricing was unknown?

Which prompts repeated?

Which models caused budget pressure?

Which agent workflows produced commits only after many failed attempts?

Which agents consumed budget without progress?

That is runtime telemetry.

Guardrails to implement first

Max-step limits

Agents should not run forever.

if (stepCount >= maxSteps) {
return {
allowed: false,
reason: "max_steps_exceeded",
};
}

This is basic.

It is also one of the highest-value controls.

Unknown pricing blocks

If the runtime cannot price the model, it cannot enforce a budget.

if (!pricingCatalog[model]) {
return {
allowed: false,
reason: "unknown_model_pricing",
};
}

Do not guess.

Fail closed.

Budget guards

Budgets should exist at the task level, not only at the account level.

if (estimatedNextCallCost > budgetRemaining) {
return {
allowed: false,
reason: "budget_exceeded",
};
}

A small refactor and a multi-hour migration should not share the same ceiling.

Retry-storm detection

Retries are normal.

Retry storms are not.

if (retryCount > maxRetries && recentErrorsAreSimilar(errors)) {
return {
allowed: false,
reason: "retry_storm_detected",
};
}

The goal is not to ban retries.

The goal is to stop blind repetition.

Prompt-loop detection

If the current prompt is almost the same as previous failed prompts, the agent may be stuck.

if (similarToRecentPrompt(currentPrompt, previousPrompts)) {
return {
allowed: false,
reason: "similar_prompt_loop",
};
}

Even a simple similarity check can catch obvious waste.

No-progress detection

A run can be active and still not moving.

Track progress signals:

tests passing
errors decreasing
files changing meaningfully
checklist items completing
user-defined success criteria improving

If those signals do not change after several steps, stop.

Why this matters now

GitHub has already said Copilot moved to usage-based billing on June 1, 2026, with usage calculated from token consumption including input, output, and cached tokens. GitHub also described Copilot as moving from an in-editor assistant into an agentic platform capable of long, multi-step coding sessions across repositories.

That means agent runtime behavior increasingly has direct cost impact.

A loop is no longer just a UX problem.

It is a billing problem.

A retry storm is not just noisy.

It is spend.

A prompt loop is not just inefficient.

It is measurable waste.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript / Node.js runtime safety layer I’m building for this problem.

It focuses on stopping agent failures before provider calls execute:

retry storms
prompt loops
max-step explosions
no-progress runs
budget overruns
unknown model pricing
runaway agent behavior

The key design question is simple:

Should this next provider call be allowed?

If the answer is no, the runtime should return a structured stop reason before the call happens.

Takeaway

The new arXiv paper shows that even detecting AI coding-agent activity in repositories requires multiple signals.

That lesson applies directly to runtime engineering.

Do not wait for the commit.

Do not wait for the dashboard.

Do not wait for the invoice.

Instrument the loop.
Add one pre-call decision log to your agent runtime before adding another dashboard.
https://github.com/salimassili62-afk/ai-costguard

Usage-Based AI Coding Needs Runtime Budgets, Not Just Billing Dashboards

Assili Salim — Thu, 25 Jun 2026 08:29:05 +0000

The signal

GitHub reportedly had its “best month ever” in June because demand for AI coding kept growing after Copilot moved to usage-based billing. Business Insider also reported that increased usage has contributed to major outages in 2026 and capacity pressure.

GitHub’s own billing announcement explains the underlying shift: Copilot moved to AI Credits on June 1, and usage is calculated from token consumption, including input, output, and cached tokens. GitHub also described Copilot as evolving into an agentic platform capable of long, multi-step coding sessions across repositories.

That matters for engineering.

A long, multi-step coding session is not a chat message.

It is a loop.

And loops need runtime budgets.

Why this is not just a pricing issue

Usage-based billing makes one thing very clear:

runtime behavior has financial consequences.

A coding agent can spend because it is useful.

It can also spend because it is stuck.

The failure mode usually does not look dramatic.

It looks like this:

inspect files
call a model
edit code
run tests
fail
add context
call the model again
retry with a similar prompt
switch strategy
call another model
run tests again
keep going

Every step can look reasonable.

The whole run can still be waste.

That is why a billing dashboard is not enough.

A dashboard tells you what happened after usage exists.

A runtime budget decides whether the next call should happen.

A naive agent loop

A simple coding agent loop might look like this:

while (!task.done) {
const response = await provider.call({
model: task.model,
messages: task.messages,
});

task = await applyAgentStep(task, response);
}

This is easy to understand.

It is also dangerous.

There is no budget.

No max-step limit.

No retry-storm detection.

No prompt-loop detection.

No known-pricing check.

No no-progress detection.

If this loop gets stuck, it keeps spending until something else stops it.

That “something else” might be a provider limit, a user interruption, an admin cap, or the bill.

None of those are ideal runtime controls.

Add a pre-call decision

A safer pattern puts a guard before the provider call:

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
error: decision.error,
};
}

const response = await provider.call({
model: task.model,
messages: task.messages,
});

The exact API does not matter.

The placement matters.

The check happens before the provider call.

That means the runtime can stop unsafe execution before token usage is created.

What should the runtime check?

Known model pricing

If the runtime does not know the model price, it cannot enforce a reliable budget.

Do not guess.

Fail closed.

if (!pricingCatalog.has(model)) {
throw new Error(Unknown pricing for model: ${model});
}

In usage-based billing, model identity is part of the cost contract.

A typo, alias, fallback, or wrapper mismatch can break assumptions.

Task-level budget

Monthly limits are useful.

But agent runs also need task-level budgets.

A code review task should not have the same spend ceiling as a multi-hour migration.

if (estimatedNextCallCost > budgetRemaining) {
return {
allowed: false,
reason: "budget_exceeded",
};
}

This lets the agent stop before the next call.

Not after.

Max-step protection

Agent loops need step limits.

if (stepCount >= maxSteps) {
return {
allowed: false,
reason: "max_steps_exceeded",
};
}

This is basic, but important.

An agent that cannot finish within a reasonable number of steps may be stuck.

Retry-storm detection

Retries are useful.

Retry storms are not.

if (retryCount >= maxRetries && lastErrorsAreSimilar(task.errors)) {
return {
allowed: false,
reason: "retry_storm_detected",
};
}

The goal is not to remove retries.

The goal is to prevent blind retries.

Prompt-loop detection

Agents sometimes send nearly the same prompt repeatedly.

Small wording changes can hide the fact that the run is not moving.

if (isSimilarToRecentPrompt(currentPrompt, previousPrompts)) {
return {
allowed: false,
reason: "similar_prompt_loop",
};
}

Prompt-loop detection is not perfect.

But even a simple version can catch obvious waste.

No-progress detection

A run can be active and still useless.

The agent may be editing files, calling tools, and producing logs.

But the task may not be converging.

A runtime should track progress signals:

tests passing
errors decreasing
files changing meaningfully
plan steps completing
final answer getting closer
user-defined success criteria

If the run consumes steps without progress, stop.

Why admin caps are not enough

GitHub’s announcement says admins can set budgets at enterprise, cost center, and user levels, and decide whether to allow additional usage once included credits are exhausted.

That is useful.

But admin caps operate at a broad level.

They do not know why a single agent run is stuck.

They do not know whether a prompt is repeating.

They do not know whether this specific task is worth another call.

That decision belongs closer to the runtime.

Where AI CostGuard fits

AI CostGuard is the local-first TypeScript runtime layer I’m building for this problem.

It is designed to stop expensive agent failure modes before provider calls execute:

retry storms
prompt loops
max-step explosions
no-progress runs
budget overruns
unknown model pricing
runaway agent behavior

It is not a billing dashboard.

It is not a cloud control plane.

It is not a hard security boundary.

It is a pre-call kill switch for agent cost and loop failures.

The takeaway

Usage-based AI coding changes the engineering model.

You cannot only ask:

“How much does this tool cost per month?”

You have to ask:

“What does my agent do when it gets stuck?”

For agentic coding, cost control belongs in the runtime.

Before the provider call.

Before the bill.

Before the loop becomes waste.
https://github.com/salimassili62-afk/ai-costguard

Claude Had Elevated Errors. Your AI Agent Should Not Turn That Into a Retry Storm.

Assili Salim — Wed, 24 Jun 2026 09:29:02 +0000

The event

On June 23, Anthropic’s Claude status page listed elevated error rates affecting Claude.ai, resolved at 18:32 UTC. It also showed elevated error rates across multiple models around the same period. On June 24, the status page listed no incidents for the day.

This kind of incident is easy to frame as a provider problem.

And it is.

But for AI-agent developers, it is also a runtime-design problem.

Because provider instability exposes what your agent does when the model layer becomes unreliable.

The hidden problem

A normal chatbot failure is visible.

The user asks a question.

The model fails.

The interface shows an error.

The user stops.

Agents are different.

An agent may be running in a CLI, background job, coding workflow, CI task, support automation, or internal tool.

It may not stop cleanly.

It may retry.

It may call tools again.

It may add context.

It may switch models.

It may repeat the same prompt with minor edits.

It may continue even though the task is not making progress.

That turns provider instability into local runtime waste.

Bad retry logic

A naive agent loop might look like this:

for (let attempt = 0; attempt < 3; attempt++) {
try {
return await provider.call({
model,
messages,
});
} catch (err) {
continue;
}
}

This is simple.

Too simple.

It does not know whether the provider is degraded.

It does not know whether the next prompt is similar to the previous one.

It does not know whether the run has exceeded budget.

It does not know whether the fallback model is more expensive.

It does not know whether the agent is making progress.

For a normal HTTP call, this may be acceptable.

For an AI agent, it is weak.

Agents need retry budgets

A retry count says:

“Try three times.”

A retry budget says:

“Retry only while the run remains safe.”

That means the runtime should check more than the attempt number.

For example:

const decision = guard.beforeCall({
runId,
model,
messages,
stepCount,
retryCount,
providerStatus,
estimatedInputTokens,
estimatedOutputTokens,
budgetRemaining,
previousPrompts,
progressState,
});

if (!decision.allowed) {
throw decision.error;
}

const result = await provider.call({
model,
messages,
});

The exact API is not the point.

The important part is placement.

The guard runs before the provider call.

That means it can stop a bad run before more spend happens.

What should the runtime check?
Budget

Every agent run should have a task-level budget.

If the next estimated call would exceed that budget, stop.

Max steps

If the agent has already used too many steps, stop.

A confused agent should not be allowed to run forever.

Retry storms

Retries are useful.

Retry storms are dangerous.

If the agent keeps hitting similar failures, the runtime should block the next attempt.

Prompt loops

If the prompt is almost the same as previous failed attempts, the agent may be stuck.

Changing a few words is not progress.

Unknown model pricing

Fallback logic can be dangerous during provider incidents.

If the runtime does not know the price of the fallback model, it should fail closed.

Do not guess inside an autonomous loop.

No progress

A run can be active and still useless.

If the agent is consuming steps and budget without moving closer to a valid result, stop and return a structured error.

A safer retry shape

A better agent retry loop treats every provider call as a decision.

while (!task.done) {
const decision = guard.beforeCall({
runId: task.id,
model: task.model,
messages: task.messages,
stepCount: task.steps.length,
retryCount: task.retryCount,
budgetRemaining: task.budgetRemaining,
previousPrompts: task.previousPrompts,
progressState: task.progress,
});

if (!decision.allowed) {
return {
status: "stopped",
reason: decision.reason,
error: decision.error,
};
}

const response = await provider.call({
model: task.model,
messages: task.messages,
});

task = updateTaskState(task, response);
}

This does not make the model smarter.

It makes the runtime less naive.

That matters when providers have elevated errors.

Where AI CostGuard fits

This is the layer I am building with AI CostGuard.

AI CostGuard is a local-first TypeScript runtime safety layer for AI agents.

It is designed to stop expensive agent failure modes before provider calls execute:

retry storms
prompt loops
max-step explosions
no-progress runs
budget overruns
unknown model pricing
runaway agent behavior

It is not a billing dashboard.

It is not a hard security boundary.

It is not a cloud control plane.

It is a pre-call runtime kill switch for agent cost and loop failures.

The takeaway

Every provider will have incidents.

Claude.

OpenAI.

Gemini.

Any model API.

That is normal infrastructure reality.

The important question is what your agent does during the incident.

If it retries blindly, it can turn provider instability into local cost waste.

If it has runtime guardrails, it can stop cleanly.

AI agents need retry budgets.

Not just retries.
https://github.com/salimassili62-afk/ai-costguard

AI Coding Agents Need Runtime Guardrails, Not Just Human Review

Assili Salim — Tue, 23 Jun 2026 09:07:12 +0000

The recent signal

Anthropic engineering leader Fiona Fung, who leads teams behind Claude Code and Cowork, said AI coding agents have changed how her teams work.

The tools help engineers ship more code, but they also make the work lonelier. Developers spend more time working with agents and less time learning directly with other engineers. Fung’s response was to create more intentional collaboration: programming lunches, hackathons, and shared maker time.

That is an interesting culture story.

It is also an engineering story.

Because when developers move from working with humans to operating agents alone, some safety moves out of the room.

Human collaboration used to catch runtime mistakes

In normal engineering work, many mistakes are caught informally.

Someone asks:

Why are we retrying this again?

Why does this script have no timeout?

Why is this touching production?

Why did the dependency change?

Why is the task not making progress?

Why are we calling this API again?

These are not formal verification systems.

They are human runtime checks.

AI coding agents change the workflow.

A developer gives a task.

The agent plans.

The agent edits.

The agent runs tools.

The agent calls models.

The agent retries.

The agent adds context.

The agent tries again.

That loop can be useful.

It can also continue long after a human teammate would have stopped the run.

Agents are loops

A basic model call is simple.

You send input.

You get output.

You inspect the result.

An agent is different.

An agent can run through many steps:

while (!taskDone) {
const plan = await model.call(context);
const toolResult = await runTool(plan.tool);
context = updateContext(context, toolResult);
}

That shape is powerful, but incomplete.

There is no budget.

No max-step limit.

No retry-storm detection.

No prompt-loop detection.

No unknown-pricing check.

No no-progress detection.

A safer version has to ask whether the next provider call should happen at all.

const decision = guard.beforeCall({
runId,
model,
prompt,
stepCount,
budget,
previousPrompts,
retryState,
});

if (!decision.allowed) {
throw decision.error;
}

const result = await provider.call({
model,
prompt,
});

The important part is placement.

The guard runs before the provider call.

Not after.

Why dashboards are not enough

Dashboards are useful.

They show usage.

They show cost.

They help debug.

But they are usually post-execution.

They answer:

What happened?

Agent runtimes need to answer:

Should this next call happen?

That question matters because most AI-agent cost failures are not one huge request.

They are usually sequences of normal-looking calls.

A retry.

A similar prompt.

A tool call.

Another retry.

A longer context.

Another model call.

Each step may look reasonable.

The whole run may be waste.

Useful pre-call checks

A practical agent runtime should stop execution when basic safety conditions fail.

Budget guard

A run should have a task-level budget.

If the next estimated call crosses that budget, stop before the provider call.

Max-step protection

An agent that cannot finish within a reasonable number of steps may be stuck.

Max-step limits are not fancy.

They are production safety.

Retry-storm detection

Retries are normal.

Retry storms are not.

The runtime should detect when the agent keeps repeating failing behavior.

Similar prompt-loop detection

If prompts are nearly identical across repeated attempts, the agent may not be exploring new information.

That should trigger a stop or structured failure.

Unknown model pricing

If the runtime does not know the price of the model, it should fail closed.

Guessing inside an autonomous loop is dangerous.

Where AI CostGuard fits

This is the layer I am building with AI CostGuard.

AI CostGuard is a local-first TypeScript runtime safety layer for AI agents.

It is designed to prevent expensive agent failure modes before provider calls execute:

retry storms
prompt loops
max-step explosions
no-progress runs
budget overruns
unknown model pricing
runaway agent behavior

It is developer-focused.

No SaaS by default.

No cloud dashboard by default.

No login.

No tracking.

The goal is not to replace provider dashboards.

The goal is to stop known bad runtime patterns before they become provider spend.

The takeaway

AI agents make individual developers more capable.

They also reduce the human friction between intention and execution.

That means some safety has to move closer to the runtime.

If an agent can run alone, it needs limits.

If it can retry, it needs retry control.

If it can loop, it needs loop detection.

If it can call providers, it needs budgets.

If it can run unattended, it needs a kill switch.
https://github.com/salimassili62-afk/ai-costguard

Why Unknown Model Pricing Should Fail Closed in AI-Agent Runtimes

Assili Salim — Mon, 22 Jun 2026 12:40:56 +0000

The problem

Unknown model pricing looks like a small configuration issue.

For AI agents, it can become a runtime failure.

A basic LLM application might send one request and inspect the cost later.

An AI agent is different.

An agent can:

call a model
retry
call tools
add more context
switch models
continue for many steps
fail slowly without crashing

That means one pricing mistake can multiply across a run.

If the runtime does not know the model price, it cannot safely enforce a budget.

That is why unknown model pricing should fail closed before provider API calls execute.

What “fail closed” means

Failing closed means the system refuses to continue when it cannot make a safe decision.

In this case, the decision is:

Should this next provider call be allowed?

If the runtime knows the model price, it can estimate cost and compare it against the run budget.

If the runtime does not know the model price, it should stop.

Not warn only.

Not guess.

Not assume the model is cheap.

Stop and return a structured error.

Why this matters more for agents

Agents are loops.

That is the core issue.

A single model call with unknown pricing is one risk.

An agent loop with unknown pricing is repeated risk.

The agent might make 10, 20, or 50 calls before anyone notices.

The failure may come from something simple:

a typo in the model name
a provider alias that changed
a fallback model that is more expensive
a wrapper passing an unexpected model string
a development config that differs from production
a new model added without pricing metadata

None of these are dramatic.

But cost failures usually are not dramatic.

They are boring runtime mistakes that continue too long.

Budget guards need reliable inputs

A budget guard depends on pricing metadata.

Before a provider call, the runtime might try to answer:

What model is being used?
What is the estimated input cost?
What is the estimated output cost?
How much budget remains for this run?
Should this call be allowed?

If pricing is missing, the budget decision becomes weak.

Here is the rough idea in TypeScript-oriented terms:

const decision = guard.beforeCall({
runId,
model,
inputTokens,
maxOutputTokens,
budget,
});

if (!decision.allowed) {
throw decision.error;
}

const result = await provider.call({
model,
messages,
});

The important part is placement.

The guard runs before the provider call.

If the model price is unknown, execution stops before spend happens.

Why a dashboard is too late

Provider dashboards are useful.

Billing dashboards are useful.

Logs and traces are useful.

But they usually answer the question after execution:

What happened?

Runtime guards answer a different question:

Should this next call happen?

For AI agents, that second question is critical.

Once the provider call executes, the cost exists.

A dashboard might help you understand the mistake.

A pre-call runtime guard can prevent the mistake from continuing.

A safer default

A safe model-pricing rule can be simple:

if (!pricingCatalog.has(model)) {
throw new UnknownModelPricingError(model);
}

This is not complex.

That is the point.

The runtime should not need advanced reasoning to avoid obvious cost ambiguity.

If the price is unknown, the call should not proceed.

The developer can then add pricing metadata, correct the model name, or explicitly change the configuration.

That is better than silent guessing.

Where AI CostGuard fits

This is one of the checks I am building into AI CostGuard.

AI CostGuard is a local-first TypeScript / Node.js runtime safety layer for AI agents.

It is designed to catch cost and loop failures before provider API calls execute.

It focuses on:

retry storm detection
similar prompt loop detection
unknown model pricing blocks
max-step protection
budget guards
middleware and wrappers
structured errors

It is not a billing ledger.

It is not a hard security boundary.

It is not an enterprise firewall.

It is a pre-call runtime kill switch for AI-agent cost and loop failures.

The npm package is @salimassili/ai-costguard.

The takeaway

AI-agent cost control needs boring rules.

Know the model.

Know the price.

Set the budget.

Limit the steps.

Detect loops.

Stop retry storms.

Return structured errors.

Unknown pricing should not be treated as a harmless warning.

For agents, unknown pricing should fail closed before the provider call executes.
https://github.com/salimassili62-afk/ai-costguard

The AI-Agent Call You Should Block Before It Happens

Assili Salim — Sun, 21 Jun 2026 06:28:35 +0000

The most expensive AI-agent call is not always the biggest one.

Sometimes it is the next one.

The one after the agent already failed.

The one after it retried the same operation.

The one after it called tools without progress.

The one after the prompt changed slightly but the task did not move forward.

That provider call may look valid by itself.

The model name may be correct.

The prompt may be well-formed.

The provider may return a normal response.

But inside the whole agent run, the call should not have happened.

That is why AI-agent cost control needs to happen before provider API calls execute.

Agents are loops

A basic LLM request is simple.

Input goes in.

Output comes back.

You can measure the cost afterward.

Agents are different.

An agent may:

call a model
call a tool
retry
modify the prompt
add more context
switch strategy
call another model
retry again

That loop is where cost failures appear.

One step can look reasonable.

Ten steps can look suspicious.

Forty steps can become a budget problem.

The runtime needs to understand the run, not just the individual call.

The boring failure modes

Most AI-agent cost failures are not dramatic.

They are usually boring.

Examples:

retry storms
similar prompt loops
max-step explosions
unknown model pricing
no-progress runs
budget overruns

These are not only billing issues.

They are runtime-control issues.

A billing dashboard can tell you that the failure happened.

A runtime guard can stop the next call before it happens.

That distinction matters.

A provider success can still be a runtime failure

This is easy to miss.

A provider API call can succeed and still be the wrong decision.

The API can return 200.

The model can generate text.

The tool can execute.

The trace can look clean.

But the agent may still be stuck.

This looks simple.

But there are missing questions.

How many steps are allowed?

How much budget can this run spend?

Are the prompts becoming too similar?

Is the agent making progress?

Is the model price known?

Should the next call be blocked?

The exact API is not the point.

The placement is the point.

The check happens before the provider call.

What should be checked before the call?

A useful runtime guard can check simple things.

Is the model price known?

If the runtime does not know the price of a model, guessing is risky.

A typo, alias, fallback, or provider change can break cost assumptions.

A safer default is to fail closed.

Has the run exceeded its budget?

A task-level budget is different from a monthly invoice.

It asks:

How much is this specific run allowed to spend?

Once that limit is reached, the next provider call should not happen.

Has the agent exceeded max steps?

Max-step protection is basic but important.

An agent that cannot finish within a reasonable number of steps may be stuck.

Letting it run forever is not intelligence.

It is missing control.

Is the prompt too similar?

A similar prompt loop is when the agent keeps asking nearly the same thing with small changes.

This can burn tokens without producing new information.

The runtime should detect that pattern.

Is the agent making progress?

No-progress runs are expensive because they look active.

The agent is doing things.

But the task is not moving forward.

A guard should be able to stop this before the loop becomes waste.

Where AI CostGuard fits

This is the layer I am building with AI CostGuard.

AI CostGuard is a local-first TypeScript / Node.js runtime safety layer for AI agents.

It is designed to catch cost and loop failures before provider API calls execute.

It currently focuses on:

retry storm detection
similar prompt loop detection
unknown model pricing blocks
max-step protection
budget guards
middleware and wrappers
structured errors

It is not a billing ledger.

It is not a hard security boundary.

It is not an enterprise firewall.

It is a pre-call runtime kill switch for AI-agent cost and loop failures.

The package is public as @salimassili/ai-costguard.

The takeaway

Cheaper tokens help.

Better routing helps.

Caching helps.

Dashboards help.

But none of them replace runtime limits.

For AI agents, the important question is not only:

How much did this model call cost?

It is:

Should this next provider call be allowed?

That question should be answered before execution.
https://github.com/salimassili62-afk/ai-costguard

Why AI Agents Need Runtime Budgets Before Provider Calls

Assili Salim — Fri, 19 Jun 2026 08:33:25 +0000

The problem

Most AI cost control happens too late.

A provider dashboard can tell you what happened after the API calls already executed.

That is useful.

But it does not stop a bad agent run while it is happening.

For basic LLM usage, this may be acceptable. You send one prompt, receive one response, and check the cost later.

Agents are different.

An AI agent is not one call.

It is a loop.

That loop may include:

model calls
tool calls
retries
fallback models
growing context
planning steps
validation steps
more retries

Each step may look reasonable by itself.

The failure appears across the whole run.

The expensive failure is usually boring

Many AI cost failures are not dramatic.

They are simple runtime failures:

the agent retries too many times
the prompt changes slightly but not meaningfully
the agent keeps calling tools without progress
the run exceeds a safe step count
the model price is unknown
the workflow crosses a budget limit

None of these require a complex theory.

They require boring runtime controls.

That is the point.

Production software already has limits everywhere.

Timeouts.

Memory limits.

Rate limits.

Retry limits.

Circuit breakers.

AI-agent runtimes need the same kind of thinking.

A dashboard is not a guardrail

A dashboard answers:

“What happened?”

A runtime guard answers:

“Should this next call happen?”

Those are different questions.

The second one is more important during execution.

Once the provider call is made, the cost is already real.

That is why AI-agent cost control should not only happen after the invoice.

It should happen before provider API calls execute.

Simple TypeScript-oriented thinking

Imagine an agent step before a provider call.

Before sending the request, the runtime can check a few things:

const decision = guard.beforeCall({
runId,
model,
prompt,
step,
estimatedCost,
});

if (!decision.allowed) {
throw decision.error;
}

const result = await provider.call({
model,
prompt,
});

The important idea is not the exact API.

The important idea is the position of the check.

It happens before the provider call.

That means the runtime can block dangerous behavior before money is spent.

Useful checks before the call

A practical guard layer can ask:

Is this model price known?

If not, fail closed.

Has this run exceeded its budget?

If yes, stop.

Has this agent exceeded max steps?

If yes, stop.

Is this prompt too similar to previous failed attempts?

If yes, block the loop.

Is the agent making no progress?

If yes, return a structured error.

These checks do not make the model smarter.

They make the runtime safer.

That matters.

Unknown model pricing should fail closed

Unknown pricing is easy to underestimate.

A typo in a model name can break assumptions.

A provider alias can change.

A fallback can route to something more expensive.

A dashboard may show this later.

A runtime guard can stop it before the call.

For production agent workflows, unknown pricing should be treated as a risk.

Failing closed is safer than guessing.

Max-step limits are production safety

A max-step limit sounds basic.

It is basic.

That is why it belongs in the runtime.

An agent that cannot finish in a reasonable number of steps may be confused.

Letting it continue forever is rarely useful.

A step limit gives the system a clear stopping point.

It also gives the developer a structured failure to inspect.

That is better than silent spending.

Where AI CostGuard fits

This is the layer I am building with AI CostGuard.

AI CostGuard is a local-first TypeScript / Node.js runtime safety layer for AI agents.

It is designed to catch cost and loop failures before provider API calls execute.

Current checks include:

retry storm detection
similar prompt loop detection
unknown model pricing blocks
max-step protection
budget guards
middleware and wrapper support
structured errors

It is not a billing ledger.

It is not a hard security boundary.

It is not an enterprise firewall.

It is a pre-call runtime kill switch for AI-agent cost and loop failures.

The takeaway

Cheaper tokens help normal runs.

Caching helps normal runs.

Routing helps normal runs.

But abnormal agent behavior needs runtime limits.

The key question is not only:

“How much did this model cost?”

The better question is:

“Should this next provider call be allowed?”

For AI agents, that question belongs before execution.