DEV Community

Hossein Hezami
Hossein Hezami

Posted on

The Hidden Cost of AI Agents: A Token Budget Framework for Production Laravel Apps

#ai #laravel #cost #agents

Your Laravel app adds an agent. The controller action looks harmless: one user message, one model call, maybe a tool or two.

Then a real tenant connects a shared inbox, a ticket thread, or a document library. The agent starts “investigating.” It calls search. It reads a record. It calls search again. It re-sends the same 40 KB conversation on every turn. It retries after a timeout. The user sees one button click. Your billing dashboard sees a tiny distributed denial-of-wallet attack.

The uncomfortable truth is that AI agents do not fail like normal features. They fail multiplicatively.

A normal API endpoint has a fairly predictable cost: one database query, one HTTP call, one render. An agent endpoint has a cost shaped by a loop:

cost = sum over turns(
  input_tokens × input_price
  + output_tokens × output_price
  + tool_output_tokens_sent_back_into_context
)
Enter fullscreen mode Exit fullscreen mode

The first prompt is usually cheap. The tenth turn, after six tool calls and three retrieved documents, is where the invoice starts to look personal.

TL;DR: Production AI agents need a token budget framework, not just a max_tokens parameter. Treat tokens like money, meter actual provider usage, cap loops, trim context, limit tool output, route models by task, enforce tenant quotas, and observe burn rate before finance finds it first.

📋 Table of Contents

The Real Problem Is the Loop, Not the Prompt

Most Laravel teams start with something like this:

$response = Http::withToken(config('services.openai.key'))
    ->post('https://api.openai.com/v1/chat/completions', [
        'model' => 'gpt-4.1',
        'messages' => $messages,
    ]);
Enter fullscreen mode Exit fullscreen mode

That is fine for a simple completion feature.

An agent is different. An agent usually does this:

  1. Receives a user request.
  2. Sends system prompt, tool definitions, conversation history, and context to a model.
  3. Receives a response or tool calls.
  4. Executes tools.
  5. Sends the tool results back to the model.
  6. Repeats until done.

Every iteration can re-send much of the previous context. Tool outputs become future input. Tool schemas consume tokens. Failed attempts can still consume input tokens. Streaming does not make tokens free. Retries can double the bill if the first attempt already processed a large prompt.

In production, the dangerous cases are not exotic. They are ordinary:

  • A support agent reads a long email thread, then reads it again after every tool call.
  • A document assistant pulls in ten chunks when two would have been enough.
  • A database agent returns a raw JSON dump instead of a summarized result.
  • A prompt-injected page causes the agent to loop through tools.
  • A queued job retries three times because of a transient provider timeout.
  • A tenant with unlimited usage discovers that your “internal helper” can summarize their entire CRM.

The fix is not one setting. It is a framework.

A token budget framework answers five questions:

  1. Who owns the budget? User, tenant, feature, agent, or request?
  2. What is the budget? Per turn, per session, per day, per month?
  3. When do we check? Before dispatch, before each model call, after usage?
  4. What happens when the budget is exhausted? Stop, degrade, summarize, ask for approval?
  5. How do we see the burn? Logs, metrics, alerts, admin dashboards?

The rest of this article builds that framework piece by piece.

1. Treat Tokens Like Money Not Debug Logs

Scenario:

Your agent works in development because you are testing short prompts. In production, a tenant pastes a 20-page contract into the prompt and asks, “What are the risks?”

Why it matters:

If tokens are only mentioned in debug logs, nobody thinks about them until the bill arrives. Budgets need to be first-class runtime values, like request input, authorization, or rate limits.

Solution:

Create an explicit budget object and pass it through the agent execution path.

<?php

namespace App\Support\Ai;

final class TokenBudget
{
    private int $used = 0;

    public function __construct(
        public readonly int $total,
        public readonly int $maxInputPerTurn,
        public readonly int $maxOutputPerTurn,
        public readonly int $maxTurns,
    ) {}

    public function consume(TokenUsage $usage): void
    {
        $newTotal = $this->used + $usage->total();

        if ($newTotal > $this->total) {
            throw BudgetExceededException::forUsage($usage, $this);
        }

        $this->used = $newTotal;
    }

    public function remaining(): int
    {
        return max(0, $this->total - $this->used);
    }

    public function canAffordEstimatedInput(int $estimatedTokens): bool
    {
        return $estimatedTokens + $this->maxOutputPerTurn <= $this->remaining();
    }
}
Enter fullscreen mode Exit fullscreen mode

This object is deliberately boring. It does not know about OpenAI, Anthropic, vector databases, or your database schema. It only knows that tokens are finite.

You can create budgets from configuration, tenant plans, or request context:

$budget = new TokenBudget(
    total: 50_000,
    maxInputPerTurn: 12_000,
    maxOutputPerTurn: 2_000,
    maxTurns: 6,
);
Enter fullscreen mode Exit fullscreen mode

Why this works:

A budget object turns a vague cost concern into an executable constraint. It can be passed into context builders, tool executors, model routers, and logging listeners. It also gives you one place to implement soft limits, hard limits, and degradation behavior.

🧠 The important part: separate context-window limits from cost limits. A request may fit in the model’s context window but still exceed the amount of money you are willing to spend on that user action.

2. Meter Actual Usage Not Estimates

Scenario:

You estimate tokens from strlen($prompt) / 4 before calling the model. It is close enough in simple cases, but tool results, JSON escaping, Unicode, and provider-specific tokenization keep shifting the real count.

Why it matters:

Estimates are useful for preflight checks, but they are not safe as the source of truth for billing, quotas, or budget enforcement. Provider APIs already report usage. Use it.

Solution:

Normalize provider usage into your own value object.

<?php

namespace App\Support\Ai;

final class TokenUsage
{
    public function __construct(
        public readonly int $input,
        public readonly int $output,
        public readonly int $cacheRead = 0,
        public readonly int $cacheWrite = 0,
        public readonly int $reasoning = 0,
    ) {}

    public function total(): int
    {
        return $this->input + $this->output + $this->reasoning;
    }
}
Enter fullscreen mode Exit fullscreen mode

Then normalize responses from different providers. The exact field names should always be checked against the provider’s current documentation, but the shape is usually similar:

<?php

namespace App\Support\Ai;

final class ProviderUsageNormalizer
{
    public function fromOpenAi(array $body): TokenUsage
    {
        $usage = $body['usage'] ?? [];

        return new TokenUsage(
            input: (int) ($usage['prompt_tokens'] ?? 0),
            output: (int) ($usage['completion_tokens'] ?? 0),
            cacheRead: (int) ($usage['prompt_tokens_details']['cached_tokens'] ?? 0),
        );
    }

    public function fromAnthropic(array $body): TokenUsage
    {
        $usage = $body['usage'] ?? [];

        return new TokenUsage(
            input: (int) ($usage['input_tokens'] ?? 0)
                + (int) ($usage['cache_read_input_tokens'] ?? 0)
                + (int) ($usage['cache_creation_input_tokens'] ?? 0),
            output: (int) ($usage['output_tokens'] ?? 0),
            cacheRead: (int) ($usage['cache_read_input_tokens'] ?? 0),
            cacheWrite: (int) ($usage['cache_creation_input_tokens'] ?? 0),
        );
    }
}
Enter fullscreen mode Exit fullscreen mode

Now your agent loop consumes actual usage:

$usage = $normalizer->fromOpenAi($response->json());
$budget->consume($usage);
Enter fullscreen mode Exit fullscreen mode

Why this works:

Actual usage is the number the provider uses to compute cost and context consumption. It also captures output tokens, cached tokens, and, depending on provider, reasoning-related tokens.

Use estimates only for early decisions:

  • Should we even attempt this request?
  • Do we need to trim context?
  • Can we afford another turn?
  • Should we switch to a cheaper model?

Then reconcile with actual usage after the call.

⚠️ Gotcha: streaming responses can make usage easy to miss. Some providers require an explicit option to include usage metadata in streamed responses, while others provide it through final stream events. If you stream, make usage collection part of the stream handler, not an afterthought.

3. Cap the Agent Loop Not Just the Completion

Scenario:

A user asks, “Find every order that failed refund validation.” The agent calls a search tool. The result is ambiguous, so it calls again. Then it calls a database tool. Then it calls search again. Each turn is individually small. The loop is not.

Why it matters:

max_tokens or max_completion_tokens only limits one model completion. It does not limit the number of completions, the amount of tool output fed back into the model, or the total cost of the agent session.

Solution:

Wrap the agent in an explicit runner with loop limits and budget checks.

<?php

namespace App\Support\Ai;

final class AgentRunner
{
    public function __construct(
        private readonly ProviderClient $provider,
        private readonly ToolExecutor $tools,
        private readonly ContextBuilder $context,
    ) {}

    public function run(AgentRequest $request, TokenBudget $budget): AgentResult
    {
        $conversation = $request->conversation;

        for ($turn = 1; $turn <= $budget->maxTurns; $turn++) {
            $prompt = $this->context->build($conversation, $budget);

            if (! $budget->canAffordEstimatedInput($prompt->estimatedTokens)) {
                return AgentResult::stoppedByBudget($budget);
            }

            $response = $this->provider->complete(
                prompt: $prompt,
                model: $request->model,
                maxOutputTokens: $budget->maxOutputPerTurn,
            );

            $budget->consume($response->usage);

            $conversation->appendAssistantMessage($response->message);

            if ($response->toolCalls === []) {
                return AgentResult::completed($response->message, $budget);
            }

            $toolResults = $this->tools->executeAll($response->toolCalls, $budget);
            $conversation->appendToolResults($toolResults);
        }

        return AgentResult::stoppedByTurnLimit($budget);
    }
}
Enter fullscreen mode Exit fullscreen mode

The exact class names are less important than the structure. The runner owns the loop. The model does not.

Why this works:

Agents are state machines. If the state machine has no exit conditions, the model can keep requesting tools indefinitely. A loop cap is not a hack; it is part of the product contract.

Good loop controls include:

  • Maximum turns.
  • Maximum total tokens.
  • Maximum repeated tool calls.
  • Maximum identical tool calls with identical arguments.
  • Maximum wall-clock time.
  • A stop reason when the budget is exhausted.

Practical note:

Be careful with retries. If a provider call fails after processing a large prompt, the provider may still bill input tokens. A retry should not reset the budget. It should consume from the same budget unless you have confirmed that no usage occurred.

4. Make Context Assembly Budget Aware

Scenario:

Your chat assistant works beautifully for five messages. Then a customer support conversation reaches 80 messages, and every new model call includes the entire history.

Why it matters:

In agent systems, input tokens usually dominate cost. Every turn can re-send the system prompt, tool definitions, previous assistant messages, previous tool calls, and previous tool outputs. If context assembly is naive, cost grows faster than value.

Solution:

Make context building a budget-aware step, not a passive implode() of messages.

<?php

namespace App\Support\Ai;

final class ContextBuilder
{
    public function build(Conversation $conversation, TokenBudget $budget): Prompt
    {
        $maxInput = min(
            $budget->maxInputPerTurn,
            $budget->remaining() - $budget->maxOutputPerTurn,
        );

        if ($maxInput <= 0) {
            throw BudgetExceededException::noRemainingInput($budget);
        }

        $messages = $this->selectRecentMessages($conversation, $maxInput);

        if ($this->estimateTokens($messages) > $maxInput) {
            $messages = $this->summarizeOlderMessages($conversation, $messages, $maxInput);
        }

        return new Prompt(
            system: $this->stableSystemPrompt(),
            tools: $this->toolDefinitions(),
            messages: $messages,
            estimatedTokens: $this->estimateTokens($messages),
        );
    }
}
Enter fullscreen mode Exit fullscreen mode

The exact trimming strategy depends on the product. Common approaches include:

  • Keep the system prompt and tool definitions stable.
  • Keep the latest user message in full.
  • Keep the last few assistant/tool exchanges in full.
  • Summarize older conversation history.
  • Replace large tool outputs with compact summaries after they have been used.
  • Drop irrelevant tool results rather than carrying them forever.

A useful mental model:

context = stable prefix + relevant retrieved context + recent working memory
Enter fullscreen mode Exit fullscreen mode

The stable prefix should rarely change. The retrieved context should be selected deliberately. The working memory should be small.

Why this works:

Budget-aware context assembly forces the system to ask, “Does this token deserve to be in the next request?” That question is the heart of cost control.

Example of a bad pattern:

$messages = $conversation->allMessages();
Enter fullscreen mode Exit fullscreen mode

Better:

$messages = $this->contextBuilder->build($conversation, $budget)->messages;
Enter fullscreen mode Exit fullscreen mode

💡 Practical note: avoid putting huge raw JSON blobs into context “just in case.” The model rarely needs the full payload. It usually needs a narrowed projection: id, status, total, error code, timestamp, and maybe one relevant nested field.

5. Put Tool Outputs on a Strict Token Diet

Scenario:

Your agent has a search_orders tool. The user asks for refunds from last month. The tool returns 300 orders as JSON. The model now has to read all of it, and the next turn has to re-send all of it.

Why it matters:

Tool output is not just an API result. It becomes future input. A single oversized tool call can poison the rest of the session.

Solution:

Give every tool a result policy.

<?php

namespace App\Support\Ai;

final class ToolResult
{
    public function __construct(
        public readonly string $toolCallId,
        public readonly string $content,
        public readonly bool $truncated,
        public readonly ?string $summary = null,
    ) {}
}
Enter fullscreen mode Exit fullscreen mode 
final class ToolExecutor
{
    public function executeAll(array $toolCalls, TokenBudget $budget): array
    {
        $results = [];

        foreach ($toolCalls as $call) {
            $raw = $this->dispatch($call);

            $maxTokensForTool = min(
                2_000,
                (int) floor($budget->remaining() / 2),
            );

            $limited = $this->truncateToTokenLimit($raw, $maxTokensForTool);

            $results[] = new ToolResult(
                toolCallId: $call->id,
                content: $limited->text,
                truncated: $limited->truncated,
                summary: $limited->summary,
            );
        }

        return $results;
    }
}
Enter fullscreen mode Exit fullscreen mode

The truncation strategy should be tool-specific.

For database tools, return aggregates instead of rows:

Found 312 orders. 287 succeeded, 25 failed. Failed total: $4,318.20.
Top failure reasons: card_declined, insufficient_funds.
Enter fullscreen mode Exit fullscreen mode

For document tools, return snippets with references:

Section 4.2 mentions liability caps. Section 7.1 mentions termination notice.
Use document_id 8841 and section anchors for follow-up.
Enter fullscreen mode Exit fullscreen mode

For API tools, return only the fields the agent needs:

return [
    'id' => $order->id,
    'status' => $order->status,
    'refund_state' => $order->refund_state,
    'error' => $order->refund_error,
    'amount' => $order->amount->format(),
];
Enter fullscreen mode Exit fullscreen mode

Why this works:

The agent does not need the world. It needs enough signal to choose the next action. Compact tool results reduce the current response, all future turns, and the probability that the model gets distracted by irrelevant fields.

🚨 Production warning: never let an agent pass raw external content directly into the next model call without limits. A tool that fetches web pages, emails, or uploaded documents can become a token amplifier and a prompt-injection vector.

6. Route Work to the Cheapest Model That Can Do It

Scenario:

Every request goes through your most capable model. Some requests genuinely need it. Others are basically intent classification, summarization, or FAQ lookup.

Why it matters:

Model pricing is not uniform. A larger model can cost many times more per token than a smaller model. If every request uses the most expensive model, your cost structure is tied to your worst-case task instead of your average task.

Solution:

Introduce a model router.

<?php

namespace App\Support\Ai;

enum ModelTier: string
{
    case Fast = 'fast';
    case Balanced = 'balanced';
    case Deep = 'deep';
}
Enter fullscreen mode Exit fullscreen mode 
final class ModelRouter
{
    public function choose(AgentRequest $request, TokenBudget $budget): string
    {
        if ($budget->remaining() < 8_000) {
            return config('ai.models.fast');
        }

        return match ($request->complexity) {
            Complexity::Low => config('ai.models.fast'),
            Complexity::Medium => config('ai.models.balanced'),
            Complexity::High => config('ai.models.deep'),
        };
    }
}
Enter fullscreen mode Exit fullscreen mode

Keep model names in configuration, not scattered through business logic:

// config/ai.php

return [
    'models' => [
        'fast' => env('AI_MODEL_FAST'),
        'balanced' => env('AI_MODEL_BALANCED'),
        'deep' => env('AI_MODEL_DEEP'),
    ],
];
Enter fullscreen mode Exit fullscreen mode

Routing can be based on several signals:

  • Task type.
  • Tenant plan.
  • Remaining budget.
  • Required tool complexity.
  • Whether the request is customer-facing or internal.
  • Whether the request requires strict structured output.
  • Whether the request involves legal, medical, financial, or other sensitive analysis.

Why this works:

A router turns model selection into a policy decision. That means finance, product, and engineering can change the policy without rewriting agent code.

Practical note:

Do not route solely on prompt length. A short prompt can require hard reasoning, and a long prompt can be a simple summarization task. Use task metadata where possible, and fall back to conservative defaults when uncertain.

A good pattern is “cheap first, escalate with budget”:

  1. Try the fast model for classification or simple extraction.
  2. If confidence is low, escalate.
  3. Check remaining budget before escalation.
  4. Log every escalation so you can tune the router later.

7. Cache Stable Context but Be Careful with Agent State

Scenario:

Every request sends the same system prompt, the same tool definitions, and the same policy document. You are paying for the same prefix repeatedly.

Why it matters:

If a large part of your prompt is stable, caching can reduce cost and latency. Some providers offer prompt caching or cached-input pricing. Even when provider-level caching is not available, application-level caching can reduce repeated retrieval and preprocessing work.

Solution:

Separate stable prompt material from volatile prompt material.

Good candidates for stable caching:

  • System prompt.
  • Tool definitions.
  • Product policy text.
  • Compliance instructions.
  • Retrieved document chunks that rarely change.
  • Embeddings or search indexes for immutable documents.

Bad candidates for blind caching:

  • Live database state.
  • User-specific permissions.
  • Time-sensitive data.
  • Agent state that includes side effects.
  • Answers that must reflect the current moment.

A simple Laravel cache can help with expensive preprocessing:

$policyChunks = Cache::remember(
    "ai:policy:v{$policy->version}:chunks",
    now()->addHour(),
    fn () => $this->chunkPolicy($policy),
);
Enter fullscreen mode Exit fullscreen mode

For provider-level prompt caching, the implementation depends on the provider. The architectural rule is the same: put stable content early and volatile content late.

[stable system prompt]
[stable tool definitions]
[stable policy context]
[tenant-specific context]
[conversation history]
[current user message]
Enter fullscreen mode Exit fullscreen mode

If you put a timestamp, random request ID, or user name at the top of the prompt, you may reduce cache effectiveness.

Why this works:

Caching rewards determinism. If your prompt prefix changes every request, the cache cannot help. If your prompt prefix is stable, repeated requests can reuse work.

⚠️ Gotcha: do not cache final agent answers if the answer can trigger side effects, depends on permissions, or may become stale. Caching is excellent for stable context and expensive preprocessing. It is much riskier for autonomous action.

8. Enforce Tenant Budgets Before Queues Multiply Your Costs

Scenario:

A user clicks “Run agent.” Your controller dispatches a queued job. The job fails because of a provider timeout. Laravel retries it. The user clicks again. Now the same expensive operation is running multiple times, and none of the attempts know about the others.

Why it matters:

Queues are excellent for reliability, but they can turn one expensive request into several expensive requests. Budget enforcement must happen before dispatch and inside the job.

Solution:

Use a server-side quota store. Redis is a good fit because it supports atomic increments and expirations.

A quota key might look like this:

ai:tokens:tenant_42:2026-06-22
Enter fullscreen mode Exit fullscreen mode

Use a Lua script to check and increment atomically:

local current = tonumber(redis.call('GET', KEYS[1]) or '0')
local delta = tonumber(ARGV[1])
local limit = tonumber(ARGV[2])
local ttl = tonumber(ARGV[3])

if current + delta > limit then
    return -1
end

local used = redis.call('INCRBY', KEYS[1], delta)

if current == 0 then
    redis.call('EXPIRE', KEYS[1], ttl)
end

return used
Enter fullscreen mode Exit fullscreen mode

In Laravel:

use Illuminate\Support\Facades\Redis;

final class TenantTokenQuota
{
    private const SCRIPT = <<<'LUA'
local current = tonumber(redis.call('GET', KEYS[1]) or '0')
local delta = tonumber(ARGV[1])
local limit = tonumber(ARGV[2])
local ttl = tonumber(ARGV[3])

if current + delta > limit then
    return -1
end

local used = redis.call('INCRBY', KEYS[1], delta)

if current == 0 then
    redis.call('EXPIRE', KEYS[1], ttl)
end

return used
LUA;

    public function recordUsage(string $tenantId, string $day, int $tokens): void
    {
        $key = "ai:tokens:{$tenantId}:{$day}";
        $limit = $this->dailyLimitFor($tenantId);

        $result = Redis::eval(self::SCRIPT, 1, $key, $tokens, $limit, 86_400);

        if ($result === -1) {
            throw BudgetExceededException::forTenant($tenantId);
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Then check before starting expensive work:

final class EnsureAgentBudget
{
    public function __construct(
        private readonly TenantTokenQuota $quota,
    ) {}

    public function handle(Request $request, Closure $next)
    {
        $tenant = $request->attributes->get('tenant');

        if ($this->quota->isExhausted($tenant)) {
            abort(429, 'AI token budget exhausted.');
        }

        return $next($request);
    }
}
Enter fullscreen mode Exit fullscreen mode

For queued jobs, add another check inside the job:

public function handle(): void
{
    if ($this->quota->isExhausted($this->tenantId)) {
        $this->release(300);

        return;
    }

    // Run the agent, then record actual usage.
}
Enter fullscreen mode Exit fullscreen mode

Why this works:

Middleware can stop obvious abuse before work begins, but the real enforcement needs to be close to the model call. The job may run seconds or minutes later. The tenant may have exhausted the budget in another request.

A more advanced pattern is reserve and settle:

  1. Reserve an estimated number of tokens before dispatch.
  2. Run the agent.
  3. Record actual usage.
  4. Release or consume the difference.

This prevents two concurrent jobs from both believing there is budget available.

🔍 Why this matters: client-side estimates are not a security boundary. A malicious or buggy client can claim the prompt is small. Your server must meter actual usage and enforce quotas where it controls the provider call.

9. Watch Burn Rate Before It Becomes an Invoice

Scenario:

The agent works. Users like it. Then, at the end of the month, someone opens the provider dashboard and asks why the AI line item tripled.

Why it matters:

Token budgets are not set-and-forget. They need feedback. If you cannot see which tenant, agent, model, tool, or route is consuming tokens, you cannot tune the system.

Solution:

Emit structured usage events after every model call.

<?php

namespace App\Events;

final class TokenUsageRecorded
{
    public function __construct(
        public readonly string $tenantId,
        public readonly string $agent,
        public readonly string $model,
        public readonly int $turn,
        public readonly TokenUsage $usage,
        public readonly bool $budgetRejected,
    ) {}
}
Enter fullscreen mode Exit fullscreen mode

A listener can write to logs, metrics, or a database:

final class LogTokenUsage
{
    public function handle(TokenUsageRecorded $event): void
    {
        logger()->channel('metrics')->info('ai.token_usage', [
            'tenant' => $event->tenantId,
            'agent' => $event->agent,
            'model' => $event->model,
            'turn' => $event->turn,
            'input_tokens' => $event->usage->input,
            'output_tokens' => $event->usage->output,
            'cache_read_tokens' => $event->usage->cacheRead,
            'cache_write_tokens' => $event->usage->cacheWrite,
            'reasoning_tokens' => $event->usage->reasoning,
            'budget_rejected' => $event->budgetRejected,
        ]);
    }
}
Enter fullscreen mode Exit fullscreen mode

The metrics that matter most are usually not raw token counts alone. Track ratios and percentiles:

  • Tokens per completed task.
  • Tokens per session.
  • Tokens per tenant per day.
  • Input-to-output ratio.
  • Cache hit ratio.
  • Average turns per agent run.
  • p95 tool output size.
  • Budget rejection rate.
  • Escalation rate from small model to large model.
  • Cost per successful resolution.

Why this works:

Averages hide problems. One tenant with a pathological document can dominate your spend. One tool returning giant JSON blobs can inflate every agent run. Percentiles and per-task metrics reveal those issues quickly.

Useful alerts include:

  • A tenant exceeds 80% of daily budget.
  • An agent’s average turns suddenly increase.
  • Tool output p95 grows beyond a threshold.
  • Budget rejections spike.
  • Cache hit ratio drops.
  • A new model version changes token consumption.

Practical note:

Log the reason a budget stopped an agent. “Completed,” “turn limit,” “token budget exhausted,” and “tool output too large” are different product problems. If they all look like generic failures, you will not know what to fix.

Comparison of Budget Controls

No single control solves the problem. The framework works because each control catches a different failure mode.

Control What it stops Cost impact Implementation effort Best used for
Output token limit Runaway completions Moderate Low Every model call
Max turns Infinite tool loops High Low Every agent loop
Total session budget Expensive sessions High Medium User-facing agents
Context trimming Bloated history High Medium Chat and support agents
Tool output limits Oversized tool results Very high Medium Database, document, API tools
Model routing Overusing expensive models High Medium Mixed-complexity workloads
Prompt caching Repeated stable context Moderate to high Medium Stable prompts and docs
Tenant quotas One tenant consuming everything Very high Medium to high Multi-tenant SaaS
Usage observability Silent cost growth Preventive Medium All production systems

If you only implement three controls first, choose:

  1. Max turns.
  2. Tool output limits.
  3. Actual usage logging.

Those three will catch the most common production explosions.

What I Would Ship First

If I were adding agents to a production Laravel app, I would not start by optimizing prompt wording or chasing the newest model. I would ship the minimum cost-control surface first.

Before the first production tenant

  • Create a TokenBudget object.
  • Record actual provider usage after every call.
  • Set a hard maximum number of agent turns.
  • Set a hard maximum output token limit.
  • Log tenant, agent, model, turn, input tokens, output tokens, and stop reason.

Before enabling multiple tenants

  • Add tenant daily and monthly token quotas.
  • Enforce quotas before dispatching queued jobs.
  • Add idempotency keys for agent actions that cause side effects.
  • Prevent queued retries from silently multiplying model calls.
  • Add an admin view showing top tenants by token consumption.

Before optimizing cost aggressively

  • Add context trimming and summarization.
  • Limit tool output by tool type.
  • Route simple tasks to cheaper models.
  • Cache stable prompt prefixes and retrieved documents.
  • Measure cost per completed task, not just cost per API call.

When the system is already expensive

Do not start by lowering quality. Start by finding the leak.

Look for:

  • Tools returning raw database dumps.
  • Conversations carrying huge tool outputs across many turns.
  • Retries caused by provider timeouts.
  • Prompts with volatile prefixes defeating caching.
  • One tenant using the agent as a batch-processing engine.
  • Agents escalating to expensive models unnecessarily.

The goal is not to make agents unusable. It is to make their cost legible.

A production-ready Laravel agent should behave like a well-designed employee: it should know when to stop, what it is allowed to read, which tools are expensive, and when to ask for approval.

The model provides the reasoning. Your application provides the boundaries.

A token budget framework is how you make those boundaries executable.

Top comments (0)