DEV Community: Dewald Hugo

Laravel AI Integration: A Production-Ready Architecture Guide (OpenAI vs Gemini vs Claude)

Dewald Hugo — Mon, 27 Apr 2026 00:47:06 +0000

Every Laravel AI integration starts the same way. You install a package, copy an API key into .env, write a controller method that fires a prompt, and it works. Then you ship it. Then you get your first production incident, a 429 rate limit silences a customer-facing feature at 2AM, a Gemini model deprecation breaks a workflow three sprints before anyone noticed the announcement, or a runaway token loop burns through your monthly budget in four hours.

That is the moment you realise that Laravel AI integration is not an API problem. It is an architecture problem. The API call is the easy part. The hard part is building the layer around it that makes AI behaviour predictable, cost-controlled, and provider-agnostic.

This guide does not walk you through any provider’s authentication flow. It gives you the system-level map: the problem space, the architectural patterns that address it, a clear-eyed comparison of OpenAI, Gemini, and Claude, and a decision framework you can put in front of your team before the next kickoff.

The Laravel AI Integration Problem Space

Before picking a provider or drawing a service diagram, you need to be honest about what you are actually building. The problems that make AI difficult in production are not unique to any one provider. They are structural.

Provider Abstraction

You will almost certainly swap or combine providers over the lifecycle of a product. The model that makes sense at launch is rarely the model that makes sense at scale, or after a pricing change, or when your use case evolves. If your application code couples directly to OpenAI’s SDK or Anthropic’s client, a provider change means rewriting feature logic, not just swapping a configuration value.

The correct answer is a provider interface your application depends on, with concrete adapters behind it. We cover the interface contract below.

Token Usage and Cost Control

Language models charge by the token. That sounds simple. In practice it means a single misconfigured prompt, multiplied by your queue throughput, can produce a five-figure bill in an afternoon. We have seen this happen.

Token management is not optional in production. You need budget enforcement before the API call, usage logging at the response level, and alerting when spend crosses threshold. This belongs in the service layer – not in a controller, not in a scheduled job that runs after the damage is done.

Latency and Streaming

A synchronous AI call in an HTTP handler is almost always wrong. Inference latency runs from 800ms on a lightweight model to 30 seconds on a long-form reasoning task. PHP-FPM holds a connection open for that entire window. Under any meaningful traffic, that collapses your worker pool.

Streaming helps on the UX side, but it does not fix the connection pressure problem. Background jobs fix that. The streaming question is a separate decision, covered in its own section below.

Error Handling and Retries

AI APIs fail. They return 429s during traffic spikes, 500s during model instability, and occasionally time out with no response body at all. None of these are exceptional conditions. They are routine.

Your retry logic must distinguish between transient failures (worth retrying with backoff) and structural failures (wrong API key, malformed payload, context length exceeded). Treating all failures the same is one of the most common production bugs we see in AI-integrated Laravel applications.

Vendor Lock-in

Every provider has proprietary extensions – function calling formats, system prompt conventions, streaming event shapes, vision attachment schemas. The moment you hardcode those shapes into your application logic, you own a migration project the next time you need to switch.

Abstraction is the mitigation. But abstraction has a cost. Build it at the right layer, and keep provider-specific behaviour in the adapter, not in your domain services.

Provider Comparison: OpenAI vs Gemini vs Claude

No provider is the right answer for every use case. Here is an honest comparison of the three dominant options for Laravel AI integration.

Capability	OpenAI (GPT-4o)	Google (Gemini 2.5 Pro)	Anthropic (Claude Sonnet 4.5)
Context window	128K tokens	1M+ tokens	200K tokens
Streaming support	✓	✓	✓
Function / tool calling	✓	✓	✓
Vision / multimodal	✓	✓	✓
Native embeddings	✓ (text-embedding-3)	✓	✗ (use OpenAI or Gemini)
Laravel SDK support	First-party	Via laravel/ai	Via laravel/ai
Reasoning quality	Strong, broad	Strong, long-context	Exceptional, safety-aware
Rate limit tolerance	Good	Variable	Conservative
Pricing tier	Mid	Mid	Mid-High

Honest notes on each:

OpenAI has the most mature Laravel ecosystem. The first-party package is stable, the API surface is well-documented, and function calling is production-tested at scale. It is the right default for teams building general-purpose AI features with limited architectural budget.

Gemini’s one-million-token context window is not a marketing number, it genuinely changes what is possible for document processing, long-session memory, and RAG-less retrieval patterns. The laravel/ai SDK wraps it cleanly. The downside is that rate limits are inconsistently enforced and the model behaviour at the boundary of that context window is less predictable than Anthropic’s or OpenAI’s.

Claude is the most consistent model for reasoning-heavy tasks, structured output extraction, code review pipelines, and anything that requires following complex multi-step instructions reliably. The safety constraints are real and occasionally frustrating for edge cases, but the instruction-following quality justifies the slightly higher cost for high-stakes features.

Recommended Architecture for Laravel AI Systems

The pattern that holds up across every provider, every scale, and every use case is the same: isolate AI behind a typed contract, implement concrete adapters per provider, and register the active provider via the Service Container.

namespace App\Contracts\AI;

interface AIProviderInterface
{
    public function generate(string $prompt, array $options = []): string;

    public function stream(string $prompt, array $options = []): \Generator;

    public function embed(string $text): array;
}

Each provider - OpenAI, Gemini, Claude - gets its own adapter class that implements this interface. Your application services, queued jobs, and controllers depend on AIProviderInterface. They never know which provider sits behind it.

// AppServiceProvider or bootstrap/app.php
$this->app->bind(
    \App\Contracts\AI\AIProviderInterface::class,
    \App\AI\Adapters\ClaudeAdapter::class,
);

Swapping providers is now a one-line configuration change. No domain logic moves.

On top of this interface sits your governance layer – budget enforcement, telemetry, prompt versioning. Our guide on production-grade AI architecture in Laravel covers that layer in full, including typed AiResult data objects, decorator chains, and audit trail patterns. Read that guide before you build the adapter layer, the contract it defines will inform how you structure the interface above.

[Architect’s Note] The three-method interface above is intentionally minimal. Resist the temptation to add provider-specific capabilities - vision input, citation formats, tool schemas - to the shared interface. Those belong in extended interfaces or provider-specific services consumed explicitly. The shared interface exists to make the 80% case portable. The 20% case is allowed to know which provider it is talking to.

Configuration Management

Provider credentials live in config/ai.php, not scattered across service provider constructors. Each adapter reads from config('ai.providers.claude.api_key') and so on. Environment variables map cleanly through the config layer, and you can define per-provider defaults - temperature, max tokens, retry budget - without touching application code.

Token Management and Cost Control

This is the section most architectural guides skip. It is also the section that determines whether your AI feature is financially viable.

Token costs compound. A feature that runs 500 times per day at 2,000 tokens per call is one million tokens daily. Multiply by your model’s per-token rate, multiply by 30, and you have a monthly infrastructure line item that belongs in your budget conversations from day one.

The enforcement point must be pre-dispatch. That means checking the estimated token count against a configured budget before the API call fires, not after the response arrives. You cannot claw back tokens you have already consumed.

For systems that incorporate retrieval, the token problem gets harder. Injecting retrieved documents into a prompt is the right architectural move for accuracy, but each document chunk contributes to input token count. RAG systems need token-aware chunking at retrieval time, not just at prompt assembly time. If you are building retrieval into your Laravel application, the RAG and vector retrieval implementation guide covers the chunking and embedding strategy in detail.

Telemetry is the other half of cost control. Log prompt tokens, completion tokens, model, latency, and user or tenant identity on every response. Without that data, you cannot attribute cost, identify expensive call patterns, or make informed decisions about model selection.

Streaming vs Standard Responses

The choice between streaming and synchronous responses is a UX decision that has infrastructure consequences.

Synchronous calls are appropriate for background jobs, scheduled tasks, and any context where the result is written to storage before the user sees it. They are simpler to implement, easier to test, and do not require any special transport infrastructure. If your AI call feeds a Queue Worker writing to Eloquent, synchronous is the right default.

Streaming is appropriate when a human is waiting and the output is long enough that a full-response delay degrades the experience. The threshold in practice is around 1.5–2 seconds. Below that, streaming adds complexity for no perceivable gain.

The transport mechanism behind streaming is a separate decision. Server-Sent Events are the right default for Laravel, they require no additional infrastructure, work through standard HTTP, and integrate cleanly with Livewire. WebSockets are justified when you need bidirectional communication or multi-client broadcast. Livewire polling is a reasonable fallback for teams that want to avoid streaming infrastructure entirely, at the cost of perceived responsiveness.

The tradeoffs between these three approaches are significant enough to warrant their own analysis. Our streaming transport decision guide covers all three with production code and explicit recommendations for each scenario.

Choosing the Right Provider: A Decision Framework

Stop treating provider selection as a preference. Treat it as an engineering decision with clear inputs.

Use OpenAI when:

You need native embeddings alongside generation (text-embedding-3-small covers most retrieval use cases)
Your team needs the most widely documented integration path
You are building function-calling pipelines where tool reliability matters more than reasoning depth
You want the lowest overhead Laravel AI integration for an MVP that may evolve later

The complete OpenAI integration guide covers transport, function calling, and retry patterns against the GPT-4o and GPT-4o-mini endpoints.

Use Gemini when:

Your use case involves documents or sessions that exceed 128K tokens
You are processing multimodal inputs — documents, images, mixed media — at scale
You want to experiment with RAG-less patterns using Gemini’s long context as a retrieval proxy
Cost at volume is a primary concern (Gemini 2.0 Flash is competitive at high call rates)

For Gemini setup via the Laravel AI SDK, including the GeminiManager failover pattern and model string verification, that guide covers everything from provider registration to streaming response handling.

Use Claude when:

The task requires following multi-step instructions precisely and reliably
You are extracting structured output from unstructured text (JSON extraction, data normalisation)
Safety and refusal behaviour matter — regulated industries, content moderation pipelines
Code generation or review is a significant part of the workload

The Laravel Claude API integration guide covers the full Anthropic client setup, system prompt patterns, and streaming implementation with Laravel’s HTTP client.

Mixed-provider architectures

The interface pattern described earlier makes multi-provider setups practical. Routing by task type, OpenAI for embeddings, Claude for structured extraction, Gemini for long-document summarisation, is a legitimate production architecture, not over-engineering. The Service Container resolves the correct adapter per task type, and your domain code stays clean.

[Production Pitfall] Mixed-provider setups introduce a subtle budget tracking problem. If you log token costs per provider separately, you lose the aggregate view. Make sure your telemetry table records the provider name alongside every call, and your cost dashboards aggregate across providers by feature or tenant, not just by provider account.

Conclusion

If you take one thing from this guide, make it this: the providers are interchangeable. The architecture is not.

OpenAI, Gemini, and Claude will all improve, deprecate models, change pricing, and introduce capabilities over the next twelve months. Any of the three can be the right choice today and the wrong choice in two quarters. The teams that weather those changes without incident are the teams that built the abstraction layer before they needed it, not the teams that coupled their application to a specific SDK and are now rewriting feature logic to swap providers.

Production Laravel AI integration is a system design problem. The interface contract, the adapter layer, the token budget enforcement, the streaming transport decision, the telemetry pipeline — those are the decisions that determine whether your AI feature is operationally sustainable, or just a demo that scaled badly.

Build the layer. Then pick the provider.

Laravel OpenAI Integration: The Complete Production Guide

Dewald Hugo — Sun, 26 Apr 2026 20:59:38 +0000

Laravel OpenAI integration is not simpler than Claude integration. The APIs differ, the model families differ — but the architectural requirements are nearly identical once you care about production stability.

Laravel works well here because it already solves the problems AI integrations create: dependency boundaries, environment isolation, background work, failure handling, and observability. You’re not fighting the framework. You’re leaning on it.

On model names: This guide uses gpt-4o and gpt-4o-mini as examples. These are current as of early 2026. OpenAI’s model catalogue evolves quickly — always verify against OpenAI’s official models reference before deploying. gpt-3.5-turbo is deprecated; gpt-4o-mini is its cost-effective replacement.

Before committing to OpenAI as your sole provider, it is worth understanding where it sits in the broader Laravel AI provider landscape. Our provider comparison and architecture decision guide covers how OpenAI stacks up against Gemini and Claude for different production use cases — useful context before you build the service layer around any single provider.

Understanding the OpenAI API Surface

OpenAI’s platform has evolved faster than its documentation. Many tutorials conflate multiple API generations or mix incompatible approaches.

For Laravel applications, think in terms of three distinct capabilities: the Chat Completions API for text generation and reasoning, the Embeddings API for semantic search and retrieval, and the Audio/Vision APIs for transcription and multimodal inputs.

For image generation specifically – gpt-image-1, queued jobs, S3 storage, and per-request cost tracking — see the Laravel OpenAI image generation guide.

OpenAI is not “one API”. Your Laravel architecture should reflect that separation early. Controllers that call the Completions and Embeddings endpoints directly will become unmaintainable fast.

Foundations

Authentication and Environment Configuration

OpenAI uses API keys issued at the account or project level. Treat them as production secrets — same as database credentials or payment tokens.

OPENAI_API_KEY=sk-...
OPENAI_ORG_ID=optional

Never read these values directly from env() outside of configuration files. This is one of the most common structural mistakes we see in Laravel AI codebases. Expose them through a dedicated config file:

// config/openai.php
return [
    'api_key'      => env('OPENAI_API_KEY'),
    'organization' => env('OPENAI_ORG_ID'),
    'base_url'     => env('OPENAI_BASE_URL', 'https://api.openai.com/v1'),
];

This centralizes OpenAI configuration and allows overrides for testing, proxying, or self-hosted gateways. Skipping this step is how vendor logic leaks across the codebase and makes rotation painful.

Security: Never commit API keys to git. Use Laravel’s encrypted environment handling in production, and rotate keys periodically. Treat a leaked OpenAI key with the same urgency as a leaked database password — it has a direct billing impact.

Why You Should Not Use the OpenAI SDK Directly

OpenAI provides official SDKs, including PHP community implementations. Injecting these directly into controllers or jobs is a mistake.

SDKs change. Model names change. Response formats evolve. Your application should not absorb that churn.

Access OpenAI through a service boundary you control:

class OpenAiClient
{
    public function __construct(
        protected HttpClient $http
    ) {}
}

And then:

class OpenAiService
{
    public function __construct(
        protected OpenAiClient $client
    ) {}
}

This prevents your controllers from becoming vendor adapters. It also makes the Laravel Service Container work for you — you can bind a mock in tests without touching a single controller.

HTTP Client Configuration and Timeouts

AI requests can be long-running, bursty, expensive, and sensitive to retries. A default Http::post() is insufficient. Configure a dedicated client with explicit behaviour:

Http::withHeaders([
    'Authorization' => 'Bearer ' . config('openai.api_key'),
    'Content-Type'  => 'application/json',
])
->timeout(60)
->retry(0, 0);

The timeout(60) matters because OpenAI requests can legitimately take tens of seconds with large prompts or slow models. The retry(0, 0) matters even more — retrying AI requests blindly can double your cost and produce inconsistent outputs. If you need retries, implement them at the application level with full context awareness, not at the transport layer.

Error Handling as a First-Class Concern

OpenAI’s Chat Completions API fails in ways standard REST APIs rarely do: partial responses, rate limit throttling, model overload, malformed outputs despite 200 responses.

Your service layer must treat OpenAI responses as untrusted input, even when HTTP status codes indicate success.

$response = $this->client->post(...);

if (! $response->successful()) {
    throw new OpenAiRequestFailed(
        $response->status(),
        $response->body()
    );
}

But that’s only half the problem. You must also validate semantic correctness. Did the model return text when you expected JSON? Did it ignore system instructions? Did it hallucinate fields? That validation logic does not belong in a controller.

Service Architecture: Designing for Change

Your OpenAI service should not expose low-level API primitives unless absolutely necessary.

Instead of:

$openai->chatCompletion([...]);

Prefer intent-based methods:

$openai->generateText(...)
$openai->summarize(...)
$openai->extractStructuredData(...)

Internally, these methods may all use the same API endpoint. Externally, they express business intent. This gives you three architectural wins: prompts become versionable assets, you can A/B test models without touching callers, and you can migrate providers with minimal surface-area changes.

If you later introduce Claude alongside OpenAI — a decision many teams make as they want provider redundancy — this design pays for itself immediately. The service boundary is what makes multi-model routing tractable.

Dependency Injection and Testability

Every OpenAI-facing class should be injectable and mockable. No static helpers, no facades hiding HTTP calls, no env() reads at runtime.

In tests, you should be able to swap the OpenAI client with a fake implementation that returns deterministic responses. If you cannot do that without modifying production code, you are not integrating OpenAI — you are coupling your application to it.

Core Integration Patterns

Once your foundation is in place, OpenAI becomes a capability your application depends on. These patterns are not OpenAI-specific — they’re standard Laravel patterns applied to probabilistic systems.

Pattern 1: Simple Request–Response (One-Shot Tasks)

The simplest OpenAI interaction is still the most common: you send a prompt, you receive a response, the request lifecycle ends. Classification, extraction, summarization, content drafting — all one-shot.

Most tutorials implement this directly in controllers. Don’t.

Designing the Service Method

class OpenAiService
{
    public function generateText(string $prompt): string
    {
        // implementation
    }
}

Implementing the API Call

$response = $this->client->post('/chat/completions', [
    'model'    => 'gpt-4o',
    'messages' => [
        [
            'role'    => 'user',
            'content' => $prompt,
        ],
    ],
]);

Model selection is explicit. Never rely on defaults — model choice is an architectural decision with cost and quality implications.

Extracting the Result Safely

$data    = $response->json();
$content = data_get($data, 'choices.0.message.content');

if (! $content) {
    throw new UnexpectedOpenAiResponse($data);
}

return $content;

Use data_get() here. Future models may return multiple content blocks, safety filters may suppress output, and partial responses can occur under load. Your service method should either return valid output or fail loudly — silent nulls are debugging nightmares.

Pattern 2: Streaming Responses (Long-Running Output)

Streaming is not about novelty. It’s about user experience under latency. Large outputs can take seconds. Streaming turns that wait into visible progress.

When Streaming Makes Sense

Use streaming when output length is unbounded and perceived latency matters. Avoid it when you need structured JSON output, must validate the full response before use, or results are consumed only by background jobs.

Server-Side Streaming with OpenAI

OpenAI supports streaming via Server-Sent Events (SSE). On the Laravel side:

return response()->stream(function () use ($prompt) {
    $this->openAi->streamText($prompt, function ($chunk) {
        echo "data: " . json_encode($chunk) . "\n\n";
        ob_flush();
        flush();
    });
}, 200, [
    'Content-Type'     => 'text/event-stream',
    'Cache-Control'    => 'no-cache',
    'X-Accel-Buffering' => 'no',
]);

Implementing the Stream Method

⚠️ The sink Option Defeats Streaming Entirely

A common implementation mistake is using ->withOptions(['stream' => true, 'sink' => fopen('php://temp', 'w+')]) in Laravel’s HTTP client. The sink option tells Guzzle to buffer the entire response to that destination before returning. Your while (!feof($handle)) loop runs after the connection is already closed. You are not streaming — you are reading a completed buffer with extra steps. The UX benefits you expect don’t exist.

For true streaming, use Guzzle directly with stream => true and read the PSR-7 body incrementally:

public function streamText(string $prompt, callable $onChunk): void
{
    $client   = new \GuzzleHttp\Client();
    $response = $client->post('https://api.openai.com/v1/chat/completions', [
        'headers' => [
            'Authorization' => 'Bearer ' . config('openai.api_key'),
            'Content-Type'  => 'application/json',
        ],
        'json'   => [
            'model'    => 'gpt-4o',
            'messages' => [['role' => 'user', 'content' => $prompt]],
            'stream'   => true,
        ],
        'stream' => true,
    ]);

    $body = $response->getBody();
    $resource = $body->detach();
    while (! feof($resource)) {
        $line = rtrim(stream_get_line($resource, 4096, "\n"));

        if (empty($line) || $line === 'data: [DONE]') {
            continue;
        }

        if (str_starts_with($line, 'data: ')) {
            $data = json_decode(substr($line, 6), true);

            if (isset($data['choices'][0]['delta']['content'])) {
                $onChunk($data['choices'][0]['delta']['content']);
            }
        }
    }
}

Senior Dev Tip: If you need to decide between Livewire polling, SSE, and WebSockets for your AI UI, that architectural choice has more impact than the streaming implementation itself. We compared all three approaches in Livewire vs SSE vs WebSockets for AI UIs — worth reading before you commit.

Pattern 3: Conversation Memory (Multi-Turn Interactions)

Chatbots are one example. Multi-step reasoning, guided forms, iterative content refinement — all of these are multi-turn workflows. The key challenge is state management, not prompt engineering.

Representing Conversation State

Schema::create('conversations', function (Blueprint $table) {
    $table->id();
    $table->json('context')->nullable();
    $table->timestamps();
});

Schema::create('conversation_messages', function (Blueprint $table) {
    $table->id();
    $table->foreignId('conversation_id');
    $table->string('role'); // 'user' or 'assistant'
    $table->text('content');
    $table->timestamps();
});

Rehydrating Context

Before sending a request, load the full message history via Eloquent and build the message array:

$messages = $conversation->messages()
    ->orderBy('created_at')
    ->get()
    ->map(fn ($msg) => [
        'role'    => $msg->role,
        'content' => $msg->content,
    ])
    ->toArray();

Then append the new user input and send the full list to OpenAI. Large language models do not remember anything between API calls. Memory is entirely your responsibility. Persisting messages also lets you truncate context intelligently and summarize older turns when token counts grow large.

Pattern 4: Background Processing (Queued AI Work)

Not every AI task belongs in the request–response cycle. Bulk processing, document analysis, scheduled generation — these belong in queued jobs.

class GenerateSummaryJob implements ShouldQueue
{
    public function handle(OpenAiService $openAi): void
    {
        $summary = $openAi->summarize($this->document);

        $this->document->update(['summary' => $summary]);
    }
}

Laravel’s queue system provides retry semantics, failure hooks, and concurrency control — all essential when working with a rate-limited, cost-based API. A background job that silently retries three times on a large document can triple your bill. Jobs must cap token usage, log cost metadata, and fail deterministically.

Connecting the Patterns

These four patterns combine in real applications: a Livewire UI streams output, backed by a conversation record, which occasionally offloads heavy steps to queues, all of which reuse the same OpenAI service layer. That consistency is the real win — one abstraction, composable in every direction.

Building a Document Q&A Feature

Chatbots are not the most interesting application of LLMs in production. The real leverage comes from augmenting your existing data — documents, records, internal knowledge — with natural language access.

We’ll implement a Document Q&A feature: a user uploads a document, the system extracts and stores its content, and the user can ask questions grounded strictly in that document’s content. This exercises nearly every core pattern: one-shot generation, multi-turn context, background processing, and defensive prompt design.

We deliberately skip embeddings and vector databases here. That’s a scoped decision, not a shortcut. The simpler architecture works today and degrades predictably when it eventually outgrows itself.

Problem Definition and Constraints

What we want: natural language questions over uploaded documents, answers limited to document content, reasonable performance for mid-sized files, and clear failure modes.

What we’re not solving yet: semantic search across large corpora, cross-document reasoning, or high-recall retrieval at scale. Many “RAG tutorials” prematurely introduce vector databases when a simpler architecture would suffice.

High-Level Architecture

Four stages: ingestion (document uploaded and stored), extraction (text extracted and normalized), context preparation (content transformed into prompt context), and question answering (user questions answered via OpenAI). Each maps cleanly to a Laravel responsibility.

Stage 1: Document Upload and Persistence

Schema::create('documents', function (Blueprint $table) {
    $table->id();
    $table->string('original_name');
    $table->string('path');
    $table->longText('content')->nullable();
    $table->timestamps();
});

At upload time, do not call OpenAI:

public function store(Request $request): JsonResponse
{
    $file = $request->file('document');
    $path = $file->store('documents');

    $document = Document::create([
        'original_name' => $file->getClientOriginalName(),
        'path'          => $path,
    ]);

    ExtractDocumentContent::dispatch($document);

    return response()->json(['id' => $document->id]);
}

Dispatch a job. Extraction may be slow. Upload requests should return quickly. Extraction failures should be isolated — not cascade into upload failures.

Stage 2: Text Extraction as a Background Job

class ExtractDocumentContent implements ShouldQueue
{
    public function handle(): void
    {
        $text = $this->extractText(
            Storage::get($this->document->path)
        );

        $this->document->update([
            'content' => $this->normalize($text),
        ]);
    }
}

No AI calls, no tokens consumed. Failures are deterministic. You want extraction bugs to be boring.

Stage 3: Chunking Document Content

Here’s where many implementations fail. The naive approach — stuffing the entire document into a prompt — works briefly, then collapses under token limits and cost pressure.

Split the document into chunks based on size:

function chunkText(string $text, int $size = 800): array
{
    $chunks = [];
    $length = mb_strlen($text);
    $offset = 0;

    while ($offset < $length) {
        $chunks[] = mb_substr($text, $offset, $size);
        $offset  += $size;
    }

    return $chunks;
}

⚠️ Gotcha — str_split() Will Corrupt Multibyte Text

The PHP standard library’s str_split($text, $size) splits by bytes, not characters. Any document with UTF-8 content — French accents, German umlauts, Arabic, CJK — will produce chunks with broken characters at boundaries. Always use mb_substr() and mb_strlen() for text chunking. This is a silent production bug that only surfaces with real-world content.

Stage 4: Answering Questions

public function answerQuestion(Document $document, string $question): string
{
    $chunks = chunkText($document->content);
    $prompt = $this->buildPrompt($chunks, $question);

    return $this->openAi->generateText($prompt);
}

Prompt Construction

protected function buildPrompt(array $chunks, string $question): string
{
    $context = implode("\n\n", $chunks);

    return <<<PROMPT
You are answering questions based ONLY on the following document content.
If the answer cannot be found in the document, respond with:
"I don't know based on the provided document."

Document:
{$context}

Question:
{$question}
PROMPT;
}

This prompt constrains hallucination, defines a fallback response, and makes incorrect answers detectable. The goal is not perfection — it’s bounded failure. You’ll eventually outgrow this approach, but it fails gracefully, which makes it an excellent foundation.

Handling Multi-Turn Follow-Ups

Users rarely ask one question. Instead of re-sending the entire document every time, summarize prior context, append follow-ups as messages, and re-inject only relevant chunks. The conversation schema from Pattern 3 handles this directly.

Observability

Every OpenAI call in this feature should log document ID, token count, model, and latency. Not for analytics — for debugging. When a user reports “the AI gave a wrong answer,” you need the exact inputs that produced it. Without that log, you’re guessing.

Production Concerns

By the time an OpenAI integration reaches production, the question shifts from “Can we generate an answer?” to “How often does this fail? How expensive is it over time? What happens under load?”

AI integrations are probabilistic, rate-limited, and cost-based. Treating them like traditional REST APIs produces brittle systems.

Rate Limiting as a System Behaviour

OpenAI enforces rate limits at the account and model level. Most developers respond to sporadic 429s by adding retries. That’s the wrong mental model.

Rate limits are not an error condition. They’re a signal that your system’s demand exceeds its contract.

Centralizing Rate Awareness

Your OpenAI service layer should be the only place where rate-limit responses are interpreted:

if ($response->status() === 429) {
    throw new OpenAiRateLimited(
        retryAfter: (int) $response->header('Retry-After', 30)
    );
}

This exception should not be caught silently. Its consumers — controllers, jobs — must decide what to do next.

Different Strategies for Different Contexts

User-facing requests should fail fast with a clear message or degraded experience. Background jobs can be delayed and rescheduled:

public function handle(): void
{
    try {
        // OpenAI call
    } catch (OpenAiRateLimited $e) {
        $this->release($e->retryAfter ?? 30);
    }
}

This aligns retries with system capacity instead of guessing at backoff timing.

Cost as an Operational Metric

Token usage is not an abstract billing detail. It’s a runtime characteristic of your application. Two requests that look identical in code can differ by orders of magnitude in cost depending on prompt length, document size, conversation history, and model choice.

Recording Token Usage

Every OpenAI response includes token metadata. Capture it:

$usage = data_get($response->json(), 'usage');

OpenAiUsage::create([
    'model'             => $model,
    'prompt_tokens'     => $usage['prompt_tokens']     ?? 0,
    'completion_tokens' => $usage['completion_tokens'] ?? 0,
    'total_tokens'      => $usage['total_tokens']      ?? 0,
]);

This data is not for dashboards at first. It’s for answering questions like: Why did costs spike yesterday? Which feature is expensive? Which users generate the most load? You cannot optimize what you do not measure.

Cost-Aware Design Decisions

Once usage is visible, architectural decisions become concrete: summarizing older conversation turns to stay within token budgets, truncating document context for cheaper models, and routing simple tasks to gpt-4o-mini instead of gpt-4o. These are the difference between sustainable and runaway costs.

Caching AI Outputs (Carefully)

Caching AI responses is tempting. It’s also dangerous if done naively.

What Is Safe to Cache

Caching works well for deterministic prompts, non-user-specific inputs, and expensive but repeatable operations — document summaries, extracted metadata, classification labels:

Cache::remember(
    "doc:{$document->id}:summary",
    now()->addDay(),
    fn () => $openAi->summarize($document->content)
);

What Should Not Be Cached

Avoid caching conversational replies, user-personalized outputs, or anything derived from mutable state. If you cannot confidently answer “When should this be invalidated?”, don’t cache it.

Senior Dev Tip: Use Redis for AI output caching, not the file driver. Redis gives you atomic expiry, explicit invalidation by key pattern, and the ability to inspect what’s cached without reading from disk. File-based caches become unmanageable quickly when you’re caching per-document, per-user outputs.

Testing OpenAI Integrations Without Lying

This is where many teams get stuck. Mocking the OpenAI client to return a fixed string proves nothing useful. You need two distinct kinds of tests.

1. Contract Tests (Fast, Deterministic)

These verify that your service layer calls OpenAI correctly, that failures are handled, and that parsing logic works. Mocking is appropriate here:

Http::fake([
    'api.openai.com/*' => Http::response([
        'choices' => [
            ['message' => ['content' => 'Test output']],
        ],
        'usage' => [
            'prompt_tokens'     => 10,
            'completion_tokens' => 5,
            'total_tokens'      => 15,
        ],
    ]),
]);

You’re testing plumbing, not intelligence.

2. Behavioural Tests (Slow, Real)

These hit the real OpenAI API, use real prompts, and validate constraints — not specific wording:

$response = $service->answerQuestion($doc, 'What is the refund policy?');

$this->assertStringContainsString('refund', strtolower($response));

These tests should run infrequently, be clearly labelled, use dedicated API keys, and have cost limits. They catch failures no mock ever will.

Dealing With Non-Determinism

AI output is not stable across runs. Tests must reflect that. Assert structure, presence or absence, and refusal behaviour. If a model is instructed to say “I don’t know” under certain conditions, test for that branch explicitly. You’re testing behavioural boundaries, not string values.

Deployment Considerations

Increase PHP and proxy timeouts for streaming routes. Ensure queue workers can handle long-running jobs. If you haven’t locked down your production environment yet, deploy Laravel to production covers the infrastructure baseline these integrations depend on. Separate API keys across local, staging, and production — never reuse them. Wrap major prompt changes behind feature flags; a prompt change can alter behaviour as dramatically as a code change.

At minimum, you should be able to answer: Which OpenAI calls failed today? Which features are most expensive? Which prompts changed recently? This doesn’t require a full observability stack. It requires discipline: structured logs, request IDs, and consistent metadata.

Advanced Patterns

Your Laravel application now has a clean OpenAI service layer with request/response, streaming, memory, queues, cost tracking, rate limiting, testability, and observability. Let’s extend it.

Livewire Integration: Real-Time AI Without a Frontend Framework

Not every Laravel team wants a JavaScript SPA. Livewire allows server-driven interactivity that pairs naturally with AI workflows.

Livewire is request-driven, not socket-driven. Each interaction is an HTTP round trip, and long-running calls block the request lifecycle unless handled intentionally. For a complete working implementation, see Laravel Livewire + Claude Integration — the architectural patterns are identical for OpenAI.

Basic Livewire AI Component

use Livewire\Component;

class AiAssistant extends Component
{
    public string $prompt   = '';
    public string $response = '';
    public bool   $loading  = false;

    public function generate(OpenAiService $openAi): void
    {
        $this->loading  = true;
        $this->response = $openAi->generateText($this->prompt);
        $this->loading  = false;
    }

    public function render()
    {
        return view('livewire.ai-assistant');
    }
}

This works immediately but it’s blocking. If generation takes 10 seconds, the request takes 10 seconds. For short responses, acceptable. For longer outputs, you have three realistic strategies:

Polling: Trigger generation in a queued job, persist partial output, and poll from Livewire until complete. Simple and resilient.
Hybrid Route for Streaming: Livewire initiates the request, a separate streaming endpoint handles chunked output, and a thin JavaScript layer listens and updates the DOM.
Accept Blocking: For short, predictable outputs, it may be entirely acceptable.

The decision depends on latency tolerance, not preference.

Inertia + Vue: SPA-Style AI Interfaces

If your application already uses Inertia, you can leverage SSE natively on the client side.

Laravel Route:

Route::get('/ai/stream', function (Request $request, OpenAiService $openAi) {
    return response()->stream(function () use ($request, $openAi) {
        $openAi->streamText(
            $request->query('prompt'),
            function ($chunk) {
                echo "data: " . json_encode($chunk) . "\n\n";
                ob_flush();
                flush();
            }
        );
    }, 200, [
        'Content-Type'      => 'text/event-stream',
        'Cache-Control'     => 'no-cache',
        'X-Accel-Buffering' => 'no',
    ]);
});

Vue Client:

const eventSource = new EventSource(`/ai/stream?prompt=${encodeURIComponent(prompt)}`);

eventSource.onmessage = (event) => {
    response.value += JSON.parse(event.data);
};

Streaming is an architectural decision, not a cosmetic enhancement. It changes how users perceive your application’s responsiveness.

Retrieval-Augmented Generation (RAG) Fundamentals

Eventually, chunking entire documents into prompts stops scaling. You need selective context retrieval.

RAG solves this by converting text into embeddings, storing them in a vector store, and injecting only semantically relevant chunks into the prompt at query time — rather than the full document.

Step 1: Generating Embeddings

$response = $this->client->post('/embeddings', [
    'model' => 'text-embedding-3-small',
    'input' => $textChunk,
]);

$vector = data_get($response->json(), 'data.0.embedding');

Step 2: Storing Vectors

At small scale, PostgreSQL with the pgvector extension is sufficient. If you need a deeper introduction to how embeddings actually work before choosing your storage strategy, Embeddings and Vector Databases: The Foundation of Semantic AI Systems covers the concepts and storage trade-offs in detail.

Step 3: Retrieving Relevant Chunks

At query time, convert the question to an embedding, perform a cosine similarity search, and retrieve the top N chunks. Only those chunks enter the OpenAI prompt. This reduces token usage, latency, and hallucination. RAG is not magic — it’s controlled context injection.

Multi-Model Strategies

Assuming one model should do everything is one of the most expensive architectural mistakes in LLM-backed applications. Large reasoning models are slow and costly. Smaller models are fast and cheap. Embedding models serve an entirely different purpose.

Intent-Based Model Routing

$model = match ($taskType) {
    'summarization'    => 'gpt-4o-mini',
    'complex_reasoning' => 'gpt-4o',
    'embedding'        => 'text-embedding-3-small',
    default            => 'gpt-4o-mini',
};

This prevents overpaying for simple tasks. A document classification job does not need the same model as a multi-step reasoning pipeline.

Fallback Strategies

You can also implement primary → fallback model routing, high-accuracy → fast mode switching, and user-tier-based model selection. These are business decisions with direct cost and SLA implications, not technical hacks.

Senior Dev Tip: If you want to go further and treat your AI layer as a genuinely provider-agnostic abstraction — with tool calling, multi-model orchestration, and structured outputs built in — take a look at Building Agentic Laravel Apps with Prism PHP. Prism handles the abstraction layer so you’re not maintaining it yourself.

Where This Leaves You

Your Laravel + OpenAI system now supports real-time interfaces, retrieval-based document reasoning, multi-model routing, and a structured architecture ready to scale.

The service boundary protects you from API changes, model shifts, and provider swaps. OpenAI becomes infrastructure — replaceable, measurable, and contained. That’s the outcome worth building toward.

Next Steps

Ready to build with real conversation memory and streaming? Building a Claude API Chatbot in Laravel demonstrates both patterns in a complete, working implementation.

Hit rate limits in production? The queue-based throttling and backpressure strategies in Laravel Claude API Integration Guide apply directly to OpenAI workloads.

Want a server-driven real-time UI without JavaScript frameworks? Laravel Livewire + Claude Integration shows how to build AI features that stay in PHP.

Questions? Ask in our Developer Q&A or reach out directly.

Official References:

RAG-less Architecture in Laravel: Long-Context Caching with Gemini

Dewald Hugo — Thu, 23 Apr 2026 03:01:13 +0000

For the past few years, RAG has been presented as the only serious approach to grounding AI on large private datasets. Build a vector database, chunk your documents, embed the chunks, retrieve the closest matches, inject them into the prompt. Repeat for every query. It works, and for certain use cases it is genuinely the right call.

But for others — particularly repeated, deep-analysis tasks on the same corpus — RAG is an expensive workaround for a context window problem that the models have largely outgrown.

Google’s Gemini 2.5 Flash and 2.5 Pro both ship with a 1-million-token context window. That is approximately 750,000 words, or a mid-sized Laravel monolith with room to spare. More importantly, the Gemini API now supports Explicit Context Caching: you upload that corpus once, pay full input price once, and reference the cached context on every subsequent query at up to 75% off the standard input rate. For workloads that involve many questions against the same codebase, document set, or knowledge base, this is a fundamentally different cost model — and a fundamentally different architecture.

This guide walks you through building a GeminiContextCacheService in Laravel 11/12, binding it to the Service Container, and building a production-ready audit endpoint around it. We will cover the full cache lifecycle, including the storage cost trap that most tutorials skip entirely.

RAG vs. Long-Context Caching with Gemini: The Architectural Trade-Off

Before touching code, we need to be honest about what each approach actually is. Neither is universally superior.

RAG trades context completeness for cost predictability. You pay only for the retrieved chunks, not the full corpus — which is excellent when your dataset is enormous (hundreds of GB) and any single query only needs a narrow slice of it. The problem is that retrieval is fundamentally lossy. A semantic search might pull the right controller method and miss the BaseModel that overrides save(). It will find the public interface and skip the private helper that silently transforms the input. That retrieval gap is not a bug you can patch; it is the architecture.

Long-context caching with Gemini trades retrieval precision for corpus completeness. The model sees everything. There is no retrieval step, no chunking pipeline, no embedding model to maintain. The trade-off is cost structure: the first query is expensive (you pay to ingest the full context), and you carry a per-hour storage fee while the cache lives. Beyond the break-even point — which arrives faster than you expect — repeated queries cost significantly less than the RAG equivalent.

The Cost Curve: Why Caching Is the Financial Lever

Let us put real numbers on this. Using Gemini 2.5 Flash, which offers the best price-to-context ratio for this workload:

Standard input rate (long context >128K tokens): $0.30/MTok
Cached input rate: $0.03/MTok (90% reduction at Flash long-context tier)
Cache storage: billed per hour per MTok while the cache lives

For a 500K-token codebase audit with 20 questions:

Without caching: Each query sends 500K tokens to the API. That is $0.15 per question × 20 = $3.00 total.

With caching: Initial cache creation: $0.15 (one-time). Each subsequent query references the cache at $0.015. Twenty queries: $0.15 + ($0.015 × 19) = $0.435 total — plus a small storage fee for however long the cache lives.

The break-even is query 2. Every question after that is 90% cheaper than the RAG alternative that re-sends the same context.

When RAG Still Wins

We should be direct about this before anyone rips out a working vector pipeline.

If your corpus exceeds 1M tokens — a multi-year ticket archive, a large documentation set, a multi-repo monorepo — the long-context window is not a solution. You physically cannot fit the data. RAG is the architecture. Our existing guide on Laravel Embeddings, Vector Databases, and RAG covers pgvector and the full embedding pipeline if that is your situation.

Similarly, if your corpus changes frequently — user-generated content, live database records — cache invalidation becomes expensive. Re-creating a 1M-token cache on every data change defeats the cost benefit.

Long-context caching targets a specific sweet spot: a corpus that is large enough to matter (above RAG’s retrieval quality threshold), small enough to fit in 1M tokens, and stable enough that you will run many queries before the next update. B2B codebase auditing, legal document analysis, product catalog deep-dives — these are the use cases.

Implementation: The Stateful Expert Pattern

Prerequisites

Add your Gemini API key to .env:

GEMINI_API_KEY=your_key_here
GEMINI_MODEL=gemini-2.5-flash
GEMINI_CACHE_TTL=7200

Then register the configuration values in config/services.php:

'gemini' => [
    'api_key'   => env('GEMINI_API_KEY'),
    'base_url'  => 'https://generativelanguage.googleapis.com/v1beta',
    'model'     => env('GEMINI_MODEL', 'gemini-2.5-flash'),
    'cache_ttl' => (int) env('GEMINI_CACHE_TTL', 7200),
],

The minimum content size for Gemini context caching is 32,768 tokens — roughly 25,000 words. You cannot cache a handful of files. You need to be pushing a full module, a full codebase, or a substantial document set. Keep that constraint in mind when designing your corpus preparation step.

If you have not yet set up your Gemini client or worked through model selection on your Laravel stack, our Integrating Gemini into the Laravel AI SDK guide covers the foundational setup decisions before you start hitting the API directly.

Step 1: The Context Caching Service

This service owns two responsibilities: cache lifecycle (create, retrieve, delete) and query dispatch. We keep them in one class because the cache name is required for query — splitting them would force unnecessary dependency injection.

<?php

namespace App\Services;

use Illuminate\Support\Facades\Cache;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Log;
use RuntimeException;

class GeminiContextCacheService
{
    private string $apiKey;
    private string $baseUrl;
    private string $model;
    private int    $ttl;

    public function __construct()
    {
        $this->apiKey  = config('services.gemini.api_key');
        $this->baseUrl = config('services.gemini.base_url');
        $this->model   = config('services.gemini.model');
        $this->ttl     = config('services.gemini.cache_ttl');
    }

    /**
     * Return an existing live cache name from Redis, or create a new one.
     * The cache key is your application's identifier (e.g. "audit:project-42").
     * The Gemini cache name is what the API issues back — it looks like
     * "cachedContents/abc123xyz" and is what you reference in generation requests.
     */
    public function getOrCreateCache(
        string $cacheKey,
        string $systemPrompt,
        array  $documents
    ): string {
        $geminiCacheName = Cache::get("gemini_cache:{$cacheKey}");

        if ($geminiCacheName) {
            Log::debug('Gemini cache hit', ['key' => $cacheKey]);
            return $geminiCacheName;
        }

        return $this->createCache($cacheKey, $systemPrompt, $documents);
    }

    private function createCache(
        string $cacheKey,
        string $systemPrompt,
        array  $documents
    ): string {
        // Each document becomes a separate part in the cached content
        $parts = array_map(
            fn(string $doc) => ['text' => $doc],
            $documents
        );

        $response = Http::withHeaders([
            'x-goog-api-key' => $this->apiKey,
            'Content-Type'   => 'application/json',
        ])->post("{$this->baseUrl}/cachedContents", [
            'model'            => "models/{$this->model}",
            'displayName'      => $cacheKey,
            'systemInstruction' => [
                'parts' => [['text' => $systemPrompt]],
            ],
            'contents' => [
                ['role' => 'user', 'parts' => $parts],
            ],
            'ttl' => "{$this->ttl}s",
        ]);

        if ($response->failed()) {
            Log::error('Gemini cache creation failed', [
                'status' => $response->status(),
                'body'   => $response->json(),
                'key'    => $cacheKey,
            ]);

            throw new RuntimeException(
                'Failed to create Gemini context cache: ' . $response->body()
            );
        }

        $cacheName = $response->json('name');

        // Store with a 60-second buffer before the Gemini TTL expires
        // so we never reference a cache the API has already deleted
        Cache::put(
            "gemini_cache:{$cacheKey}",
            $cacheName,
            $this->ttl - 60
        );

        Log::info('Gemini context cache created', [
            'name' => $cacheName,
            'key'  => $cacheKey,
        ]);

        return $cacheName;
    }

    /**
     * Run a user question against an existing cached context.
     */
    public function query(string $cacheName, string $question): string
    {
        $response = Http::withHeaders([
            'x-goog-api-key' => $this->apiKey,
            'Content-Type'   => 'application/json',
        ])->post("{$this->baseUrl}/models/{$this->model}:generateContent", [
            'cachedContent' => $cacheName,
            'contents'      => [
                ['role' => 'user', 'parts' => [['text' => $question]]],
            ],
        ]);

        if ($response->failed()) {
            if ($response->status() === 429) {
                throw new RuntimeException(
                    'Gemini rate limit exceeded. Implement exponential backoff before retrying.'
                );
            }

            Log::error('Gemini query failed', [
                'status' => $response->status(),
                'cache'  => $cacheName,
            ]);

            throw new RuntimeException('Gemini query failed: ' . $response->body());
        }

        $text = $response->json('candidates.0.content.parts.0.text');

        if ($text === null) {
            throw new RuntimeException(
                'Unexpected Gemini response structure: ' . $response->body()
            );
        }

        return $text;
    }

    /**
     * Explicitly delete a cache when the session ends.
     * Do not leave caches running — storage costs accumulate per hour.
     */
    public function deleteCache(string $cacheKey): void
    {
        $geminiCacheName = Cache::get("gemini_cache:{$cacheKey}");

        if (! $geminiCacheName) {
            return;
        }

        Http::withHeaders(['x-goog-api-key' => $this->apiKey])
            ->delete("{$this->baseUrl}/{$geminiCacheName}");

        Cache::forget("gemini_cache:{$cacheKey}");

        Log::info('Gemini context cache deleted', ['key' => $cacheKey]);
    }
}

[Architect’s Note] The 60-second TTL buffer on the Redis key is not optional. If your Laravel cache expiry aligns exactly with Gemini’s TTL, a query that arrives in the final seconds will find a valid Redis key but hit a deleted Gemini cache. The API returns a 404, your query() method throws, and your user gets a 503. Build the buffer in. If you are concerned about token governance and want to log every token consumed by cached vs. standard queries separately, our Laravel AI Middleware: Token Tracking & Rate Limiting guide covers the instrumentation layer you would wrap around this service.

Step 2: Binding to the Service Container

Register the service as a singleton in AppServiceProvider. Because the service reads from config in its constructor, a singleton is correct here — we want one instance per request cycle, not one per injection.

<?php

namespace App\Providers;

use App\Services\GeminiContextCacheService;
use Illuminate\Support\ServiceProvider;

class AppServiceProvider extends ServiceProvider
{
    public function register(): void
    {
        $this->app->singleton(GeminiContextCacheService::class);
    }
}

Laravel’s Service Container handles the rest. Any controller, job, or command that type-hints GeminiContextCacheService receives the same instance within a given request.

Step 3: The Audit Controller

<?php

namespace App\Http\Controllers;

use App\Services\GeminiContextCacheService;
use Illuminate\Http\JsonResponse;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Log;
use RuntimeException;

class CodeAuditController extends Controller
{
    public function __construct(
        private readonly GeminiContextCacheService $gemini
    ) {}

    public function audit(Request $request): JsonResponse
    {
        $validated = $request->validate([
            'project_id' => ['required', 'string', 'max:100'],
            'question'   => ['required', 'string', 'max:2000'],
        ]);

        try {
            $documents = $this->loadProjectDocuments($validated['project_id']);

            $systemPrompt  = 'You are a senior Laravel architect reviewing a production codebase. '
                . 'When answering questions, cite specific files and line numbers. '
                . 'Flag security risks, legacy patterns, and architectural concerns explicitly.';

            $cacheName = $this->gemini->getOrCreateCache(
                cacheKey:     "audit:{$validated['project_id']}",
                systemPrompt: $systemPrompt,
                documents:    $documents,
            );

            $answer = $this->gemini->query($cacheName, $validated['question']);

            return response()->json(['answer' => $answer]);

        } catch (RuntimeException $e) {
            Log::error('Code audit failed', [
                'project_id' => $validated['project_id'],
                'error'      => $e->getMessage(),
            ]);

            return response()->json(
                ['error' => 'Audit service temporarily unavailable.'],
                503
            );
        }
    }

    private function loadProjectDocuments(string $projectId): array
    {
        $path = storage_path("app/projects/{$projectId}/codebase.txt");

        if (! file_exists($path)) {
            throw new RuntimeException(
                "Codebase not found for project: {$projectId}"
            );
        }

        // Return as array — each element becomes a separate Part in the cached content.
        // Split large codebases into logical chunks (per-module) if approaching 1M tokens.
        return [file_get_contents($path)];
    }
}

Register the route with Sanctum auth and a conservative throttle. Twenty questions per minute is generous for a codebase audit endpoint — adjust to match your billing ceiling.

// routes/api.php
use App\Http\Controllers\CodeAuditController;

Route::middleware(['auth:sanctum', 'throttle:20,1'])
    ->post('/audit', [CodeAuditController::class, 'audit']);

Securing this endpoint properly is not optional — an unauthenticated audit endpoint is a direct vector for API cost abuse. Our Laravel Sanctum API Authentication guide covers token scoping and rate-limit configuration if you need to tighten this down further.

Cache Lifecycle Management

This is the section most tutorials omit. Skip it and you will find $800 in unexpected Gemini storage charges at the end of the month.

Gemini charges per MTok per hour for every active cache. The pricing varies by model — check Google’s official API pricing page for current rates. The pattern you want is: create the cache when the audit session starts, delete it when the session ends, and run a scheduled cleanup job to catch anything that leaked through.

The cleanup command:

<?php

namespace App\Console\Commands;

use Carbon\Carbon;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Log;

class CleanExpiredGeminiCaches extends Command
{
    protected $signature   = 'gemini:clean-caches';
    protected $description = 'Delete expired or stale Gemini context caches';

    public function handle(): int
    {
        $apiKey  = config('services.gemini.api_key');
        $baseUrl = config('services.gemini.base_url');

        $response = Http::withHeaders(['x-goog-api-key' => $apiKey])
            ->get("{$baseUrl}/cachedContents");

        if ($response->failed()) {
            $this->error('Failed to list caches: ' . $response->body());
            return self::FAILURE;
        }

        $caches = $response->json('cachedContents', []);

        foreach ($caches as $cache) {
            $expireTime = Carbon::parse($cache['expireTime']);

            if ($expireTime->isPast()) {
                Http::withHeaders(['x-goog-api-key' => $apiKey])
                    ->delete("{$baseUrl}/{$cache['name']}");

                Log::info('Deleted stale Gemini cache', ['name' => $cache['name']]);
                $this->info("Deleted: {$cache['name']}");
            }
        }

        return self::SUCCESS;
    }
}

Schedule it in bootstrap/app.php:

->withSchedule(function (Schedule $schedule) {
    $schedule->command('gemini:clean-caches')->hourly();
})

[Edge Case Alert] If a queue worker crashes mid-audit before deleteCache() is called, the Gemini cache lives until its TTL expires and storage charges accumulate. The scheduled cleanup command is your safety net, not a replacement for explicit deletion. Consider also dispatching a DeleteGeminiCacheJob via defer() at the end of each audit session as a belt-and-suspenders measure.

Use Case: The Persistent Auditor in Practice

Here is where the architecture earns its keep. You are auditing a legacy Laravel 6 application for a client. The codebase is 400K tokens — controllers, models, migrations, the lot.

The RAG approach: You ask about UserController::store(). The retrieval pulls three relevant chunks. The answer looks correct. But the client’s BaseModel has an overridden save() method that intercepts every Eloquent write and transforms certain fields. RAG did not retrieve BaseModel — it was not semantically close enough to the query. The AI gives you an answer that describes the surface behaviour and misses the hack underneath. The audit is wrong.

The cached-context approach: You upload the full codebase. When you ask about UserController::store(), Gemini has already read BaseModel. It answers that the apparent save logic is intercepted downstream, identifies the override, and flags it as a maintenance risk. The context is complete. The answer is complete.

This is what we mean by Architectural Integrity. RAG optimises for retrieval cost. Long-context caching optimises for answer correctness. For an audit product where a wrong answer has professional liability implications, correctness wins.

The cost math, using our earlier figures: 20 audit questions on a 500K-token codebase costs $3.00 without caching and approximately $0.44 with caching — plus storage for however long the session runs. For a B2B audit priced at $200 per report, the infrastructure cost is negligible either way. But the quality difference is not.

For production agentic pipelines — where the “questions” are not human prompts but tool calls from an orchestrated agent — the cost advantage compounds quickly. Our guide on Hardening Laravel Agentic Workflows: Schema Validation Against LLM Hallucinations covers the validation layer you will want around those agent outputs, regardless of which AI architecture you choose.

Preparing Your Corpus for the Cache

The code examples above load a pre-processed codebase.txt file. In production, you need a pipeline that generates this file from your actual source. A simple Artisan command handles it:

<?php

namespace App\Console\Commands;

use Illuminate\Console\Command;
use Illuminate\Support\Facades\File;
use Symfony\Component\Finder\Finder;

class PrepareCodebaseCorpus extends Command
{
    protected $signature   = 'corpus:prepare {project_id} {path}';
    protected $description = 'Concatenate a project codebase into a single corpus file';

    public function handle(): int
    {
        $projectId = $this->argument('project_id');
        $path      = $this->argument('path');

        $finder = (new Finder())
            ->files()
            ->in($path)
            ->name(['*.php', '*.json', '*.yaml', '*.env.example'])
            ->notPath(['vendor', 'node_modules', 'storage', '.git'])
            ->sortByName();

        $corpus  = '';
        $counter = 0;

        foreach ($finder as $file) {
            $relativePath = str_replace($path . '/', '', $file->getRealPath());
            $corpus .= "\n\n// FILE: {$relativePath}\n";
            $corpus .= $file->getContents();
            $counter++;
        }

        $outputDir  = storage_path("app/projects/{$projectId}");
        $outputPath = $outputDir . '/codebase.txt';

        File::ensureDirectoryExists($outputDir);
        File::put($outputPath, $corpus);

        $sizeKb = round(strlen($corpus) / 1024, 1);
        $this->info("Corpus prepared: {$counter} files, {$sizeKb} KB → {$outputPath}");

        return self::SUCCESS;
    }
}

Run it before starting an audit session: php artisan corpus:prepare project-42 /path/to/legacy-app. The file headers (// FILE: app/Models/BaseModel.php) give Gemini the file path context it needs to cite locations in its answers.

[Efficiency Gain] Strip comments and docblocks from PHP files before creating the corpus. A typical Laravel application loses 20–30% of its token count after comment stripping, which can push a borderline 1.2M-token codebase under the 1M limit and save meaningfully on cache creation costs. PHP’s token_get_all() makes this straightforward; a simple token filter removes T_COMMENT and T_DOC_COMMENT tokens before concatenation.

Conclusion: Context Is the New Database

RAG will not disappear. For certain corpus shapes — enormous, dynamic, multi-tenant — it remains the correct architecture. But it has been treated as the default answer for too long, applied reflexively to problems that long-context models can now solve more cleanly and more accurately.

Explicit context caching in Gemini changes the trade-off. The vector database, the chunking strategy, the embedding model, the retrieval tuning — for a well-bounded corpus, all of that overhead collapses into a single cached upload and a TTL you manage in Redis. The model sees the whole codebase. The answers reflect the whole codebase.

The architecture we have built here — GeminiContextCacheService bound as a singleton, TTL-buffered cache keys in Redis, explicit lifecycle management, a corpus preparation command — is production-ready. Wire up your Sanctum-protected audit endpoint, run the corpus preparation pipeline, and you have a stateful expert that knows your codebase better than any retrieval pipeline can.

For official context caching documentation and current pricing, refer directly to Google’s Gemini API Context Caching guide — pricing has shifted frequently in 2026 and the source of truth is Google’s pricing page, not any third-party aggregator.

Integrating Gemini into the Laravel AI SDK

Dewald Hugo — Thu, 16 Apr 2026 13:45:05 +0000

If your Laravel application already speaks to OpenAI or Anthropic through the first-party laravel/ai SDK, adding Gemini feels like it should be a five-minute job. Set a new API key. Point to a new provider. Deploy. Done. That assumption is mostly correct, but the gap between “mostly” and “done” is where production systems fall apart. This guide closes that gap.

We will cover the full Laravel Gemini integration path: the OpenAI-compatible soft entry that lets you validate the connection in under ten minutes, the native Gemini provider configuration that actually belongs in a long-lived codebase, and a GeminiManager service pattern that gives you model failover, rate-limit awareness, and a response contract your Blade templates never have to think about.

By the end, Gemini will be a first-class citizen in your AI stack, not an afterthought bolted to a controller.

Why Add Gemini At All?

Fair question. If you are already running claude-sonnet-4-6 or gpt-4o in production, the case for a third provider needs to be more compelling than “Google exists.” Here is the honest pitch.

Gemini 2.5 Flash is fast, cheap, and ships with a one-million-token context window as standard. For high-throughput tasks—document summarisation, classification pipelines, RAG over large corpora—it undercuts the competition on cost without giving up much capability. Gemini 2.5 Pro sits at the other end: serious reasoning, multimodal by default, and Vertex AI integration if you are already inside the Google Cloud ecosystem.

The second reason is resilience. Single-provider AI architectures have a hard failure mode. When OpenAI rate-limits your 2 AM batch job, the entire feature goes dark. Gemini as a fallback provider changes that calculation. We are not talking about a hot standby you never test; we are talking about a properly designed failover that routes intelligently.

The third reason is the laravel/ai SDK itself. As covered in detail in What Laravel 13 Actually Changes for AI Development, the SDK was designed from the start to be provider-agnostic. Gemini is a first-class supported provider. The architecture already expects you to run multiple models.

Route 1: The OpenAI-Compatible Endpoint (The Soft Entry)

Before committing to the native integration, you can validate that Gemini works for your use case using Google’s OpenAI-compatible endpoint. This is genuinely useful for a quick proof-of-concept, or when you are migrating from an existing openai-php/client setup rather than the laravel/ai SDK.

Google maintains an endpoint at https://generativelanguage.googleapis.com/v1beta/openai/ that accepts Chat Completions API requests in the standard OpenAI format. The swap is mechanical: change the base URL, swap the API key, and change the model string.

Start with your .env:

GEMINI_API_KEY=your-google-ai-studio-key
GEMINI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/
GEMINI_MODEL=gemini-2.5-flash

If you are on an existing OpenAI configuration in config/ai.php, you can add a second connection:

// config/ai.php
'connections' => [
    'openai' => [
        'driver'  => 'openai',
        'api_key' => env('OPENAI_API_KEY'),
    ],

    'gemini_compat' => [
        'driver'    => 'openai',           // still using the OpenAI driver
        'api_key'   => env('GEMINI_API_KEY'),
        'base_url'  => env('GEMINI_BASE_URL'),
    ],
],

You can now call Gemini through any code that already uses the openai driver, just by swapping the connection name:

use Illuminate\Support\Facades\AI;

$response = AI::connection('gemini_compat')->text(
    model: 'gemini-2.5-flash',
    messages: [
        ['role' => 'user', 'content' => $prompt],
    ]
);

This works well enough for a smoke test. Resist the urge to ship it as your production integration.

[Production Pitfall] The OpenAI-compatible endpoint has real coverage gaps. Gemini-native features—grounding via Google Search, the thinking budget parameter, native multimodal tool use, and fine-grained safety settings—are not available through the compat layer. If your feature depends on any of those, the compat approach will silently drop them and return a degraded response with no error. You will not find out until QA, or worse, production.

Route 2: The Native Gemini Provider (The Right Way)

The laravel/ai SDK ships with a first-party Gemini driver. Use it. The compat approach is a migration crutch; the native driver is what belongs in your config/ai.php long-term.

Install the SDK if you have not already:

composer require laravel/ai
php artisan vendor:publish --provider="Laravel\Ai\AiServiceProvider"
php artisan migrate

Then register the native Gemini connection:

// config/ai.php
'connections' => [
    'openai' => [
        'driver'  => 'openai',
        'api_key' => env('OPENAI_API_KEY'),
    ],

    'anthropic' => [
        'driver'  => 'anthropic',
        'api_key' => env('ANTHROPIC_API_KEY'),
    ],

    'gemini' => [
        'driver'  => 'gemini',
        'api_key' => env('GEMINI_API_KEY'),
    ],
],

'default' => env('AI_DEFAULT_CONNECTION', 'openai'),

Your .env stays clean:

GEMINI_API_KEY=your-google-ai-studio-key

A basic request through the native driver looks identical to your existing OpenAI calls:

use Illuminate\Support\Facades\AI;

$response = AI::connection('gemini')->text(
    model: 'gemini-2.5-flash',
    messages: [
        ['role' => 'user', 'content' => $prompt],
    ]
);

return $response->text;

That’s the surface. Now let’s build something that actually runs in production.

Model Selection: Which Gemini Model?

Not all Gemini models are created equal, and Google’s naming conventions have become genuinely confusing over the last year. Here is the practical breakdown for production use.

gemini-2.5-flash is your default. It hit general availability in June 2025, it is fast, it is cost-efficient, and it handles up to one million tokens of context. For most text generation, summarisation, and classification tasks, it is the correct choice.

gemini-2.5-pro is for heavy lifting. Complex reasoning chains, code generation over large codebases, long-form synthesis tasks. It is meaningfully more expensive than Flash and slower under load, so do not reach for it by default.

[Edge Case Alert] Do not use gemini-2.0-flash in new integrations. Google has announced that Gemini 2.0 Flash and Flash-Lite models will be shut down on June 1, 2026. Any production system pointing at those model strings will start throwing 404 errors at that date with no warning beyond the deprecation announcement. If you have any existing references to gemini-2.0-flash in your codebase, change them now.

For preview models in the Gemini 3.x family, treat them exactly as you would treat preview Claude models: useful for internal testing, not for customer-facing production paths until they have a stable designation.

The GeminiManager Pattern

A single AI connection configured in config/ai.php is fine for prototyping. Production systems need something more deliberate: model selection logic, rate-limit handling, failover between providers, and a response contract that the rest of your application can depend on regardless of which model actually ran.

The GeminiManager service class is that layer. It sits between the laravel/ai facade and your application code, owns the failover logic, and returns a typed response object your Blade templates or API controllers can render without knowing which provider answered the request.

Start by creating the service in app/Services/AI/:

<?php

namespace App\Services\AI;

use Illuminate\Support\Facades\AI;
use Illuminate\Support\Facades\Log;
use App\DTOs\AiResponse;

class GeminiManager
{
    private array $modelPriority = [
        'gemini-2.5-flash',
        'gemini-2.5-pro',
    ];

    private string $fallbackConnection = 'openai';
    private string $fallbackModel      = 'gpt-4o-mini';

    public function generate(string $prompt, array $options = []): AiResponse
    {
        foreach ($this->modelPriority as $model) {
            try {
                $response = $this->attemptGemini($prompt, $model, $options);

                return new AiResponse(
                    content:    $response->text,
                    model:      $model,
                    provider:   'gemini',
                    inputTokens:  $response->usage?->inputTokens ?? 0,
                    outputTokens: $response->usage?->outputTokens ?? 0,
                );

            } catch (\Exception $e) {
                Log::warning('Gemini model failed', [
                    'model'   => $model,
                    'error'   => $e->getMessage(),
                    'prompt_length' => strlen($prompt),
                ]);

                continue;
            }
        }

        // All Gemini models exhausted. Fall back to configured provider.
        return $this->fallback($prompt, $options);
    }

    private function attemptGemini(string $prompt, string $model, array $options): mixed
    {
        return AI::connection('gemini')->text(
            model: $model,
            messages: [['role' => 'user', 'content' => $prompt]],
            ...$options
        );
    }

    private function fallback(string $prompt, array $options): AiResponse
    {
        Log::error('All Gemini models exhausted. Falling back.', [
            'fallback_connection' => $this->fallbackConnection,
            'fallback_model'      => $this->fallbackModel,
        ]);

        $response = AI::connection($this->fallbackConnection)->text(
            model: $this->fallbackModel,
            messages: [['role' => 'user', 'content' => $prompt]],
        );

        return new AiResponse(
            content:     $response->text,
            model:       $this->fallbackModel,
            provider:    $this->fallbackConnection,
            inputTokens:  $response->usage?->inputTokens ?? 0,
            outputTokens: $response->usage?->outputTokens ?? 0,
        );
    }
}

The AiResponse DTO keeps your response contract stable:

<?php

namespace App\DTOs;

readonly class AiResponse
{
    public function __construct(
        public string $content,
        public string $model,
        public string $provider,
        public int    $inputTokens,
        public int    $outputTokens,
    ) {}
}

Bind it in your bootstrap/app.php, using Laravel 11/12 syntax:

use App\Services\AI\GeminiManager;

->withProviders([
    function () {
        app()->singleton(GeminiManager::class);
    },
])

[Architect’s Note] This pattern mirrors exactly what we described in Production-Grade AI Architecture in Laravel: Contracts, Governance & Telemetry—the response DTO is your contract, and nothing outside the service layer should care which provider produced the response. The moment your Blade template starts conditionally rendering based on $response->provider, you have an architecture problem, not a template problem.

Rate Limit Handling

The GeminiManager above catches all exceptions and continues to the next model. That is fine for a first pass but too broad for production. You want to distinguish between rate-limit errors, which should trigger immediate failover, and authentication or configuration errors, which should not silently retry.

Gemini rate-limit responses return HTTP 429. A tighter catch block:

use Illuminate\Http\Client\RequestException;

private function attemptGemini(string $prompt, string $model, array $options): mixed
{
    try {
        return AI::connection('gemini')->text(
            model: $model,
            messages: [['role' => 'user', 'content' => $prompt]],
            ...$options
        );

    } catch (RequestException $e) {
        $status = $e->response->status();

        if ($status === 429) {
            // Rate limited. Try next model.
            throw $e;
        }

        if ($status === 401 || $status === 403) {
            // Auth failure. Do not retry silently.
            Log::critical('Gemini authentication failure', ['status' => $status]);
            throw new \RuntimeException('Gemini authentication failed. Check GEMINI_API_KEY.');
        }

        // Anything else: surface it.
        throw $e;
    }
}

For consistent rate-limit and token tracking across all your AI providers, the middleware pattern we detailed in Laravel AI Middleware: Token Tracking & Rate Limiting plugs directly into this architecture. Wire it as HTTP middleware on your AI routes and your AiResponse DTO’s token counts feed it automatically.

Keeping Your Blade Templates Clean

This is the part most tutorials ignore. It is also where the design falls apart in real applications.

Your Blade templates should receive an AiResponse object and render $response->content. That is it. They should not know the provider is Gemini, they should not branch on $response->model, and they should never contain try/catch logic. If your failover logic is working correctly, the view never knows a failover happened.

In a controller:

use App\Services\AI\GeminiManager;

class ArticleSummaryController extends Controller
{
    public function __construct(
        private readonly GeminiManager $ai
    ) {}

    public function show(Article $article): View
    {
        $summary = $this->ai->generate(
            prompt: "Summarise this article in three sentences: {$article->body}"
        );

        return view('articles.show', [
            'article' => $article,
            'summary' => $summary,
        ]);
    }
}

The Blade template:

<div class="summary">
    {{ $summary->content }}
</div>

Done. No provider logic. No model names. No error handling. The controller injects the service, the service handles everything messy, and the view renders a string.

[Word to the Wise] The discipline to keep AI provider logic out of your views pays off when you swap models six months from now. And you will swap models. Gemini 2.5 Flash will eventually give way to whatever Google releases next. If your template has @if ($summary->model === 'gemini-2.5-flash') buried inside it, that migration becomes a grep exercise across your entire view layer. Design the abstraction correctly now.

Registering Gemini as a Queued Job Provider

Synchronous AI calls in web requests are a bad idea at scale. Wrap your GeminiManager calls in a queued job for anything that is not a real-time user interaction:

<?php

namespace App\Jobs;

use App\Services\AI\GeminiManager;
use App\Models\Article;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class GenerateArticleSummary implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public int $tries    = 3;
    public int $backoff  = 30; // seconds

    public function __construct(
        private readonly int $articleId
    ) {}

    public function handle(GeminiManager $ai): void
    {
        $article = Article::findOrFail($this->articleId);

        $summary = $ai->generate(
            prompt: "Summarise this article in three sentences: {$article->body}"
        );

        $article->update([
            'summary'          => $summary->content,
            'summary_model'    => $summary->model,
            'summary_provider' => $summary->provider,
        ]);
    }

    public function failed(\Throwable $e): void
    {
        Log::error('Article summary generation failed', [
            'article_id' => $this->articleId,
            'error'      => $e->getMessage(),
        ]);
    }
}

The $tries = 3 and $backoff = 30 give you three attempts with a 30-second gap between them, which is usually enough to ride out a transient Gemini rate limit without wasting queue resources. The failed hook gives you an audit trail.

Official Documentation

Two references worth bookmarking as you build this:

Google Gemini OpenAI Compatibility Documentation — the canonical reference for what is and is not supported through the compat layer, including current model strings and endpoint coverage.
Laravel AI SDK Documentation — the official introduction covering provider setup, agent workflows, and the provider-agnostic design philosophy.

Production Deployment Checklist

A few things that catch teams off-guard when they move this from staging to production.

Google AI Studio API keys are rate-limited by default. For production workloads, you need to request increased quota through Google Cloud, or switch to Vertex AI. The Vertex AI endpoint requires a different authentication mechanism (service account credentials rather than a plain API key) and a slightly different base URL, so plan for that migration if your volume grows.

Environment-specific model configuration matters. You might run gemini-2.5-flash in production for cost reasons and gemini-2.5-pro in staging for testing purposes. Keep that in your .env and out of your GeminiManager:

# .env.production
GEMINI_MODEL=gemini-2.5-flash

# .env.staging
GEMINI_MODEL=gemini-2.5-pro

Read it in the service:

private array $modelPriority = [];

public function __construct()
{
    $this->modelPriority = [
        config('services.gemini.model', 'gemini-2.5-flash'),
        'gemini-2.5-pro',    // fallback within Gemini
    ];
}

And add to config/services.php:

'gemini' => [
    'model' => env('GEMINI_MODEL', 'gemini-2.5-flash'),
],

If you are deploying to Laravel Forge or a similar managed environment, confirm your queue workers restart after deployment. A worker that caches the old GeminiManager configuration will ignore model changes until it restarts. This is not Gemini-specific, but it catches teams every time they change provider configuration without restarting workers.

[Efficiency Gain] Gemini 2.5 Flash’s one-million-token context window is genuinely useful for RAG pipelines. If you are building document search or retrieval-augmented generation, the ability to stuff a much larger context window means fewer chunking edge cases and simpler retrieval logic. If you are already running embeddings and vector search in Laravel, the integration pattern described in Laravel Embeddings, Vector Databases, and RAG: A Production Implementation Guide maps directly to Gemini’s native multimodal context handling.

Where to Go From Here

The GeminiManager pattern above is a solid foundation. The logical next step is versioning your prompts so that model upgrades do not introduce silent regressions—something that becomes critical the moment you start comparing output quality between gemini-2.5-flash and whatever Gemini 3.x stable lands in your production window. The Prompt Migrations: Bringing Determinism to AI in Laravel tutorial covers exactly that workflow.

The Google AI ecosystem moves fast. gemini-3.1-flash-lite-preview is already in the preview tier as of this writing, and Google is actively iterating. The architecture we have built here isolates model selection to a single config value and a single service class. When the next stable Gemini model lands, your migration is a one-line change.

Laravel Filament Admin Dashboard for AI Applications: Token Costs, Prompt Management, and Agent Audit Trails

Dewald Hugo — Thu, 16 Apr 2026 03:34:17 +0000

You have the AI pipeline. You have the service classes, the middleware, the queued jobs. The backend is instrumented. What you do not have is any way to look at it without writing a SQL query at 2am because costs just spiked.

That is the gap this article closes.

A Laravel Filament admin dashboard is not the glamorous part of AI development. But the teams that skip it are the ones who discover they have been over-spending on tokens for six weeks, or that an agent ran a destructive tool call nobody authorised, or that a prompt change from last Tuesday quietly broke output quality across the board. You cannot govern what you cannot see.

The Laravel Filament admin dashboard built in this guide surfaces four things: real-time token cost monitoring, usage analytics broken down by user and model, a prompt version management panel, and a full agent audit trail. Each section is a Filament v3 Resource or Widget, built on top of the telemetry data your existing middleware and service layer already generate.

Why Filament, and Why Now

There is no serious competition in the Laravel ecosystem for building admin interfaces right now. Filament v3 ships with a full panel system, Livewire-powered tables, stats widgets, chart widgets, custom pages, and a role-based authorization layer. It works inside your existing Laravel application. It does not require a separate process or a Node frontend.

The Livewire-driven rendering keeps things reactive without JavaScript complexity. The Eloquent-backed Resources mean your existing models map directly to admin panels with almost no boilerplate. And because it is all Laravel under the hood, your Service Container, policies, events, and observers all still apply — you are not learning a new paradigm, just adding a UI layer on top of architecture you have already built.

Filament v3 requires PHP 8.1+ and Laravel 10+. On Laravel 11 or 12 you will be fine.

Installation and Panel Setup

Install Filament with the panel scaffolding command:

composer require filament/filament:"^3.0" -W
php artisan filament:install --panels

This generates app/Providers/Filament/AdminPanelProvider.php and registers it via bootstrap/app.php. The panel provider is where you wire discovery, colours, middleware, and authentication. Here is a clean starting configuration:

// app/Providers/Filament/AdminPanelProvider.php

namespace App\Providers\Filament;

use Filament\Http\Middleware\Authenticate;
use Filament\Http\Middleware\DisableBladeIconComponents;
use Filament\Http\Middleware\DispatchServingFilamentEvent;
use Filament\Panel;
use Filament\PanelProvider;
use Filament\Support\Colors\Color;
use Illuminate\Cookie\Middleware\AddQueuedCookiesToResponse;
use Illuminate\Cookie\Middleware\EncryptCookies;
use Illuminate\Foundation\Http\Middleware\VerifyCsrfToken;
use Illuminate\Routing\Middleware\SubstituteBindings;
use Illuminate\Session\Middleware\AuthenticateSession;
use Illuminate\Session\Middleware\StartSession;
use Illuminate\View\Middleware\ShareErrorsFromSession;

class AdminPanelProvider extends PanelProvider
{
    public function panel(Panel $panel): Panel
    {
        return $panel
            ->default()
            ->id('admin')
            ->path('admin')
            ->login()
            ->colors(['primary' => Color::Violet])
            ->discoverResources(
                in: app_path('Filament/Resources'),
                for: 'App\\Filament\\Resources'
            )
            ->discoverPages(
                in: app_path('Filament/Pages'),
                for: 'App\\Filament\\Pages'
            )
            ->discoverWidgets(
                in: app_path('Filament/Widgets'),
                for: 'App\\Filament\\Widgets'
            )
            ->middleware([
                EncryptCookies::class,
                AddQueuedCookiesToResponse::class,
                StartSession::class,
                AuthenticateSession::class,
                ShareErrorsFromSession::class,
                VerifyCsrfToken::class,
                SubstituteBindings::class,
                DisableBladeIconComponents::class,
                DispatchServingFilamentEvent::class,
            ])
            ->authMiddleware([
                Authenticate::class,
            ])
            ->authorization(fn () => auth()->user()?->is_admin ?? false);
    }
}

That last ->authorization() call is the line most tutorials omit. Without it, any authenticated user can reach your admin panel. We will come back to access control later — but set that guard from day one.

Modeling the Data Your Dashboard Will Surface

Before you build a single Filament Resource, you need to decide what data you are collecting. On a production AI application, there are two primary log types worth persisting.

AI Usage Logs capture every inference call: which provider, which model, how many tokens, what it cost, which feature triggered it, and whether it succeeded.

Agent Audit Logs capture agentic actions: which agent class ran, what action it performed, which tool it called, what the input and output were, and how long it took.

Both should be Eloquent models backed by indexed tables. Start with the usage log migration:

// database/migrations/2026_04_15_000001_create_ai_usage_logs_table.php

Schema::create('ai_usage_logs', function (Blueprint $table) {
    $table->id();
    $table->foreignId('user_id')->nullable()->constrained()->nullOnDelete();
    $table->string('provider');           // 'anthropic', 'openai'
    $table->string('model');
    $table->string('feature')->nullable(); // 'chat', 'summarize', 'agent'
    $table->integer('prompt_tokens');
    $table->integer('completion_tokens');
    $table->integer('total_tokens');
    $table->decimal('cost_usd', 10, 6)->nullable();
    $table->string('prompt_version')->nullable();
    $table->json('metadata')->nullable();
    $table->boolean('success')->default(true);
    $table->text('error_message')->nullable();
    $table->timestamps();

    $table->index(['user_id', 'created_at']);
    $table->index(['provider', 'model', 'created_at']);
    $table->index(['feature', 'created_at']);
});

The compound indexes on created_at are non-negotiable. Dashboard widgets run aggregation queries constantly, and without them you will feel the difference under real traffic.

The agent audit log has a slightly different shape:

// database/migrations/2026_04_15_000002_create_agent_audit_logs_table.php

Schema::create('agent_audit_logs', function (Blueprint $table) {
    $table->id();
    $table->foreignId('user_id')->nullable()->constrained()->nullOnDelete();
    $table->string('agent_class');
    $table->string('action');
    $table->string('tool_used')->nullable();
    $table->json('input_payload')->nullable();
    $table->json('output_payload')->nullable();
    $table->integer('tokens_used')->default(0);
    $table->float('duration_ms')->nullable();
    $table->boolean('success')->default(true);
    $table->text('error')->nullable();
    $table->timestamps();

    $table->index(['agent_class', 'created_at']);
    $table->index(['user_id', 'created_at']);
    $table->index(['success', 'created_at']);
});

Both models should cast their JSON columns and use standard Eloquent relationships back to User. Nothing exotic required.

Token Cost Monitoring: The Stats Overview Widget

The first thing an operator needs to see when they open the dashboard is a top-level summary of what the AI layer has cost in the last 24 hours. Filament’s StatsOverviewWidget handles this cleanly.

php artisan make:filament-widget AiStatsOverview --stats-overview

// app/Filament/Widgets/AiStatsOverview.php

namespace App\Filament\Widgets;

use App\Models\AiUsageLog;
use Filament\Widgets\StatsOverviewWidget as BaseWidget;
use Filament\Widgets\StatsOverviewWidget\Stat;

class AiStatsOverview extends BaseWidget
{
    protected static ?int $sort = 1;

    protected function getStats(): array
    {
        $today  = AiUsageLog::whereDate('created_at', today());
        $month  = AiUsageLog::whereMonth('created_at', now()->month)
                            ->whereYear('created_at', now()->year);

        return [
            Stat::make("Today's Tokens", number_format($today->sum('total_tokens')))
                ->description('Across all providers and models')
                ->color('primary'),

            Stat::make("Today's Cost", '$' . number_format($today->sum('cost_usd'), 4))
                ->description('USD — live tally')
                ->color('warning'),

            Stat::make('Monthly Spend', '$' . number_format($month->sum('cost_usd'), 2))
                ->description(now()->format('F Y'))
                ->color('danger'),

            Stat::make('Failures Today', number_format($today->where('success', false)->count()))
                ->description('Inference errors across all features')
                ->color('gray'),
        ];
    }
}

Register this widget on your dashboard in the panel provider’s ->widgets() array. It polls on a configurable interval using Livewire’s polling mechanism — add protected static ?string $pollingInterval = '30s'; if you want it to refresh automatically.

If you have not yet built the middleware layer that feeds these logs, the Laravel AI Middleware guide on token tracking and rate limiting covers the exact pattern for intercepting inference calls and persisting usage data before the response reaches the controller.

Daily Token Usage: The Chart Widget

Raw totals are useful. Trend lines are what actually surface problems — a cost spike on Tuesday, a failure rate that spiked after a deployment. Add a chart widget to give operators a 30-day view:

php artisan make:filament-widget TokenUsageChart --chart

// app/Filament/Widgets/TokenUsageChart.php

namespace App\Filament\Widgets;

use App\Models\AiUsageLog;
use Filament\Widgets\ChartWidget;

class TokenUsageChart extends ChartWidget
{
    protected static ?string $heading = 'Daily Token Usage — Last 30 Days';
    protected static ?int $sort = 2;
    public ?string $filter = 'tokens';

    protected function getFilters(): ?array
    {
        return [
            'tokens' => 'Total Tokens',
            'cost'   => 'Cost (USD)',
        ];
    }

    protected function getData(): array
    {
        $column = $this->filter === 'cost' ? 'cost_usd' : 'total_tokens';
        $label  = $this->filter === 'cost' ? 'Cost (USD)' : 'Total Tokens';

        $data = AiUsageLog::selectRaw("DATE(created_at) as date, SUM({$column}) as value")
            ->where('created_at', '>=', now()->subDays(30))
            ->groupBy('date')
            ->orderBy('date')
            ->get();

        return [
            'datasets' => [
                [
                    'label'       => $label,
                    'data'        => $data->pluck('value')->toArray(),
                    'borderColor' => '#8b5cf6',
                    'tension'     => 0.3,
                    'fill'        => false,
                ],
            ],
            'labels' => $data->pluck('date')->toArray(),
        ];
    }

    protected function getType(): string
    {
        return 'line';
    }
}

The $filter toggle lets operators switch between token volume and cost in a single widget, which keeps the dashboard clean without multiplying panels.

The Usage Log Resource: Per-User and Per-Model Breakdown

Stats and charts give you the summary. When you need to investigate — which user ran 400,000 tokens in one session, which model is responsible for the cost spike — you need a sortable, filterable table.

php artisan make:filament-resource AiUsageLog --generate

Then replace the generated table definition with something that has real operator value:

// app/Filament/Resources/AiUsageLogResource.php

namespace App\Filament\Resources;

use App\Filament\Resources\AiUsageLogResource\Pages;
use App\Models\AiUsageLog;
use Filament\Resources\Resource;
use Filament\Tables;
use Filament\Tables\Table;
use Filament\Tables\Columns\IconColumn;
use Filament\Tables\Columns\TextColumn;
use Filament\Tables\Filters\SelectFilter;
use Filament\Tables\Filters\Filter;
use Illuminate\Database\Eloquent\Builder;

class AiUsageLogResource extends Resource
{
    protected static ?string $model           = AiUsageLog::class;
    protected static ?string $navigationIcon  = 'heroicon-o-chart-bar';
    protected static ?string $navigationGroup = 'AI Governance';
    protected static ?string $navigationLabel = 'Usage Logs';

    public static function table(Table $table): Table
    {
        return $table
            ->columns([
                TextColumn::make('user.name')
                    ->label('User')
                    ->searchable()
                    ->sortable()
                    ->placeholder('—'),

                TextColumn::make('provider')
                    ->badge()
                    ->sortable(),

                TextColumn::make('model')
                    ->searchable()
                    ->copyable(),

                TextColumn::make('feature')
                    ->badge()
                    ->color('info')
                    ->placeholder('—'),

                TextColumn::make('total_tokens')
                    ->numeric()
                    ->sortable()
                    ->summarize(Tables\Columns\Summarizers\Sum::make()->label('Total')),

                TextColumn::make('cost_usd')
                    ->money('usd')
                    ->sortable()
                    ->summarize(Tables\Columns\Summarizers\Sum::make()->label('Total')),

                TextColumn::make('prompt_version')
                    ->badge()
                    ->color('warning')
                    ->placeholder('—'),

                IconColumn::make('success')
                    ->boolean(),

                TextColumn::make('created_at')
                    ->dateTime()
                    ->sortable()
                    ->since()
                    ->toggleable(),
            ])
            ->filters([
                SelectFilter::make('provider')
                    ->options([
                        'anthropic' => 'Anthropic',
                        'openai'    => 'OpenAI',
                    ]),

                SelectFilter::make('success')
                    ->options([
                        '1' => 'Successful',
                        '0' => 'Failed',
                    ]),

                Filter::make('today')
                    ->label('Today only')
                    ->query(fn (Builder $query) => $query->whereDate('created_at', today())),
            ])
            ->defaultSort('created_at', 'desc')
            ->paginated([25, 50, 100]);
    }

    public static function getPages(): array
    {
        return [
            'index' => Pages\ListAiUsageLogs::route('/'),
        ];
    }

    public static function canCreate(): bool
    {
        return false;
    }
}

Two things to note here. First, ->summarize() on the token and cost columns gives you running totals in the table footer — incredibly useful when you have applied filters to isolate a user or a date range. Second, canCreate(): false disables the “New record” button. Usage logs are system-generated. Nobody should be hand-entering them.

Production-Grade AI Architecture in Laravel: Contracts, Governance & Telemetry details the telemetry layer that populates this data. If your AI services are not already emitting structured usage events to an observable logging interface, that article is the right place to start before wiring up this dashboard.

Prompt Version Management

Operators need to know which prompt is active. They also need to see when it changed, what the previous version said, and whether cost or quality shifted after a change. This is where the dashboard and your version control system connect.

Prompt Migrations: Bringing Determinism to AI in Laravel covers building the underlying prompt migration system. Assuming you have a prompt_versions table from that guide, the Filament Resource surfaces it with minimal effort:

php artisan make:filament-resource PromptVersion --generate

// app/Filament/Resources/PromptVersionResource.php

namespace App\Filament\Resources;

use App\Filament\Resources\PromptVersionResource\Pages;
use App\Models\PromptVersion;
use Filament\Forms\Components\Textarea;
use Filament\Forms\Components\TextInput;
use Filament\Forms\Components\Toggle;
use Filament\Forms\Form;
use Filament\Resources\Resource;
use Filament\Tables;
use Filament\Tables\Table;
use Filament\Tables\Columns\IconColumn;
use Filament\Tables\Columns\TextColumn;

class PromptVersionResource extends Resource
{
    protected static ?string $model           = PromptVersion::class;
    protected static ?string $navigationIcon  = 'heroicon-o-document-text';
    protected static ?string $navigationGroup = 'AI Governance';
    protected static ?string $navigationLabel = 'Prompt Versions';

    public static function form(Form $form): Form
    {
        return $form
            ->schema([
                TextInput::make('key')->required()->disabled(),
                TextInput::make('version')->required()->disabled(),
                Textarea::make('system_prompt')->rows(8)->columnSpanFull(),
                Textarea::make('description')->rows(3)->columnSpanFull(),
                Toggle::make('is_active')->label('Active')->disabled(),
            ]);
    }

    public static function table(Table $table): Table
    {
        return $table
            ->columns([
                TextColumn::make('key')->searchable()->sortable(),
                TextColumn::make('version')->badge()->color('primary'),
                TextColumn::make('description')->limit(60)->placeholder('—'),
                IconColumn::make('is_active')->boolean()->label('Active'),
                TextColumn::make('created_at')->dateTime()->sortable(),
            ])
            ->defaultSort('created_at', 'desc');
    }

    public static function getPages(): array
    {
        return [
            'index' => Pages\ListPromptVersions::route('/'),
            'view'  => Pages\ViewPromptVersion::route('/{record}'),
        ];
    }

    public static function canCreate(): bool { return false; }
    public static function canEdit($record): bool { return false; }
    public static function canDelete($record): bool { return false; }
}

All three can* methods return false. Prompt changes belong in migrations under version control. The admin panel gives you read access only. This is intentional. The moment you allow operators to edit prompt content via a UI form, you have created an unversioned, un-reviewable, un-rollbackable change path that will cause you pain.

[Architect’s Note] Keep your admin panel read-only for prompt content. The dashboard is a governance tool, not a content editor. Every prompt mutation should flow through a migration, be committed to version control, and be deployable like code. If your team is asking for a UI to edit prompts live, that is a process problem masquerading as a tooling request. Fix the process.

Agent Audit Trails

This is the section most teams skip. It is also the one that saves them when something goes wrong.

Agentic workflows make decisions. They call tools. They read data and write data. When a user complains that the agent deleted something it should not have, or an integration breaks because the agent started formatting its output differently, you need a record of exactly what happened.

If you are building agentic pipelines, the Hardening Laravel Agentic Workflows guide covers schema validation and output hardening. The audit trail built here is the operational companion to that — it answers “what did it do?” as opposed to “did it do it correctly?”

Emit an audit log from your base agent class on every action:

// app/AI/Agents/BaseAgent.php

namespace App\AI\Agents;

use App\Models\AgentAuditLog;

abstract class BaseAgent
{
    protected function logAction(
        string $action,
        array $input,
        array $output,
        ?string $toolUsed = null,
        int $tokensUsed = 0,
        float $durationMs = 0.0,
        bool $success = true,
        ?string $error = null
    ): void {
        AgentAuditLog::create([
            'user_id'        => auth()->id(),
            'agent_class'    => static::class,
            'action'         => $action,
            'tool_used'      => $toolUsed,
            'input_payload'  => $input,
            'output_payload' => $output,
            'tokens_used'    => $tokensUsed,
            'duration_ms'    => $durationMs,
            'success'        => $success,
            'error'          => $error,
        ]);
    }
}

The Filament Resource for the audit log is where the detail view earns its keep. Operators need to be able to click into a single agent run and see the full input/output payload:

// app/Filament/Resources/AgentAuditLogResource.php

namespace App\Filament\Resources;

use App\Filament\Resources\AgentAuditLogResource\Pages;
use App\Models\AgentAuditLog;
use Filament\Infolists\Components\KeyValueEntry;
use Filament\Infolists\Components\Section;
use Filament\Infolists\Components\TextEntry;
use Filament\Infolists\Infolist;
use Filament\Resources\Resource;
use Filament\Tables;
use Filament\Tables\Table;
use Filament\Tables\Columns\IconColumn;
use Filament\Tables\Columns\TextColumn;
use Filament\Tables\Filters\SelectFilter;

class AgentAuditLogResource extends Resource
{
    protected static ?string $model           = AgentAuditLog::class;
    protected static ?string $navigationIcon  = 'heroicon-o-cpu-chip';
    protected static ?string $navigationGroup = 'AI Governance';
    protected static ?string $navigationLabel = 'Agent Audit Trail';

    public static function table(Table $table): Table
    {
        return $table
            ->columns([
                TextColumn::make('user.name')->label('User')->searchable()->placeholder('System'),
                TextColumn::make('agent_class')->badge()->searchable(),
                TextColumn::make('action')->searchable(),
                TextColumn::make('tool_used')->badge()->color('info')->placeholder('—'),
                TextColumn::make('tokens_used')->numeric()->sortable(),
                TextColumn::make('duration_ms')->label('ms')->numeric()->sortable(),
                IconColumn::make('success')->boolean(),
                TextColumn::make('created_at')->dateTime()->sortable()->since(),
            ])
            ->filters([
                SelectFilter::make('success')->options(['1' => 'Successful', '0' => 'Failed']),
                SelectFilter::make('agent_class')->relationship('agentClass', 'agent_class'),
            ])
            ->defaultSort('created_at', 'desc')
            ->recordUrl(fn ($record) => static::getUrl('view', ['record' => $record]));
    }

    public static function infolist(Infolist $infolist): Infolist
    {
        return $infolist
            ->schema([
                Section::make('Run Details')
                    ->columns(2)
                    ->schema([
                        TextEntry::make('agent_class')->badge(),
                        TextEntry::make('action'),
                        TextEntry::make('tool_used')->badge()->color('info')->placeholder('None'),
                        TextEntry::make('tokens_used')->numeric(),
                        TextEntry::make('duration_ms')->label('Duration (ms)')->numeric(),
                        TextEntry::make('success')->badge()->getStateUsing(
                            fn ($record) => $record->success ? 'Success' : 'Failed'
                        )->color(fn ($state) => $state === 'Success' ? 'success' : 'danger'),
                    ]),

                Section::make('Input Payload')
                    ->schema([
                        KeyValueEntry::make('input_payload')->label(''),
                    ]),

                Section::make('Output Payload')
                    ->schema([
                        KeyValueEntry::make('output_payload')->label(''),
                    ]),

                Section::make('Error')
                    ->schema([TextEntry::make('error')->placeholder('None')])
                    ->visible(fn ($record) => !$record->success),
            ]);
    }

    public static function getPages(): array
    {
        return [
            'index' => Pages\ListAgentAuditLogs::route('/'),
            'view'  => Pages\ViewAgentAuditLog::route('/{record}'),
        ];
    }

    public static function canCreate(): bool { return false; }
    public static function canEdit($record): bool { return false; }
}

The Infolist view is the right choice here over a form, because you are displaying data, not editing it. The KeyValueEntry component renders JSON payloads cleanly without any custom Blade work.

Locking Down the Admin Panel

Your Filament admin is a high-value target. It contains cost data, user data, prompt intellectual property, and agent decision history. Basic access control is not optional.

The ->authorization() callback in your panel provider is your first line of defence. The example earlier used is_admin on the User model. In practice, most teams reach for Spatie’s Laravel Permission package for this. Your panel provider call then becomes:

->authorization(fn () => auth()->user()?->hasRole('admin') ?? false)

For more granular control over individual Resources, use Laravel Policies. Filament automatically discovers and respects a policy for any model that has one. A AiUsageLogPolicy with viewAny returning $user->hasRole('admin') gives you fine-grained control without putting access logic inside the Resource class itself.

One pattern worth enforcing: scope your AI admin panel to a separate admin guard, and ensure it sits behind your Sanctum token authentication if your application issues API tokens to admin users. The Laravel Sanctum API Authentication guide covers token scoping and ability verification, and the same principles apply when you are protecting admin panel access programmatically.

Do not expose the Filament panel under the default /admin path in production. Change the path in your panel provider:

->path('ops-internal') // or any non-obvious string

This is not security through obscurity as a replacement for authentication. It is a minor but free win against automated scanning.

Deploying Filament Alongside Your AI Stack

Filament adds Livewire, Alpine.js, and its own asset pipeline to your application. Running php artisan filament:assets publishes vendor assets to your public/ directory. Make sure your deployment script runs this command after every Filament update — if you skip it, operators will see unstyled panels or JavaScript errors after a version bump.

php artisan filament:assets
php artisan filament:optimize  # Filament v3.1+

filament:optimize caches component discovery. On applications with many Resources and Widgets, it makes a measurable difference to panel load time.

If your application uses Redis for caching, you may want to cache the stats queries powering your overview widgets. Filament’s widgets do not cache their data by default. For a stats widget that runs four SUM() aggregations on every page load, wrapping them in cache()->remember() with a 60-second TTL is worth doing before you go to production.

The complete Laravel production deployment guide covers the full deployment pipeline in detail. When you add Filament, extend the deploy script to include filament:assets and filament:optimize alongside your standard migrate, config:cache, and queue:restart steps. Missing any of these on a Filament upgrade is a common source of post-deploy surprises.

Organising the Navigation: The AI Governance Group

With three or more Filament Resources all related to AI observability, you want them grouped in the sidebar. Every Resource in this article already carries protected static ?string $navigationGroup = 'AI Governance';. That single property is enough to group them under a collapsible sidebar section.

You can control group ordering in the panel provider:

->navigationGroups([
    'AI Governance',
    'User Management',
    'System',
])

Groups listed here appear in that order in the sidebar. Groups not listed appear at the bottom. Keep AI Governance at the top — it is why operators are opening this panel.

What to Cache, What to Queue

Two performance considerations you will hit once real traffic arrives.

The stats overview widget runs database queries on every page render. If 10 admins have this dashboard open, that is 10 concurrent aggregation queries on your ai_usage_logs table. Cache the results. Override getStats() to use cache()->remember('ai_stats_today', 60, fn() => ...) for each query group.

Bulk insert your usage logs via a queued job rather than synchronously inside the middleware response cycle. A single queued insert dispatched after the AI response returns adds zero latency to the user-facing request and prevents your ai_usage_logs table from becoming a write bottleneck under load. This is especially relevant if you are logging agent audit trails — agent runs that call multiple tools can generate a burst of log writes in a short window.

[Production Pitfall] If you are writing AiUsageLog::create() synchronously inside an AI middleware or a service class, you will see that call become a bottleneck under sustained load. A single slow database write can add 20–50ms to every AI response. Dispatch a queued job instead. Your usage logs do not need to be written before the response returns — they need to be written eventually and reliably.

Extending the Dashboard Over Time

This guide builds the foundation. Here is what comes next.

Budget alerts. Add an Eloquent Observer on AiUsageLog that triggers a notification when daily spend crosses a configurable threshold. Filament’s database notifications work well here — the panel will show the alert in the notification bell without any additional UI work.

Per-feature cost attribution. The feature column on ai_usage_logs lets you break cost down by product feature. A second chart widget comparing cost by feature over time gives product teams the data they need to make informed decisions about which AI features are worth their current token spend.

Prompt performance correlation. Once you have both usage logs (with prompt_version) and an output quality signal — whether that is a user rating, a downstream validation pass/fail, or a schema validation score from your agentic pipeline — you can join them in a custom Filament page and show quality versus cost per prompt version. That is a genuinely differentiated internal tool that most teams never build.

Model comparison. If you are testing multiple models for the same feature (Claude versus GPT-4o, for example), the model column combined with a quality signal gives you a cost-per-quality-point comparison you can act on. That decision gets much easier when you can see it in a table rather than inferring it from CloudWatch logs.

Refer to the official Filament v3 documentation for the full API surface on widgets, infolists, and panel configuration. The patterns in this guide are production-tested starting points, not exhaustive implementations.

Laravel Sanctum API Authentication: The Complete Production Guide

Dewald Hugo — Thu, 09 Apr 2026 22:06:40 +0000

There's a quiet assumption baked into almost every Laravel AI integration tutorial: authentication exists. Routes are protected. Tokens are issued. The API is locked down.

That assumption breaks the moment you sit down to build something real.

Laravel Sanctum is the framework's answer to lightweight API token authentication. It ships with Laravel, it integrates cleanly with Eloquent, and it handles the two most common authentication patterns - SPA cookie-based sessions and mobile/external API token issuance, without pulling in a full OAuth server. This guide covers both patterns, but it leans hard into the personal access token model, because that's what you need when you're building an API that your own frontend, mobile app, or third-party client will consume.

By the end, you'll have a production-ready authentication layer: token issuance with ability scoping, protected routes, revocation endpoints, rate limiting via Redis, and a multi-tenant token pattern that holds up under real load. We're also covering the Laravel 11/12 bootstrap/app.php configuration style throughout, no legacy Kernel.php references.

Let's get into it.

What Sanctum Actually Does

Before writing a single line of code, you need to understand where Sanctum fits in the ecosystem. Sanctum is not OAuth. It doesn't issue refresh tokens. It doesn't support third-party authorization flows. If you need those things, reach for Laravel Passport.

What Sanctum does exceptionally well is issue hashed personal access tokens tied to your users table via a polymorphic personal_access_tokens table. Each token can carry a set of abilities - scoped permissions that your application checks at runtime. The token itself is a random string; only the SHA-256 hash lives in the database. That's a sensible default.

For SPAs on the same domain, Sanctum piggybacks on Laravel's existing session authentication via a cookie. This is the EnsureFrontendRequestsAreStateful middleware doing its job. We'll touch on this briefly, but the majority of this guide focuses on token-based auth for API clients.

Installation and Setup (Laravel 11 / 12)

Sanctum ships with Laravel 11 and 12. If you're starting a fresh project, it's already in your composer.json. Confirm it's there:

composer show laravel/sanctum

If you're on an older install that upgraded to Laravel 11, you may need to install it:

composer require laravel/sanctum
php artisan vendor:publish --provider="Laravel\Sanctum\SanctumServiceProvider"
php artisan migrate

That publish step drops a config/sanctum.php file and a migration for the personal_access_tokens table. Run the migration before anything else.

bootstrap/app.php - The Laravel 11/12 Way

In Laravel 11, middleware registration moved out of Kernel.php and into bootstrap/app.php. This is where you register the Sanctum stateful middleware for SPA authentication:

// bootstrap/app.php
use Laravel\Sanctum\Http\Middleware\EnsureFrontendRequestsAreStateful;

->withMiddleware(function (Middleware $middleware) {
    $middleware->statefulApi();
})

Calling $middleware->statefulApi() is the clean Laravel 11 way to register Sanctum's SPA middleware. Avoid manually listing the middleware class inline, statefulApi() handles the correct ordering.

For token-only APIs (no SPA), you can skip this entirely. Your token-protected routes will use the auth:sanctum guard, which handles everything through the Authorization: Bearer header.

Configuring the Guard

In config/auth.php, make sure your api guard points to Sanctum:

'guards' => [
    'web' => [
        'driver'   => 'session',
        'provider' => 'users',
    ],

    'api' => [
        'driver'   => 'sanctum',
        'provider' => 'users',
    ],
],

This means auth:api and auth:sanctum both resolve to the same thing. Pick one and be consistent across your routes.

Preparing the User Model

Add the HasApiTokens trait to your User model:

use Laravel\Sanctum\HasApiTokens;

class User extends Authenticatable
{
    use HasApiTokens, HasFactory, Notifiable;
}

That trait gives Eloquent's User model the createToken(), tokens(), and currentAccessToken() methods. Everything downstream depends on this being in place.

Issuing Personal Access Tokens

Token issuance happens through a dedicated endpoint. Keep it clean: one controller, one responsibility.

// app/Http/Controllers/Auth/TokenController.php

namespace App\Http\Controllers\Auth;

use App\Http\Controllers\Controller;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Hash;
use Illuminate\Validation\ValidationException;
use App\Models\User;

class TokenController extends Controller
{
    public function issue(Request $request): \Illuminate\Http\JsonResponse
    {
        $request->validate([
            'email'       => ['required', 'email'],
            'password'    => ['required'],
            'device_name' => ['required', 'string', 'max:255'],
        ]);

        $user = User::where('email', $request->email)->first();

        if (! $user || ! Hash::check($request->password, $user->password)) {
            throw ValidationException::withMessages([
                'email' => ['The provided credentials are incorrect.'],
            ]);
        }

        $token = $user->createToken(
            $request->device_name,
            ['api:read', 'api:write']
        );

        return response()->json([
            'token' => $token->plainTextToken,
        ]);
    }

    public function revoke(Request $request): \Illuminate\Http\JsonResponse
    {
        $request->user()->currentAccessToken()->delete();

        return response()->json(['message' => 'Token revoked.']);
    }
}

The device_name field is part of Sanctum's design, it lets users see which device or client issued each token. The ValidationException approach returns a clean 422 with field-level errors rather than a 401, which is what most frontend clients expect.

Route registration:

// routes/api.php
use App\Http\Controllers\Auth\TokenController;

Route::post('/auth/token', [TokenController::class, 'issue']);
Route::delete('/auth/token', [TokenController::class, 'revoke'])->middleware('auth:sanctum');

Architect's Note: Never issue tokens from inside a route closure. The moment you need to add logging, rate limiting, or ability scoping, you're refactoring. Always use a dedicated controller from day one, the Service Container will thank you when you need to inject dependencies later.

Token Abilities: Scoped Permissions

Token abilities are Sanctum's lightweight alternative to OAuth scopes. You define them as strings at issuance time, then check them at the route or controller level.

// Read-only token
$token = $user->createToken('mobile-client', ['api:read']);

// Full-access token
$token = $user->createToken('admin-panel', ['api:read', 'api:write', 'api:delete']);

// AI feature token
$token = $user->createToken('ai-assistant', ['ai:query', 'api:read']);

Check abilities in your controllers using tokenCan():

public function store(Request $request): JsonResponse
{
    if (! $request->user()->tokenCan('api:write')) {
        return response()->json(['error' => 'Insufficient token abilities.'], 403);
    }

    // proceed with write operation
}

Or apply the abilities middleware directly on route groups:

Route::middleware(['auth:sanctum', 'abilities:api:read,api:write'])
    ->group(function () {
        Route::post('/documents', [DocumentController::class, 'store']);
    });

Route::middleware(['auth:sanctum', 'ability:api:read'])
    ->group(function () {
        Route::get('/documents', [DocumentController::class, 'index']);
    });

Note the difference: abilities (plural) requires the token to have all listed abilities. ability (singular) requires at least one. This distinction catches people off guard in production.

Protecting Routes

Standard protected route group for a JSON API:

// routes/api.php

Route::middleware('auth:sanctum')->group(function () {

    Route::apiResource('documents', DocumentController::class);

    Route::prefix('ai')->group(function () {
        Route::post('/query', [AiQueryController::class, 'query'])
            ->middleware('ability:ai:query');
        Route::get('/history', [AiQueryController::class, 'history'])
            ->middleware('ability:api:read');
    });

});

The auth:sanctum guard resolves the authenticated user from the Bearer token, making $request->user() available throughout the request lifecycle.

Token Revocation

You need at minimum three revocation endpoints:

// Revoke current token (logout this device)
Route::delete('/auth/token', [TokenController::class, 'revoke'])
    ->middleware('auth:sanctum');

// Revoke a specific token by ID
Route::delete('/auth/tokens/{tokenId}', [TokenController::class, 'revokeById'])
    ->middleware('auth:sanctum');

// Revoke all tokens (logout everywhere)
Route::delete('/auth/tokens', [TokenController::class, 'revokeAll'])
    ->middleware('auth:sanctum');

The controller methods:

public function revokeById(Request $request, int $tokenId): JsonResponse
{
    $deleted = $request->user()
        ->tokens()
        ->where('id', $tokenId)
        ->delete();

    if (! $deleted) {
        return response()->json(['error' => 'Token not found.'], 404);
    }

    return response()->json(['message' => 'Token revoked.']);
}

public function revokeAll(Request $request): JsonResponse
{
    $request->user()->tokens()->delete();

    return response()->json(['message' => 'All tokens revoked.']);
}

The ->where('id', $tokenId) scoping on the tokens relationship is critical. Without it, a user could delete another user's token by guessing IDs — an IDOR vulnerability. The relationship scope ensures only that user's tokens are in play before the delete fires.

Production Pitfall: Token revocation is only as fast as your database query. Under heavy concurrent traffic, a "revoke all" operation that hits an unindexed tokenable_id column will cause measurable latency spikes. Add an index to personal_access_tokens.tokenable_id, Laravel's migration doesn't include it by default in all versions.

// In a new migration
Schema::table('personal_access_tokens', function (Blueprint $table) {
    $table->index(['tokenable_type', 'tokenable_id']);
});

Token Expiry

By default, Sanctum tokens don't expire. Set an expiry in config/sanctum.php:

'expiration' => 60 * 24 * 30, // 30 days, in minutes

Pair this with a scheduled prune command:

// routes/console.php (Laravel 11)
Schedule::command('sanctum:prune-expired --hours=24')->daily();

Left unchecked, the personal_access_tokens table grows indefinitely and starts dragging down every authenticated request.

Rate Limiting with Redis

The auth:sanctum middleware doesn't include rate limiting. You're responsible for adding it. This is not optional for any API that calls an external AI provider - without it, a single misbehaving client can burn through your budget in minutes.

Define rate limiters in bootstrap/app.php:

use Illuminate\Cache\RateLimiting\Limit;
use Illuminate\Support\Facades\RateLimiter;

RateLimiter::for('api', function (Request $request) {
    return $request->user()
        ? Limit::perMinute(60)->by($request->user()->id)
        : Limit::perMinute(10)->by($request->ip());
});

RateLimiter::for('ai', function (Request $request) {
    return $request->user()
        ? Limit::perMinute(10)->by($request->user()->id)
        : Limit::perMinute(2)->by($request->ip());
});

Apply them to route groups:

Route::middleware(['auth:sanctum', 'throttle:api'])->group(function () {
    // standard API routes
});

Route::middleware(['auth:sanctum', 'throttle:ai'])->group(function () {
    // AI query routes
});

Switch your cache driver to Redis. The default file driver won't share rate limit counts across multiple server instances.

CACHE_DRIVER=redis
REDIS_HOST=127.0.0.1
REDIS_PORT=6379

Efficiency Gain: Redis handles both rate limiting and Sanctum's token lookup cache on the same connection. At scale, Sanctum resolves the user on every request by hashing the incoming token and hitting the database. If you're seeing measurable auth overhead in production profiling, an application-level cache keyed to the token hash eliminates that DB hit - but profile first, optimize second.

Multi-Tenant Token Scoping

A user who belongs to multiple organizations shouldn't be able to use a token issued for Organization A to access Organization B's data. The cleanest fix: store a team_id on the token itself using a custom token model.

Extend the default token model:

// app/Models/PersonalAccessToken.php

namespace App\Models;

use Laravel\Sanctum\PersonalAccessToken as SanctumToken;

class PersonalAccessToken extends SanctumToken
{
    protected $fillable = [
        'name',
        'token',
        'abilities',
        'expires_at',
        'team_id',
    ];
}

use Laravel\Sanctum\Sanctum;
use App\Models\PersonalAccessToken;

public function boot(): void
{
    Sanctum::usePersonalAccessTokenModel(PersonalAccessToken::class);
}

Add the column:

Schema::table('personal_access_tokens', function (Blueprint $table) {
    $table->unsignedBigInteger('team_id')->nullable()->index();
});

Issue tokens with team context:

$token = $user->createToken($request->device_name, ['api:read', 'api:write']);
$token->accessToken->forceFill(['team_id' => $currentTeam->id])->save();

Authorize against both user and team on every request:

$tokenTeamId = $request->user()->currentAccessToken()->team_id;

if ($tokenTeamId !== $resource->team_id) {
    abort(403, 'Token not authorized for this team.');
}

Listing Tokens for the Authenticated User

Give users visibility into their active tokens:

public function index(Request $request): JsonResponse
{
    $tokens = $request->user()->tokens()
        ->select(['id', 'name', 'abilities', 'last_used_at', 'expires_at', 'created_at'])
        ->latest()
        ->get()
        ->map(function ($token) {
            return [
                'id'           => $token->id,
                'name'         => $token->name,
                'abilities'    => $token->abilities,
                'last_used_at' => $token->last_used_at?->toDateTimeString(),
                'expires_at'   => $token->expires_at?->toDateTimeString(),
                'created_at'   => $token->created_at->toDateTimeString(),
            ];
        });

    return response()->json(['tokens' => $tokens]);
}

The raw token value is never returned here, that's only available at issuance via plainTextToken. After that, only the hash is stored.

Testing Sanctum Authentication

Sanctum ships with a clean testing helper:

use Laravel\Sanctum\Sanctum;

class DocumentApiTest extends TestCase
{
    use RefreshDatabase;

    public function test_authenticated_user_can_list_documents(): void
    {
        $user = User::factory()->create();
        Sanctum::actingAs($user, ['api:read']);

        $this->getJson('/api/documents')->assertStatus(200);
    }

    public function test_token_without_write_ability_cannot_create_document(): void
    {
        $user = User::factory()->create();
        Sanctum::actingAs($user, ['api:read']); // no api:write

        $this->postJson('/api/documents', [
            'title'   => 'Test',
            'content' => 'Test content',
        ])->assertStatus(403);
    }

    public function test_unauthenticated_request_is_rejected(): void
    {
        $this->getJson('/api/documents')->assertStatus(401);
    }
}

Sanctum::actingAs() bypasses the actual token lookup and tells the guard to treat the request as authenticated with those abilities. Your tests stay fast; no real tokens are issued.

Word to the Wise: Test the ability checks explicitly. Developers regularly ship applications where the happy path tests pass but the authorization boundaries are never verified. A token with api:read silently passing a api:write endpoint is a data integrity problem, not just a security one. Write the negative cases, they catch the middleware misconfiguration you didn't know you'd made.

Returning Consistent Auth Error Responses

By default, an unauthenticated request to a Sanctum-protected route returns a redirect to the login page. That's wrong for a JSON API. Fix it in bootstrap/app.php:

->withExceptions(function (Exceptions $exceptions) {
    $exceptions->render(function (AuthenticationException $e, Request $request) {
        if ($request->expectsJson()) {
            return response()->json(['error' => 'Unauthenticated.'], 401);
        }
    });

    $exceptions->render(function (AuthorizationException $e, Request $request) {
        if ($request->expectsJson()) {
            return response()->json(['error' => 'Forbidden.'], 403);
        }
    });
})

A 302 redirect from an API client that expected a 401 is a confusing failure mode that causes hours of frontend debugging.

SPA Authentication (Cookie-Based)

For a Vue or React SPA on the same top-level domain, Sanctum's stateful middleware handles auth through the session cookie — no tokens needed.

The flow: the SPA GETs /sanctum/csrf-cookie to initialize the CSRF cookie, POSTs credentials to your login endpoint, then includes the X-XSRF-TOKEN header on subsequent requests.

Configure your stateful domains in config/sanctum.php:

'stateful' => explode(',', env('SANCTUM_STATEFUL_DOMAINS', sprintf(
    '%s%s',
    'localhost,localhost:3000,127.0.0.1,127.0.0.1:8000,::1',
    Sanctum::currentApplicationUrlWithPort()
))),

Set SANCTUM_STATEFUL_DOMAINS explicitly in production. Wildcard domains are not supported.

Edge Case Alert: If your SPA and API are on different subdomains (app.example.com and api.example.com), the session cookie won't work across domains due to browser SameSite restrictions. Use token-based auth even for your SPA in this case. The cookie approach only works when both are served from the same domain.

Production Deployment Checklist

Database

personal_access_tokens migration applied
Index on tokenable_type and tokenable_id exists
Token expiry configured and sanctum:prune-expired scheduled

Configuration

APP_KEY is set and rotated from the default
SESSION_DRIVER is redis or database in multi-server setups
CACHE_DRIVER is redis for rate limiting across instances
SANCTUM_STATEFUL_DOMAINS explicitly set if using SPA auth

Rate Limiting

Named rate limiters defined for all API route groups
Stricter limiters applied to AI query endpoints
Rate limit responses return Retry-After headers

Security

Authorization header stripped at the CDN for non-API routes
HTTPS enforced — tokens over HTTP is a non-starter
Tokens are never logged — audit your logging configuration

Monitoring

Track personal_access_tokens table row count as a metric
Alert on unusual token issuance spikes
Log token revocation events for audit trails

What Sanctum Doesn't Cover

Sanctum doesn't do: OAuth2 authorization flows, refresh tokens, third-party client authorization, dynamic scope negotiation, or JWT issuance. If any of those are requirements, use Passport.

For the overwhelming majority of Laravel APIs — including every AI integration pattern — Sanctum's personal access tokens, SPA sessions, ability scoping, and revocation model cover the full authentication lifecycle. Don't reach for Passport's complexity if Sanctum's model fits your use case. The operational overhead isn't trivial.

Official Documentation

Originally published at origin-main.com — Laravel + AI integration guides for production developers.

Laravel AI Middleware: Token Tracking & Rate Limiting

Dewald Hugo — Sun, 05 Apr 2026 00:10:17 +0000

Integrating Claude or GPT-4o into a Laravel app is deceptively easy. One Http::post() and you feel like a genius. Then a single user loops your endpoint, or a bot scripts it, and your OpenAI bill climbs $300 before you wake up. The answer is a dedicated Laravel AI middleware layer, and most tutorials skip it entirely.

That’s not a hypothetical. It happens.

The fix isn’t smarter prompts, it’s infrastructure. In this guide, we build a proper AI management layer: tiered rate limiting, per-user token tracking with async logging, and a Pest test suite that actually validates the guards are working. We’re not demoing a toy. We’re building something you can ship. For prompt versioning and deployment discipline on the prompt side of that infrastructure, see the prompt migrations guide.

1. The Strategy: Why a Laravel AI Middleware Layer?

Middleware in the Laravel lifecycle is the right place for this because it separates infrastructure concerns (cost, rate limits) from business logic (the prompt itself). This decoupling means you can swap gpt-4o for a self-hosted Llama instance without touching a single Controller.

For the output side of that separation, validating what the model returns before your application acts on it, see the guide on schema validation against LLM hallucinations.

We’re tracking cost using the standard token pricing formula:

$$Total Cost = \sum_{i=1}^{n} (Tokens_{in} \cdot Price_{in} + Tokens_{out} \cdot Price_{out})$$

Centralise that calculation once. Never scatter it across Controllers.

2. Database Schema: An Audit Trail, Not Just a Counter

Don’t fall into the trap of incrementing a single total_tokens integer on your users table. You need a row per request so you can audit, refund, and debug.

// database/migrations/xxxx_xx_xx_create_ai_usage_logs_table.php
public function up(): void
{
    Schema::create('ai_usage_logs', function (Blueprint $table) {
        $table->id();
        $table->foreignId('user_id')->constrained()->onDelete('cascade');
        $table->string('model_used');        // e.g., 'gpt-4o', 'claude-sonnet-4-6'
        $table->string('feature_name');      // e.g., 'blog_generator', 'chat'
        $table->integer('prompt_tokens')->default(0);
        $table->integer('completion_tokens')->default(0);
        $table->decimal('estimated_cost', 10, 6)->default(0);
        $table->timestamps();
    });
}

3. Tiered Rate Limiting

Define your tiers in AppServiceProvider. Free users get 5 requests per minute. Premium gets 100. Adjust to whatever your unit economics demand.

// app/Providers/AppServiceProvider.php
use Illuminate\Cache\RateLimiting\Limit;
use Illuminate\Support\Facades\RateLimiter;
use Illuminate\Http\Request;

public function boot(): void
{
    RateLimiter::for('ai-api', function (Request $request) {
        $user = $request->user();

        return $user?->is_premium
            ? Limit::perMinute(100)->by($user->id)
            : Limit::perMinute(5)->by($user->id);
    });
}

Senior Dev Tip: Back this with Redis, not your database. Set CACHE_DRIVER=redis in .env. If you’re running the default file or database driver, every AI request triggers a DB read-write cycle under the rate limiter — that’s a quiet bottleneck that won’t show up until you’re under load. See the Laravel Cache documentation for driver configuration.

4. Building the Middleware

Generate the class:

php artisan make:middleware EnsureUserHasAiCredits

This middleware enforces auth and your per-user spending ceiling before the request ever reaches the Controller:

// app/Http/Middleware/EnsureUserHasAiCredits.php
namespace App\Http\Middleware;

use Closure;
use Illuminate\Http\Request;
use Symfony\Component\HttpFoundation\Response;

class EnsureUserHasAiCredits
{
    public function handle(Request $request, Closure $next): Response
    {
        $user = $request->user();

        if (! $user) {
            return response()->json(['error' => 'Unauthorized'], 401);
        }

        if ($user->monthly_spend >= $user->spending_limit) {
            return response()->json([
                'error'   => 'Monthly spending limit reached.',
                'upgrade' => route('billing.upgrade'),
            ], 402);
        }

        return $next($request);
    }
}

Registering the Middleware (Laravel 11+)

Laravel 11 dropped app/Http/Kernel.php. Register your middleware in bootstrap/app.php:

// bootstrap/app.php
->withMiddleware(function (Middleware $middleware) {
    $middleware->appendToGroup('api', [
        \App\Http\Middleware\EnsureUserHasAiCredits::class,
    ]);
})

Then apply the rate limiter on your route:

// routes/api.php
Route::post('/generate', [AiController::class, 'generate'])
    ->middleware('throttle:ai-api');

5. Externalising Model Pricing

Hardcoding 0.000005 per token inside your Job is a maintenance trap. One pricing change from OpenAI and your cost accounting silently breaks. Put rates in config/ai.php:

// config/ai.php
return [
    'models' => [
        'gpt-4o' => [
            'prompt_rate'     => 0.000005,
            'completion_rate' => 0.000015,
        ],
        'claude-sonnet-4-5' => [
            'prompt_rate'     => 0.000003,
            'completion_rate' => 0.000015,
        ],
    ],
];

6. Async Usage Logging with a Queued Job

Once the API responds, log asynchronously. Don’t make the user wait for a DB write.

// app/Jobs/LogAiUsage.php
namespace App\Jobs;

use App\Models\AiUsageLog;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class LogAiUsage implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function __construct(
        public readonly int    $userId,
        public readonly array  $usageData,
    ) {}

    public function handle(): void
    {
        $model  = $this->usageData['model'];
        $rates  = config("ai.models.{$model}", [
            'prompt_rate'     => 0.000005,
            'completion_rate' => 0.000015,
        ]);

        $cost = ($this->usageData['prompt_tokens'] * $rates['prompt_rate'])
              + ($this->usageData['completion_tokens'] * $rates['completion_rate']);

        AiUsageLog::create([
            'user_id'           => $this->userId,
            'model_used'        => $model,
            'feature_name'      => $this->usageData['feature_name'],
            'prompt_tokens'     => $this->usageData['prompt_tokens'],
            'completion_tokens' => $this->usageData['completion_tokens'],
            'estimated_cost'    => $cost,
        ]);
    }
}

Notice feature_name is included in both the Job payload and the create() call. The original migration defines that column, if you omit it from the insert, Laravel will either throw a NOT NULL constraint violation or silently ignore it if it’s nullable. Either way, you’ve lost attribution data. Always match your payload to your schema.

7. The Controller: Gluing It Together

// app/Http/Controllers/AiController.php
namespace App\Http\Controllers;

use App\Jobs\LogAiUsage;
use Illuminate\Http\Request;
use Illuminate\Http\JsonResponse;
use OpenAI\Laravel\Facades\OpenAI;
use Illuminate\Validation\ValidationException;

class AiController extends Controller
{
    public function generate(Request $request): JsonResponse
    {
        $validated = $request->validate([
            'prompt'  => ['required', 'string', 'max:4000'],
            'feature' => ['required', 'string', 'in:blog_generator,chat,summariser'],
        ]);

        try {
            $result = OpenAI::chat()->create([
                'model'    => 'gpt-4o',
                'messages' => [['role' => 'user', 'content' => $validated['prompt']]],
            ]);
        } catch (\OpenAI\Exceptions\TransporterException $e) {
            report($e);
            return response()->json(['error' => 'AI service unavailable. Please try again.'], 503);
        } catch (\OpenAI\Exceptions\ErrorException $e) {
            // Catches 429s, quota exceeded, invalid requests from OpenAI
            report($e);
            return response()->json(['error' => $e->getMessage()], $e->getCode() ?: 500);
        }

        LogAiUsage::dispatch(auth()->id(), [
            'model'             => 'gpt-4o',
            'feature_name'      => $validated['feature'],
            'prompt_tokens'     => $result->usage->promptTokens,
            'completion_tokens' => $result->usage->completionTokens,
        ]);

        return response()->json([
            'content' => $result->choices[0]->message->content,
        ]);
    }
}

If you’re integrating Anthropic’s SDK instead, the pattern is identical — swap the client call and update the model string to claude-sonnet-4-5. See the Anthropic API documentation for the current SDK reference.

8. Testing the Architecture with Pest

A production system is only as solid as the test suite backing it. We need to verify three things independently:

The rate limiter blocks users who’ve exhausted their quota.
The middleware halts users who’ve hit their spending ceiling.
The LogAiUsage Job is dispatched with the correct payload — without hitting the real API.

That last point is where most AI testing tutorials fail you. Bus::fake() queues the Job but does nothing about the OpenAI facade. If you don’t mock it, your test suite is burning real API credits.

// tests/Feature/AiGenerationTest.php
use App\Jobs\LogAiUsage;
use App\Models\User;
use Illuminate\Support\Facades\Bus;
use Illuminate\Support\Facades\RateLimiter;
use OpenAI\Laravel\Facades\OpenAI;
use OpenAI\Responses\Chat\CreateResponse;

beforeEach(function () {
    Bus::fake();
});

it('blocks a free user who has hit their rate limit', function () {
    $user = User::factory()->create(['is_premium' => false]);

    // Simulate 5 prior requests — the free-tier ceiling
    foreach (range(1, 5) as $i) {
        RateLimiter::hit('ai-api:' . $user->id);
    }

    $this->actingAs($user)
        ->postJson('/api/generate', [
            'prompt'  => 'Hello AI',
            'feature' => 'chat',
        ])
        ->assertStatus(429);
});

it('blocks a user who has exceeded their monthly spending limit', function () {
    $user = User::factory()->create([
        'monthly_spend'  => 50.00,
        'spending_limit' => 50.00,
    ]);

    $this->actingAs($user)
        ->postJson('/api/generate', [
            'prompt'  => 'Hello AI',
            'feature' => 'chat',
        ])
        ->assertStatus(402)
        ->assertJsonPath('error', 'Monthly spending limit reached.');
});

it('dispatches a LogAiUsage job with correct token data on success', function () {
    $user = User::factory()->create(['is_premium' => true]);

    // Mock the OpenAI facade — never hit the real API in tests
    OpenAI::fake([
        CreateResponse::fake([
            'choices' => [
                ['message' => ['role' => 'assistant', 'content' => 'Mocked response']],
            ],
            'usage' => [
                'prompt_tokens'     => 15,
                'completion_tokens' => 42,
                'total_tokens'      => 57,
            ],
        ]),
    ]);

    $this->actingAs($user)
        ->postJson('/api/generate', [
            'prompt'  => 'Test prompt',
            'feature' => 'chat',
        ])
        ->assertOk()
        ->assertJsonPath('content', 'Mocked response');

    Bus::assertDispatched(LogAiUsage::class, function (LogAiUsage $job) use ($user) {
        return $job->userId === $user->id
            && $job->usageData['prompt_tokens'] === 15
            && $job->usageData['completion_tokens'] === 42
            && $job->usageData['feature_name'] === 'chat';
    });
});

If you’re building out the Eloquent side of this and want to understand how to test model relationships and factory states at scale, the patterns covered in building robust Laravel test factories translate directly to the User and AiUsageLog models here.

9. What You Now Have

Shipping this architecture gives you:

Cost protection — tiered rate limiting via the Laravel RateLimiter facade backed by Redis, plus a hard monthly ceiling enforced in middleware.
Full audit trail — every token consumed is recorded per-user, per-feature, per-model via an Eloquent model in the Service Container-resolved LogAiUsage Job.
Zero latency impact — async logging means your users never wait for a DB write.
A test suite that doesn’t cheat — the OpenAI facade is mocked, so your CI pipeline stays free and your assertions are actually trustworthy.

The gap between a weekend project and a production system isn’t the AI integration. It’s everything around it. This is that layer.

References:

Why I Still Choose Laravel in a World Full of Node and Python AI Stacks

Dewald Hugo — Tue, 31 Mar 2026 17:27:16 +0000

This article comes from a question I keep getting from developers working in JavaScript and Python ecosystems: “Why are you building AI systems with Laravel?” It usually comes up in Slack threads, code reviews, or on X, and more often than not there’s some skepticism behind it. With Node.js and Python dominating most AI conversations, Laravel isn’t the obvious choice. But after using it to build and run real systems, I’ve found that assumption doesn’t really hold up.

The Noise Gets Louder Every Year

Open any developer forum right now and the message is consistent. Python for AI. Node for APIs. Go for infrastructure. The JavaScript ecosystem has colonised the frontend and is making a serious run at the backend. Python owns the ML research pipeline and, by extension, a growing portion of production AI services.

The noise is real. And it’s loud.

When OpenAI shipped their Python SDK first, and their Node SDK second it sent a signal. When LangChain became the default mental model for AI agent architecture, it was written in Python. When the majority of AI tutorials you find online begin with pip install, you start to wonder whether your technology choice has quietly become an act of stubbornness.

It hasn’t. But you do need a clear-headed reason to stay. “PHP has come a long way” is not that reason. A genuine architectural argument is.

Let’s Be Honest About What Node and Python Actually Offer

Node is fast. Non-blocking I/O was genuinely transformative when it arrived, and for high-concurrency, low-computation workloads it remains a defensible choice. The npm ecosystem is enormous. TypeScript has matured to the point where large Node codebases are actually maintainable. I’m not here to dismiss it.

Python’s position in AI is structural, not accidental. The scientific computing ecosystem — NumPy, PyTorch, Hugging Face Transformers, was built in Python because that’s where ML researchers lived. That inertia doesn’t disappear. If you’re training models, fine-tuning, or doing anything below the inference layer, Python is the correct tool. Full stop.

But here’s the thing most of these comparisons consistently miss. The majority of us are not building model-training pipelines. We’re building applications that consume AI APIs. We’re wiring GPT-4o or Claude Sonnet into business logic, user interfaces, queue workers, and database-backed workflows. That is a very different problem. The 2025 Stack Overflow Developer Survey, with responses from over 49,000 developers, puts some hard numbers against the landscape we’re all navigating:

Source: Stack Overflow Developer Survey 2025 (49,000+ respondents). Usage = % of all respondents reporting active use in the past year. Ratings reflect relative strength within each category; all frameworks are production-capable. Node.js 2025 figure reflects SO methodology consolidation - direct comparison with 2024 (42.7%) is approximate.

The usage figures are worth sitting with for a moment. Node.js dominates at nearly half of all respondents, but look at the full-stack fit and AI ecosystem scores. High adoption does not mean best fit for the problem at hand. FastAPI is the fastest-growing framework in the survey, up five percentage points year-on-year, and it earns its AI ecosystem score. If you are building a dedicated inference microservice, FastAPI is a serious option. But Laravel’s full-stack cohesion score is the relevant column for the majority of us building complete, stateful applications around AI features. No other framework in this comparison comes close on that dimension.

Laravel for AI Development Is Not a Compromise

The framing I keep pushing back on is that choosing Laravel for AI development means accepting a trade-off. That you’re surrendering capability in exchange for developer ergonomics. In 2026, that framing is simply wrong.

Laravel ships with a queue system that handles background AI jobs cleanly and at scale. Redis integration is first-class - which matters the moment you start caching embeddings or enforcing per-user rate limits on API consumption. Horizon gives you real-time visibility into your queue workers without standing up a separate observability tool. Reverb brings WebSocket support in-house, which is directly relevant when you’re streaming LLM responses to a frontend in real time. Octane keeps your application warm between requests, reducing cold-start overhead that becomes noticeable in latency-sensitive AI features.

None of this requires third-party assembly. It is cohesive, well-documented, and it behaves predictably under load.

Compare that to a Node AI backend assembled from Express, BullMQ, Socket.io, and Prisma. Each of those is a capable library. Together, they are a distributed maintenance surface. The integration points between them are where your bugs live at 2am.

The Service Container Is Still the Best Idea in Backend Development

I’ve worked across enough stacks to have a view on this. Nothing in most ecosystems matches the elegance of Laravel’s Service Container for managing application complexity at the point where it actually gets complex.

When your AI integration grows beyond a single API call you need somewhere to put the abstraction. Provider contracts, retry logic, fallback behaviour, token accounting, telemetry hooks. The Service Container lets you bind all of that behind an interface and swap implementations without touching the business logic that depends on them. It’s the kind of architectural freedom that sounds theoretical until the day you need to switch from OpenAI to Anthropic under a tight deadline - and you do it in one service provider file, not forty controller methods.

If you’ve been following the production on my blog, you’ll recognise this exact approach from our piece on Production-Grade AI Architecture in Laravel: Contracts, Governance & Telemetry — binding AI providers behind contracts isn’t just clean code, it’s the architectural difference between an AI feature and an AI-dependent liability.

Eloquent Doesn’t Get Enough Credit in the AI Conversation

Everyone talks about the inference layer. Nobody talks about the data layer. But AI applications are deeply stateful. Conversation history, user context, embedding vectors, usage logs, rate-limit counters, prompt version records - all of it needs to live somewhere reliable, queryable, and maintainable.

Eloquent ORM, combined with PostgreSQL and pgvector for semantic search, handles this with a maturity that ORMs in younger ecosystems are still building toward. Eloquent scopes make filtering conversation history by user, session, or token budget trivially readable. Eloquent events let you hook telemetry into model persistence without polluting your service logic. The relationship system means your AI-related domain models compose naturally with the rest of your application data.

This is the part of the stack that’s invisible when everything works and catastrophic when it doesn’t. Laravel’s data layer has fifteen years of production hardening behind it. That’s not a minor detail.

The Ecosystem Has Quietly Caught Up

The argument that Laravel lacks AI tooling was fair eighteen months ago. It is not fair now.

Prism PHP brings a unified, multi-provider AI interface to Laravel - supporting tool calling, RAG pipelines, and streaming, without you writing SDK glue code by hand. We’ve covered what that looks like in practice in our guide to building agentic Laravel apps with Prism PHP, and the developer experience is genuinely impressive. The abstraction is clean, provider support is broad, and it composes naturally with the Service Container patterns you’d already be using. Beyond Prism, the broader package ecosystem has responded: prompt versioning tools, queue-based AI pipeline packages, vector search integrations. The gaps are closing fast.

Taylor Otwell and the core team have also signalled clearly that AI tooling is a first-party priority on Laravel’s roadmap. When the framework authors treat your use case as a core concern, that is a meaningful long-term signal. The trend data below adds useful context, and the story it tells is more nuanced than the noise in developer forums would have you believe:

Source: Stack Overflow Developer Survey 2020–2025. Usage = % of all respondents reporting active use in the past year. FastAPI was first tracked in 2021. The 2025 Node.js figure (48.7%, dashed) reflects a Stack Overflow methodology consolidation; 2024 figure was 42.7% — direct year-on-year comparison is approximate. Express and Django shown as dashed lines to indicate secondary/framework-layer status vs runtime-level Node.js. 2020–2023 intermediate values for Express, FastAPI, and Laravel are interpolated from published Stack Overflow survey trend commentary.

FastAPI’s trajectory is the most important line on that chart. It is rising sharply, and deservedly so for pure inference workloads. But look at where it is rising from, it has only just crossed Django’s share after four years of consistent growth, and it is still well below Laravel’s current position. Node.js, despite the headline numbers, has been in structural decline since its 2020 peak of 51.4%. Laravel’s decline is real and worth acknowledging honestly, but the rate is shallow, and the developer base that remains is building serious production applications. The community is consolidating around quality, not haemorrhaging. There is a difference.

Developer Velocity Is a Business Metric

Here is the argument that tends to land hardest, and it’s also the most honest one.

Speed of iteration matters more than theoretical capability, especially in the early and middle stages of an AI product. The developer who can ship a working, testable, observable AI feature in a day using Laravel is more valuable than the developer who spends three days assembling a Python FastAPI service with the same surface area of functionality. The gap between “it works locally” and “it works in production” is where Laravel’s ecosystem earns its keep - the test factories, the job batching, the queue introspection, the deployment ergonomics that come with a genuinely mature framework.

This is not tribalism. It’s a productivity calculation. And in 2026, with AI features now expected in virtually every serious web product, the developer who moves fast without breaking things has a structural advantage. Laravel, applied properly, is that advantage.

So Why Do Developers Keep Reaching for Node and Python?

Honestly? Familiarity and social proof. AI tutorials default to Python because that’s where the research originates. JavaScript developers building frontends reach for Node because the context switch is smaller. Neither instinct is wrong, learning inside your comfort zone is rational behaviour.

But comfort zone and best tool are not synonyms.

If you’re a Laravel developer who has been watching the AI conversation happen in other ecosystems, quietly wondering whether you’re missing something, you’re not. The framework you already know is more capable for this problem space than most of what you’ve been reading would suggest. The Service Container, Eloquent, Redis, Horizon, Reverb, Prism - these are not consolation prizes. They’re a coherent, production-tested AI application stack.

You don’t need to leave the ecosystem to build great AI-powered software. You need to go deeper in it.

One More Thing Before You Go

This has been an opinion piece, and I want to be transparent about that. I’ve tried to make the case honestly, acknowledging where Node and Python are genuinely strong, and being specific about where I think Laravel holds its own. Reasonable people disagree with parts of this, and I’d genuinely like to hear where you land.

Have you migrated an AI backend away from Laravel? Did it go the way you expected? Are you running a polyglot stack and making it work? Drop your thoughts in the comments below. This is exactly the kind of conversation worth having in public rather than in private Slack threads where nobody learns anything. If this resonated with you, share it with a developer who’s currently weighing up this decision. The more honest voices in that conversation, the better.

What Laravel 13 Actually Changes for AI Development

Dewald Hugo — Sat, 28 Mar 2026 13:39:56 +0000

Laravel 13 dropped a production-stable AI SDK on release day and nobody's talking about what it actually replaces. Here's the full breakdown.

Laravel 13 launched on March 17, 2026 , and Taylor Otwell kept his Laracon EU promise: zero breaking changes, a clean upgrade from Laravel 12, and one headline shift that resets the default approach to AI in PHP. The Laravel 13 AI SDK is now production-stable, first-party, and catalogued inside the official Laravel documentation as a core concern — not a side-effect of a community package you bolt on after you’ve already made architectural decisions you’ll later regret.

This is not a release you skim for changelog bullet points and move on. For anyone building AI-powered features on Laravel, this release fundamentally changes how you should be structuring that work. Let’s go through it properly.

Why Laravel 13 Is an AI-First Release

Laravel 13 continues Laravel’s annual release cadence with a focus on AI-native workflows, stronger defaults, and more expressive developer APIs. The official framing is “minimal breaking changes,” and that’s accurate — but undersells it. The infrastructure laid here directly shapes whether your AI features remain maintainable at scale or turn into the kind of controller-level spaghetti you spend the next two years paying down.

The Laravel AI SDK moves from beta to production-stable on the same day as Laravel 13. It is included as a first-party package and gives you a single, provider-agnostic interface for text generation, tool-calling agents, image creation, audio synthesis, and embedding generation. The SDK handles retry logic, error normalisation, and queue integration behind the scenes. You get all of that without writing a custom abstraction layer, and without coupling your application to a specific provider’s SDK contract.

That last point deserves more weight than the release notes give it.

The Laravel AI SDK: What It Actually Gives You

Text Generation and Agents

The simplest entry point looks like this:

use App\Ai\Agents\SalesCoach;
$response = SalesCoach::make()->prompt('Analyse this sales transcript...');
return (string) $response;

With the AI SDK, you can build provider-agnostic AI features while keeping a consistent, Laravel-native developer experience. That means your SalesCoach agent does not care whether it’s backed by OpenAI, Anthropic, or Google Gemini. You wire the provider in config/ai.php and the agent contract stays unchanged.

The default models used for text, images, audio, transcription, and embeddings are now configurable in your application’s config/ai.php file. This gives you granular control over the exact models you’d like to use if you don’t want to rely on the package defaults. A minimal configuration targeting Anthropic looks like this:

// config/ai.php
return [
    'models' => [
        'text' => [
            'default'  => env('AI_TEXT_MODEL', 'claude-sonnet-4-6'),
            'cheapest' => 'claude-haiku-4-5-20251001',
            'smartest' => 'claude-opus-4-6',
        ],
    ],
];

You are not hardcoding a model string into a service class. You are not parsing a .env file in three different controllers. One config file governs the whole application. If you need to roll back a model mid-incident, it’s one value and a php artisan config:cache.

Images and Audio

For visual generation use cases, the SDK offers a clean API for creating images from plain-language prompts. For voice experiences, you can synthesize natural-sounding audio from text for assistants, narrations, and accessibility features.

use Laravel\Ai\Image;
use Laravel\Ai\Audio;
// Image generation
$image = Image::of('A product shot of a minimalist desk lamp')->generate();
$rawContent = (string) $image;
// Audio synthesis
$audio = Audio::of('Your order has been confirmed.')->generate();
$rawContent = (string) $audio;

The fluent API is consistent across modalities. That consistency is not accidental. It means a developer who’s only worked with text generation can pick up image or audio synthesis in minutes — no mental context switch, no separate SDK documentation to parse.

Embeddings and the `Str` Helper

Here’s where it gets genuinely interesting. Embedding generation is wired directly into Laravel’s Strhelper:

use Illuminate\Support\Str;
$vector = Str::of('Napa Valley has exceptional Cabernet Sauvignon.')->toEmbeddings();

That’s not a convenience method. That’s the framework signalling that embeddings are now a first-class data type. You’re not reaching for a one-off utility class — you’re calling a helper that sits alongside Str::slug() and Str::limit() as a standard part of the toolkit.

[Architect’s Note] The fact that toEmbeddings() lives on the Strhelper rather than a dedicated Embeddingfacade is a deliberate design choice. It keeps embedding generation composable with the rest of your string manipulation pipeline. Chain it after a limit() call to trim tokens before you generate — or after markdown() to clean formatting before vectorising. Think of it as the framework nudging you toward preprocessing being part of the same expression.

Native Vector Search: Semantic Queries Without a Search Engine Subscription

Laravel 13 deepens its semantic search story with native vector query support, embedding workflows, and related APIs. These features make it straightforward to build AI-powered search experiences using PostgreSQL and pgvector, including similarity search against embeddings generated directly from strings.

The query builder extension looks like this:

$documents = DB::table('documents')
    ->whereVectorSimilarTo('embedding', 'Best restaurants in Cape Town')
    ->limit(10)
    ->get();

Under the hood, this generates a pgvector-compatible cosine similarity query against your Postgres database. No Elasticsearch cluster. No Typesense subscription. No Algolia bill. For a significant number of use cases — internal knowledge bases, product catalogue search, customer support retrieval — Postgres with pgvector is more than sufficient, and the operational overhead is zero if you’re already on Postgres.

The migration for the vector column uses a new column type:

Schema::create('documents', function (Blueprint $table) {
    $table->id();
    $table->text('content');
    $table->vector('embedding', 1536); // dimension matches your model's output
    $table->timestamps();
});

A realistic ingestion pipeline that generates and stores embeddings looks like this:

use Illuminate\Support\Str;
use App\Models\Document;
class IngestDocumentJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
    public function __construct(private readonly string $content) {}
    public function handle(): void
    {
        $embedding = Str::of($this->content)->toEmbeddings();
        Document::create([
            'content'   => $this->content,
            'embedding' => $embedding,
        ]);
    }
}

Dispatch it from a controller, and your embedding pipeline is queued, retried on failure, and observable via Horizon — exactly the same way you’d treat any other background job.

[Production Pitfall] Embedding API calls are slow — typically 300–800ms depending on payload size and provider latency. Do not generate embeddings synchronously in a request lifecycle. Every embedding generation call belongs in a queued job, period. Under load, synchronous embedding generation will saturate your PHP-FPM pool faster than almost anything else. We’ve seen this exact pattern cause cascading timeouts in staging under traffic replay. Don’t let it reach production.

Provider Agnosticism: What It Means Practically

Switch providers by changing one config value. Teams that offer professional Laravel development services can now build AI-powered features inside a standard Laravel project — no custom abstraction layers, no third-party SDK gymnastics required.

This matters more than it reads. Prior to the AI SDK going stable, the common pattern was to inject an OpenAI client through the Service Container, write a thin wrapper around it, and pray that the wrapper held when you inevitably needed to test responses or swap a model mid-sprint. Most teams ended up with one of three things: a god-service that knew too much, a leaky abstraction that only worked for their current provider, or no abstraction at all and raw Http::post() calls in controllers.

The AI SDK replaces all three anti-patterns with a single, testable, swappable interface that the framework itself maintains. If the underlying provider deprecates an API method, the fix lands in a framework update — not in your codebase.

Here’s how you’d swap to a different provider for a specific agent without touching the rest of the application:

use Laravel\Ai\Facades\Ai;
use Laravel\Ai\Enums\Provider;
$response = Ai::using(Provider::Anthropic)
    ->withModel('claude-opus-4-6')
    ->prompt('Summarise this legal document...');

That using() call overrides the config/ai.php default for this invocation only. You can do per-request provider overrides without touching global configuration. That’s useful in multi-tenant applications where different plans map to different model tiers.

If you’re already running a custom abstraction over the OpenAI PHP SDK, we wrote about how to migrate that pattern properly — including how to handle cost visibility and telemetry — in Production-Grade AI Architecture in Laravel: Contracts, Governance & Telemetry. The AI SDK doesn’t replace the need for those architectural decisions; it gives you a better foundation to implement them on.

Tool-Calling Agents: Building Agentic Workflows Natively

The AI SDK’s agent support deserves its own section. Tool-calling — the mechanism by which an LLM decides to invoke a function in your application — is the primitive that makes agentic workflows possible.

A basic agent with tools looks like this:

namespace App\Ai\Agents;
use Laravel\Ai\Agent;
use Laravel\Ai\Attributes\Tool;
class OrderAssistant extends Agent
{
    protected string $model = 'claude-sonnet-4-6';
    protected string $instructions = 'You are a helpful order management assistant.';
    #[Tool('Look up the status of an order by ID')]
    public function getOrderStatus(int $orderId): string
    {
        $order = Order::findOrFail($orderId);
        return "Order #{$orderId} is currently {$order->status}.";
    }
    #[Tool('List all open orders for a given customer')]
    public function listOpenOrders(int $customerId): array
    {
        return Order::where('customer_id', $customerId)
            ->where('status', 'open')
            ->pluck('id')
            ->toArray();
    }
}

The #[Tool] attribute wires the method directly into the model’s tool-calling interface. The SDK handles the round-trip: the model sees the tool definition, decides to call it, the SDK invokes your method, and the result is injected back into the conversation automatically.

Notice that getOrderStatusand listOpenOrdersare standard Eloquent queries. There is nothing AI-specific about the business logic inside the tools. This is the correct separation. Your tools are Laravel code. The AI SDK manages the protocol layer between your code and the model.

[Edge Case Alert] Tool-calling agents can enter infinite loops if the model repeatedly decides to call the same tool with the same arguments — this happens when the tool’s return value doesn’t advance the conversation’s goal. Always set a maxStepslimit on your agents and handle MaxStepsExceededExceptionexplicitly. Defaulting to unlimited steps in production is asking for a runaway API bill.

$response = OrderAssistant::make()
    ->maxSteps(10)
    ->prompt("What's the status of order 99?");

If you’re building more sophisticated multi-step agentic workflows and need schema validation on tool outputs — which you almost certainly will once you’re past demos — the Hardening Laravel Agentic Workflows: Schema Validation Against LLM Hallucinations guide covers exactly that.

Queue Integration and `config/ai.php` as Operational Infrastructure

One under-discussed aspect of the AI SDK is how it integrates with Laravel’s Queue system. Transcription now supports timeouts, giving you better control in production workloads and preventing long-running requests from tying up workers.

$transcript = Transcription::fromPath('./podcast.mp3')
    ->timeout(240)
    ->generate(Lab::ElevenLabs);

That timeout() call maps directly to the queue worker’s job timeout. If you’re already familiar with $timeout on job classes, this is the same mechanism — now surfaced at the SDK level so you don’t have to know to set it yourself.

Pair this with Laravel 13’s new Queue::route() method for clean queue topology:

// app/Providers/AppServiceProvider.php
use Illuminate\Support\Facades\Queue;
Queue::route([
    IngestDocumentJob::class   => 'embeddings',
    GenerateImageJob::class    => 'ai-images',
    TranscribeAudioJob::class  => 'transcription',
]);

AI jobs should never share a queue with your standard application jobs. Embedding generation and image synthesis calls have entirely different latency profiles and retry semantics than sending a welcome email. The new Queue::route() method lets you define which queue and connection each job class uses from a single location in a service provider. Previously, teams either set queue properties on each job class or repeated the configuration at every dispatch site.

Centralise that in a Service Provider and you’ve got one place to retune queue topology when your AI workload changes.

Error Handling: What the SDK Does, and What It Doesn’t

The AI SDK handles retry logic and error normalisation internally, but that does not mean you write no error handling. It means you handle fewer low-level concerns. The contracts you still own:

use Laravel\Ai\Exceptions\AiProviderException;
use Laravel\Ai\Exceptions\RateLimitException;
use Laravel\Ai\Exceptions\ContentFilterException;
try {
    $response = SalesCoach::make()->prompt($userInput);
} catch (RateLimitException $e) {
    // The SDK exhausted its internal retry budget — back off and re-queue
    InvokeSalesCoachJob::dispatch($userInput)->delay(now()->addSeconds(60));
} catch (ContentFilterException $e) {
    // The provider flagged the input — log it, don't retry
    Log::warning('Content filter triggered', ['input_hash' => hash('sha256', $userInput)]);
    return response()->json(['error' => 'Input could not be processed.'], 422);
} catch (AiProviderException $e) {
    // Unknown provider error — log full context, fail gracefully
    Log::error('AI provider failure', ['message' => $e->getMessage()]);
    return response()->json(['error' => 'Service temporarily unavailable.'], 503);
}

The SDK’s internal retry logic handles transient 5xx errors and network timeouts. RateLimitException is thrown when the SDK’s retry budget is exhausted — at that point, you need application-level backpressure, not another retry. Handle these as distinct failure modes. They are.

[Word to the Wise] Logging $e->getMessage() on a provider exception often includes the raw prompt in the message string depending on the provider. Sanitise before logging in any context where the prompt may contain user PII. This is not a theoretical concern — it’s the kind of thing that shows up in a GDPR audit.

The Bigger Picture: Laravel’s AI Roadmap Signal

The Laravel AI SDK going stable on the same day as Laravel 13 is a deliberate signal: the framework’s roadmap is now AI-first. The docs now include a dedicated AI section with entries for the AI SDK, MCP integration, and Laravel Boost — Taylor’s AI-assisted development tooling. The installation docs now include a dedicated “Laravel and AI” section, and existing apps are pointed to Laravel Boost for AI-assisted development workflows.

Read that as a directional commitment. The primitives shipped in Laravel 13 — provider-agnostic text generation, first-class embeddings, native vector queries, tool-calling agents — are the foundation. What gets built on top of them in 13.x point releases and Laravel 14 will assume they exist. If you’re planning AI features over the next 12 months and you’re not on Laravel 13, you’re doing that planning on a weaker foundation than the framework now offers.

The upgrade path is genuinely low-friction for most apps. The official release notes emphasise minimal breaking changes, and the official upgrade guide estimates about 10 minutes for many applications. The real risk is in custom cache behaviour, request forgery edge cases, and hand-rolled framework integrations.

If you’re running a custom AI integration today — your own OpenAI service class, your own retry decorator, your own token-counting middleware — the migration question is not “should I switch to the AI SDK?” It’s “how quickly can I make the switch before the ecosystem diverges enough that porting gets expensive?” For token tracking and rate limiting specifically, see Laravel AI Middleware: Token Tracking & Rate Limiting — that pattern ports cleanly onto the SDK’s event hooks.

Upgrade Checklist: AI-Specific Steps for Laravel 13

Before you run composer require laravel/framework:^13.0, address the following:

1. PHP version. Laravel 13 requires PHP 8.3 as a minimum. PHP 8.5, released November 2025, is also supported by Laravel 13, bringing further JIT improvements and native URI handling. Check your server runtime before anything else.

2. Remove redundant provider SDKs. If you’re injecting openai-php/client or anthropic/anthropic-sdk-php directly via the Service Container, evaluate whether the AI SDK replaces that dependency entirely. In most cases, it does. Keeping both creates competing abstraction layers.

3. Audit your queue configuration. With Queue::route() now available, centralise your AI job routing in AppServiceProvider. If you’re currently setting public string $queue = 'ai' on individual job classes, that works — but Queue::route() is cleaner and easier to update without touching job classes.

4. Generate and store your config/ai.php. The SDK expects this file. Publish it with:

php artisan vendor:publish --tag=ai-config

Then pin your model names explicitly rather than relying on package defaults. Model defaults change between SDK minor releases. You want to control when your application picks up a new default model — not discover that it happened because a patch updated the SDK.

5. Add pgvector to your Postgres instance if you plan to use whereVectorSimilarTo. The extension is available in RDS, Supabase, and Neon without additional configuration. On self-managed Postgres, install it with:

//sql
CREATE EXTENSION IF NOT EXISTS vector;

6. Test your request forgery protection. Laravel 13 formalises PreventRequestForgerywith origin-aware verification on top of token-based CSRF. If you have custom CSRF handling or API routes with unusual origin configurations, test them explicitly before deploying.

What’s Not Yet There

Let’s be direct about the gaps.

Streaming responses — where the model outputs tokens progressively rather than in a single payload — are not yet a first-class concern in the AI SDK’s stable release. For chat interfaces that need token streaming, you’ll still be reaching for provider-specific solutions or custom SSE handling. Watch the 13.x changelog; this is an obvious next primitive.

Multi-modal input — sending images or audio to the model rather than from it — is also not yet documented in the stable SDK surface area. It’s likely coming in a 13.x release, but don’t plan around it until it ships.

The vector search integration is Postgres-only for now. If you’re on MySQL or MariaDB, whereVectorSimilarTois not available. For those stacks, external vector stores (Pinecone, Qdrant, Weaviate) remain the path, and you’ll need your own integration layer.

Final Thoughts

Laravel 13 is not a rewrite. It does not break your application. What it does is establish a new baseline for what “standard Laravel AI work” looks like — and that baseline is considerably higher than it was under Laravel 12. The teams that adopt this now and build their AI features against SDK contracts rather than raw provider clients will have significantly less migration debt when Laravel 14 arrives and extends these primitives further.

Get on PHP 8.3, publish config/ai.php, and start moving your AI layer onto the SDK. The upgrade cost is low. The cost of not doing it compounds.

For the official release notes and full SDK documentation, see the Laravel 13 Release Notes and the Laravel AI SDK documentation directly.

The Complete Guide to Integrating Claude API with Laravel

Dewald Hugo — Sun, 15 Feb 2026 08:18:16 +0000

Most Laravel developers hit the same wall when integrating Claude: the official docs focus on JavaScript, most community guides assume Python, and Laravel tutorials stop at toy chatbots that break under real load.

This guide shows how Claude fits into a production Laravel 11 codebase—how to structure the integration cleanly and why certain decisions matter once AI becomes a dependency instead of an experiment.

We’ll build in layers. Each part expands the system, explains why a pattern exists, and walks through the code. By the end, you’ll understand not just how to call Claude, but how to reason about AI integration the same way you reason about databases, queues, and external services.

Stack: Laravel 11, Claude Messages API, raw HTTP (no SDK).

Why Integrate Claude with Laravel
Claude is designed for longer reasoning chains, clearer structured output, and safer handling of ambiguous prompts. This makes it well-suited to applications where AI output is part of a workflow, not a novelty feature.

Laravel provides exactly the tooling required to manage such workflows: service containers, background jobs, caching, and structured error handling. AI interactions become first-class application concerns.

Rather than building a single-purpose chatbot, we’ll design a Claude integration that supports synchronous requests, streaming output, conversational memory, and background processing. Each pattern builds on the same foundation.

Part 1: Foundation

Authentication and API Setup
Claude uses straightforward API key authentication—no token exchange or refresh mechanism. This simplifies integration but puts responsibility on you to manage access carefully.

Store the API key in environment configuration:

CLAUDE_API_KEY=sk-ant-…
CLAUDE_API_VERSION=2023-06-01

Laravel’s config system surfaces these cleanly:

// config/claude.php
return [
    'api_key' => env('CLAUDE_API_KEY'),
    'version' => env('CLAUDE_API_VERSION', '2023-06-01'),
    'base_url' => 'https://api.anthropic.com/v1/messages',
];

This centralizes Claude-specific configuration and makes testing and environment overrides trivial without touching application code.

Laravel Service Architecture
Don’t call the Claude API directly from controllers. Create a dedicated service:

php artisan make:class Services/ClaudeService
namespace App\Services;
use Illuminate\Support\Facades\Http;
class ClaudeService
{
    public function send(array $messages, array $options = []): array
    {
        $response = Http::withHeaders([
            'x-api-key' => config('claude.api_key'),
            'anthropic-version' => config('claude.version'),
            'content-type' => 'application/json',
        ])->post(config('claude.base_url'), [
            'model' => $options['model'] ?? 'claude-3-5-sonnet-latest',
            'messages' => $messages,
            'max_tokens' => $options['max_tokens'] ?? 1024,
        ]);
        if ($response->failed()) {
            throw new \RuntimeException(
                'Claude API error: ' . $response->body()
            );
        }
        return $response->json();
    }
}

This service becomes the single point of truth for Claude communication.

Error Handling Strategy
AI APIs fail differently from traditional services. Errors may represent transient capacity issues rather than application bugs. Error handling should prioritize resilience and observability.

Wrap calls in try/catch blocks and report exceptions. Log raw API responses for debugging, but never return them directly to users.

try {
    $result = $claude->send($messages);
} catch (\Throwable $e) {
    report($e);
    return response()->json([
        'error' => 'AI service temporarily unavailable'
    ], 503);
}

Environment Sensitivity
Different environments require different tolerances. Development benefits from low token limits and aggressive timeouts. Production requires higher ceilings.

Externalize these values to tune behavior without redeploying:

CLAUDE_MAX_TOKENS=1024
CLAUDE_TIMEOUT=30

Then:

'timeout' => env('CLAUDE_TIMEOUT', 30),

Part 2: Core Integration Patterns

Once authentication and setup are complete, the challenge isn’t calling Claude—it’s deciding where that call should live, how it should fail, and how the rest of your application should interact with it.

Designing a Claude Service Boundary
The most common mistake is letting API calls leak into controllers, jobs, or Livewire components. That works for prototypes but quickly becomes untestable and fragile.

Treat Claude like any other external dependency: Stripe, S3, or an internal microservice. Create a single, dedicated client.

namespace App\Services\Claude;
use Illuminate\Support\Facades\Http;
final class ClaudeClient
{
    public function message(array $payload): array
    {
        return Http::withHeaders([
                'x-api-key' => config('services.claude.key'),
                'anthropic-version' => '2023-06-01',
            ])
            ->post('https://api.anthropic.com/v1/messages', $payload)
            ->throw()
            ->json();
    }
}

This class does one thing: speak Claude’s protocol. It doesn’t know about your application’s domain.

You can swap models without touching business logic, mock the client cleanly in tests, and isolate vendor churn to one file.

Pattern 1: Simple Request–Response Workflows
Most Claude interactions are: generate text, summarize content, rewrite copy, extract data.

Instead of calling the client directly, wrap each use case in a domain-specific service:

final class ArticleSummarizer
{
    public function __construct(
        private readonly ClaudeClient $claude
    ) {}
    public function summarize(string $article): string
    {
        $response = $this->claude->message([
            'model' => 'claude-3-5-sonnet-latest',
            'max_tokens' => 400,
            'messages' => [
                [
                    'role' => 'user',
                    'content' => "Summarize the following article:\n\n{$article}",
                ],
            ],
        ]);
        return $response['content'][0]['text'];
    }
}

From the controller’s perspective:

public function store(Request $request, ArticleSummarizer $summarizer)
{
    $summary = $summarizer->summarize($request->input('content'));
    return response()->json(['summary' => $summary]);
}

Controllers orchestrate. Services decide how Claude is used. The Claude client handles transport.

Pattern 2: Streaming Responses
Once output grows beyond a few hundred tokens, synchronous calls start to feel slow—even if they’re technically fast.

Claude supports streaming responses, and Laravel supports streamed HTTP responses. Your Claude client exposes a streaming method:

public function stream(array $payload, callable $onChunk): void
{
    Http::withHeaders([
            'x-api-key' => config('services.claude.key'),
            'anthropic-version' => '2023-06-01',
        ])
        ->withOptions(['stream' => true])
        ->post('https://api.anthropic.com/v1/messages', $payload)
        ->onChunk(function ($chunk) use ($onChunk) {
            $onChunk($chunk);
        });
}

Then in a controller:

public function generate(Request $request)
{
    return response()->stream(function () use ($request) {
        $this->claude->stream([
            'model' => 'claude-3-5-sonnet-latest',
            'max_tokens' => 2000,
            'messages' => [
                [
                    'role' => 'user',
                    'content' => $request->input('prompt'),
                ],
            ],
        ], function ($chunk) {
            echo $chunk;
            flush();
        });
    });
}

This improves perceived performance and unlocks live writing experiences, progressive document analysis, and token-by-token UI updates.

Claude is still stateless. Streaming doesn’t change memory behavior—only delivery.

Pattern 3: Managing Conversation State
Claude doesn’t remember previous requests. Conversation memory is always an application concern.

final class ConversationResponder
{
    public function __construct(
        private readonly ClaudeClient $claude
    ) {}
    public function respond(array $history, string $input): string
    {
        $messages = [
            ...$history,
            ['role' => 'user', 'content' => $input],
        ];
        $response = $this->claude->message([
            'model' => 'claude-3-5-sonnet-latest',
            'messages' => $messages,
            'max_tokens' => 600,
        ]);
        return $response['content'][0]['text'];
    }
}

Where history comes from is a strategic decision: session storage for ephemeral chats, database for persistent conversations, Redis for fast transient workflows.

Keeping memory outside the client lets you control token growth, privacy boundaries, and cost predictability.

Pattern 4: Background Processing with Queues
Not all Claude interactions belong in HTTP requests. Batch operations, document ingestion, and analysis jobs should run asynchronously.

final class AnalyzeDocument implements ShouldQueue
{
    use Dispatchable, Queueable;
    public function __construct(
        private readonly int $documentId
    ) {}
    public function handle(ClaudeClient $claude): void
    {
        $document = Document::findOrFail($this->documentId);
        $response = $claude->message([
            'model' => 'claude-3-5-sonnet-latest',
            'messages' => [
                [
                    'role' => 'user',
                    'content' => "Extract structured data:\n\n{$document->content}",
                ],
            ],
            'max_tokens' => 1000,
        ]);
        $document->update([
            'analysis' => $response['content'][0]['text'],
        ]);
    }
}

Queues give you retries, failure isolation, and throughput control—all critical when AI latency is variable.

These patterns keep Claude calls predictable, prevent controller bloat, make failures observable, and allow features to grow independently.

Token Accounting: Making AI Costs Predictable
Tokens are a direct function of input size (prompts, conversation history, documents), output length, model choice, and retry behavior. If you don’t account for tokens early, you lose control over cost and performance.

Why Token Accounting Belongs in the Service Layer
Claude’s API returns usage information with each response. That data should never be handled in controllers or UI components. Your Claude client is the correct place to extract and normalize it.

Example response fragment:

{
  "usage": {
    "input_tokens": 812,
    "output_tokens": 436
  }
}
Modify your ClaudeClient to surface this explicitly:

final class ClaudeResponse
{
    public function __construct(
        public readonly string $text,
        public readonly int $inputTokens,
        public readonly int $outputTokens
    ) {}
}

Then adapt the client:

public function message(array $payload): ClaudeResponse
{
    $response = Http::withHeaders([
            'x-api-key' => config('services.claude.key'),
            'anthropic-version' => '2023-06-01',
        ])
        ->post('https://api.anthropic.com/v1/messages', $payload)
        ->throw()
        ->json();
    return new ClaudeResponse(
        text: $response['content'][0]['text'],
        inputTokens: $response['usage']['input_tokens'] ?? 0,
        outputTokens: $response['usage']['output_tokens'] ?? 0,
    );
}

Every Claude interaction becomes measurable.

Propagating Token Data Without Polluting Business Logic
Your domain services should use token data without being tightly coupled to Claude:

final class ArticleSummarizer
{
    public function __construct(
        private readonly ClaudeClient $claude
    ) {}
    public function summarize(string $article): array
    {
        $response = $this->claude->message([
            'model' => 'claude-3-5-sonnet-latest',
            'max_tokens' => 400,
            'messages' => [
                [
                    'role' => 'user',
                    'content' => "Summarize the following article:\n\n{$article}",
                ],
            ],
        ]);
        return [
            'summary' => $response->text,
            'tokens' => [
                'input' => $response->inputTokens,
                'output' => $response->outputTokens,
            ],
        ];
    }
}

This allows you to log usage, display cost hints in admin tools, and enforce limits per request or per user—without leaking Claude-specific details into controllers.

Estimating Cost Before You Call Claude
Pre-flight estimation is one of the most effective control mechanisms. While you can’t know the exact output token count ahead of time, input tokens are deterministic.

A simple heuristic:

final class TokenEstimator
{
    public static function estimateInputTokens(string $text): int
    {
        // Rough heuristic: ~4 characters per token
        return (int) ceil(strlen($text) / 4);
    }
}

Used defensively:

$estimatedTokens = TokenEstimator::estimateInputTokens($document);
if ($estimatedTokens > 12_000) {
    throw new DomainException('Document too large for single Claude request.');
}

This prevents accidental runaway costs and forces intentional chunking strategies.

Tracking Token Usage Over Time
Once token data is available, persisting it is trivial—and invaluable:

Schema::create('ai_usage_logs', function (Blueprint $table) {
    $table->id();
    $table->string('feature');
    $table->integer('input_tokens');
    $table->integer('output_tokens');
    $table->timestamps();
});

Logged at the service boundary:

AiUsageLog::create([
    'feature' => 'article_summary',
    'input_tokens' => $response->inputTokens,
    'output_tokens' => $response->outputTokens,
]);

This gives you per-feature cost visibility, early warning signals for prompt regressions, and data to justify caching or refactoring decisions.

Without token accounting, conversation memory grows unchecked, streaming hides true cost, and background jobs quietly explode your bill.

Read full article here: https://origin-main.com/guides/the-complete-guide-to-integrating-claude-api-with-laravel/