DEV Community

Sumeet Shroff
Sumeet Shroff

Posted on • Originally published at mumbaiwebdesigner.com

Using Multiple LLM Providers with the Laravel AI SDK

#laravel #php #ai #llm

The Laravel AI SDK (laravel/ai, v0.8.1) ships with built-in support for 14 AI providers: OpenAI, Anthropic, Google Gemini, Groq, Mistral, DeepSeek, xAI, Ollama, Azure OpenAI, Cohere, OpenRouter, Jina, VoyageAI, and ElevenLabs. That breadth is genuinely useful — but unlocking it correctly requires more than swapping an env variable. This article covers how to configure multiple providers in the same application, switch between them at runtime, handle provider-specific failures, and test the whole setup without hitting a live API.

Prerequisites

  • PHP 8.3+
  • Laravel 12 or 13
  • laravel/ai v0.8 (pin to ^0.8 in composer.json until v1.0 ships — the package is still pre-1.0)
  • At least two provider API keys (we'll use OpenAI and Anthropic for the walkthrough)

Install the SDK if you haven't already:

composer require laravel/ai
php artisan vendor:publish --tag=ai-config
Enter fullscreen mode Exit fullscreen mode

How the SDK Resolves Providers

When you call AI::text()->using('gpt-4o-mini')->..., the SDK maps the model string to a provider using the published config/ai.php. The default config reads a single AI_PROVIDER env variable. That's fine for a simple setup, but in a multi-provider application you need named provider instances instead.

Here is the relevant section of config/ai.php after customisation:

// config/ai.php
'providers' => [
    'openai' => [
        'driver'  => 'openai',
        'api_key' => env('OPENAI_API_KEY'),
    ],
    'anthropic' => [
        'driver'  => 'anthropic',
        'api_key' => env('ANTHROPIC_API_KEY'),
    ],
    'groq' => [
        'driver'  => 'groq',
        'api_key' => env('GROQ_API_KEY'),
    ],
    'openrouter' => [
        'driver'  => 'openrouter',
        'api_key' => env('OPENROUTER_API_KEY'),
    ],
],
Enter fullscreen mode Exit fullscreen mode

And the matching .env entries:

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GROQ_API_KEY=gsk_...
OPENROUTER_API_KEY=sk-or-...
Enter fullscreen mode Exit fullscreen mode

Never prefix these with NEXT_PUBLIC_ — all AI calls must go through the Laravel backend. A frontend that calls an LLM directly exposes your API key to every browser that loads the page.

Selecting a Provider at Call Time

The using() method accepts a model identifier that the SDK resolves to a provider. You can be explicit:

use Illuminate\Support\Facades\AI;

// GPT-4o-mini via OpenAI
$openAiResult = AI::text()
    ->using('gpt-4o-mini')
    ->prompt($userMessage)
    ->generate();

// Claude Sonnet via Anthropic
$claudeResult = AI::text()
    ->using('claude-sonnet-4-5')
    ->prompt($userMessage)
    ->generate();

// Llama 3 via Groq (fast inference)
$groqResult = AI::text()
    ->using('llama3-70b-8192')
    ->prompt($userMessage)
    ->generate();
Enter fullscreen mode Exit fullscreen mode

The SDK looks at the model string and matches it to the correct provider driver configured in config/ai.php. If a model string is ambiguous (e.g. two providers offer a model with the same name), qualify it with a provider prefix:

// Explicit provider prefix avoids ambiguity
$result = AI::text()
    ->using('openai:gpt-4o')
    ->prompt($prompt)
    ->generate();
Enter fullscreen mode Exit fullscreen mode

Building a Provider Router

A common production pattern is to choose the provider based on the task type. Create a simple service class rather than scattering using() strings throughout controllers:

<?php

namespace App\Services;

use Illuminate\Support\Facades\AI;

class AiProviderRouter
{
    /**
     * Fast, cheap completion — use Groq for low-latency tasks.
     */
    public function fast(string $prompt): string
    {
        return AI::text()
            ->using('llama3-70b-8192') // served by Groq
            ->prompt($prompt)
            ->generate()
            ->text();
    }

    /**
     * High-quality reasoning — use GPT-4o for complex tasks.
     */
    public function precise(string $prompt): string
    {
        return AI::text()
            ->using('gpt-4o')
            ->prompt($prompt)
            ->generate()
            ->text();
    }

    /**
     * Long-context summarisation — Anthropic handles large contexts well.
     */
    public function summarise(string $document): string
    {
        return AI::text()
            ->using('claude-sonnet-4-5')
            ->prompt('Summarise the following in 5 bullet points: ' . $document)
            ->generate()
            ->text();
    }
}
Enter fullscreen mode Exit fullscreen mode

Bind this in a service provider and inject it wherever needed. The approach keeps provider decisions out of controllers and makes them testable in isolation.

Provider Failover: Catching Provider Errors

The SDK throws distinct exceptions for provider-side failures:

  • Laravel\AI\Exceptions\RateLimitedException — HTTP 429 from the provider
  • Laravel\AI\Exceptions\ProviderOverloadedException — HTTP 5xx from the provider

You can catch these to implement fallback logic:

use Laravel\AI\Exceptions\RateLimitedException;
use Laravel\AI\Exceptions\ProviderOverloadedException;
use Illuminate\Support\Facades\AI;
use Illuminate\Support\Facades\Log;

function generateWithFallback(string $prompt): string
{
    $providers = [
        'gpt-4o-mini',           // primary: OpenAI
        'claude-haiku-3-5',      // fallback 1: Anthropic
        'llama3-70b-8192',       // fallback 2: Groq
    ];

    foreach ($providers as $model) {
        try {
            return AI::text()
                ->using($model)
                ->prompt($prompt)
                ->generate()
                ->text();
        } catch (RateLimitedException | ProviderOverloadedException $e) {
            Log::warning('AI provider unavailable, trying next', [
                'model' => $model,
                'error' => $e->getMessage(),
            ]);
            continue;
        }
    }

    throw new \RuntimeException('All AI providers failed.');
}
Enter fullscreen mode Exit fullscreen mode

Handle both exception types explicitly — a 429 (rate limit) and a 503 (overload) have different implications for retry strategy. In a queue-based system, you may want to release the job back to the queue with exponential backoff instead of falling over to a different provider immediately.

OpenRouter as a Multi-Model Aggregator

OpenRouter is a single API endpoint that routes to 100+ models. It is worth understanding the trade-off before using it:

Advantages:

  • One API key, one billing dashboard, access to models from OpenAI, Anthropic, Mistral, Meta, and dozens of others
  • Automatic cost-based routing: configure a budget and OpenRouter picks the cheapest capable model
  • Useful for prototyping without juggling multiple provider accounts

Disadvantages:

  • Per-token pricing markup compared to direct provider calls
  • An extra network hop adds latency
  • Provider-specific features (e.g. OpenAI's function calling schema format vs Anthropic's tool use format) are abstracted away — if you rely on provider-specific behaviour, OpenRouter may not expose it

To add OpenRouter as a named provider:

// config/ai.php
'openrouter' => [
    'driver'  => 'openrouter',
    'api_key' => env('OPENROUTER_API_KEY'),
],
Enter fullscreen mode Exit fullscreen mode 
$result = AI::text()
    ->using('openrouter:anthropic/claude-3-haiku')
    ->prompt($prompt)
    ->generate();
Enter fullscreen mode Exit fullscreen mode

For the main Laravel AI SDK: Complete Guide to Building AI Applications covering agents, embeddings, MCP, and structured output, that pillar article has broader coverage of the full SDK surface area.

Provider-Specific Capabilities: What Each Provider Supports

Not every provider supports every SDK feature. Before routing a task to a provider, verify capability:

Feature OpenAI Anthropic Groq Gemini Mistral
Text generation Yes Yes Yes Yes Yes
Tool calling (agents) Yes Yes Yes Yes Yes
Structured output Yes Yes Limited Yes Yes
Image generation Yes (DALL-E) No No Yes No
Audio transcription Yes (Whisper) No Yes No No
Embeddings Yes No No Yes Yes

This matters because if you write an agent that uses image generation and route it to Anthropic, the SDK will throw. Build capability checks into your router if you support a mix of features:

// Pseudocode: route based on task type
public function routeModel(string $taskType): string
{
    return match($taskType) {
        'image'         => 'dall-e-3',         // OpenAI only
        'transcription' => 'whisper-1',        // OpenAI only
        'embeddings'    => 'text-embedding-3-small', // OpenAI only
        'fast-chat'     => 'llama3-70b-8192',  // Groq
        default         => 'gpt-4o-mini',
    };
}
Enter fullscreen mode Exit fullscreen mode

Testing Multi-Provider Logic Without API Calls

The SDK ships with test fakes that prevent any network call from leaving your test suite:

use Laravel\AI\Fakes\TextFake;
use Illuminate\Support\Facades\AI;

public function test_provider_router_uses_groq_for_fast_tasks(): void
{
    AI::fake([
        'text' => new TextFake('Mocked fast response'),
    ]);

    $router = new AiProviderRouter();
    $result = $router->fast('What is 2 + 2?');

    $this->assertSame('Mocked fast response', $result);
}

public function test_fallback_triggers_on_rate_limit(): void
{
    // Simulate RateLimitedException on first call, success on second
    AI::fake([
        'text' => new TextFake(
            responses: ['Fallback response'],
            throwOnFirst: new \Laravel\AI\Exceptions\RateLimitedException(),
        ),
    ]);

    $result = generateWithFallback('Test prompt');
    $this->assertSame('Fallback response', $result);
}
Enter fullscreen mode Exit fullscreen mode

This is the correct approach — never let tests hit live providers. Tests that call real APIs are slow, cost money, and fail non-deterministically when the provider is down.

Common Mistakes

Storing provider keys in config() calls inside controllers. All API keys must live in .env, referenced via config/ai.php using env(). The config() helper is the only correct access point in application code.

Expecting every provider to behave identically. The SDK abstracts the API surface, but output quality, token limits, response latency, and pricing are provider-specific. A prompt that works well on GPT-4o may produce weaker output from a smaller Groq model. Test with your actual prompts.

Not pinning the minor version. laravel/ai is at v0.8.x and has not reached v1.0. The package can introduce breaking changes in minor versions before the stable release. Use ^0.8 in composer.json, not ^0 or *.

Running multi-provider fallback in the HTTP request cycle. If the first provider times out after 30 seconds and you try two fallbacks, you're looking at a 90-second request. Queue AI work as jobs. Return a job ID to the client and poll for the result.

Assuming Ollama (local) has the same performance characteristics as hosted providers. Ollama is useful for development and privacy-sensitive workloads, but inference speed depends on local hardware. Do not include Ollama in a production fallback chain unless you have the GPU resources to back it up.

Configuring Providers Per Environment

A common requirement is to use a cheap local model during development (to avoid API costs) and a hosted provider in production. You can handle this cleanly by conditionally loading provider configuration based on the app environment:

// config/ai.php
'providers' => [
    'primary' => [
        'driver'  => env('AI_DRIVER', 'openai'),
        'api_key' => env('AI_API_KEY'),
    ],
],
Enter fullscreen mode Exit fullscreen mode

Then in your environment files:

# .env (local development — Ollama, no API cost)
AI_DRIVER=ollama
AI_API_KEY=

# .env.production
AI_DRIVER=openai
AI_API_KEY=sk-...
Enter fullscreen mode Exit fullscreen mode

This keeps your using() calls model-agnostic in lower environments and avoids accidentally sending real user data to a third-party provider during local development. Ollama runs entirely on your machine — no network egress, no billing exposure.

Note that Ollama's performance depends entirely on local hardware. On a development MacBook with CPU-only inference, a model that responds in 0.3 seconds on Groq might take 8–15 seconds locally. This is expected and acceptable for development, but do not let it shape your expectations of hosted inference speed.

Cost Control Across Providers

When running multiple providers in production, API spend tracking becomes more complex because you have multiple billing dashboards. A few practical measures:

Set hard monthly spend limits at the provider dashboard level. Every major provider (OpenAI, Anthropic, Groq) allows you to configure billing alerts and hard caps. Do this before enabling any provider in production — an agent loop with maxSteps() not set can exhaust a spend limit in minutes.

Apply Laravel rate limiting at the route or queue level. The SDK does not meter your own spending — that is your application's responsibility:

use Illuminate\Support\Facades\RateLimiter;
use Illuminate\Cache\RateLimiting\Limit;

RateLimiter::for('ai-per-user', function ($request) {
    return Limit::perMinute(10)->by($request->user()->id);
});
Enter fullscreen mode Exit fullscreen mode

This throttles requests before they reach the provider, protecting your spend budget from a single user making too many requests.

Log every provider call with model, token count, and latency. Laravel's logging pipeline makes this straightforward with a listener or middleware on the AI facade. Aggregated over a week, this data tells you which providers are over-used and whether the cost-to-quality trade-off of your routing decisions is holding up.

Limitations to Know

Streaming is not fully first-class in v0.8. Token-by-token streaming is not a built-in SDK feature at this version. If you need character-by-character output in a chat UI, you need custom SSE handling, Laravel Reverb, and provider-specific streaming code on top of the SDK. This is a known gap and likely to be addressed before v1.0, but plan for the extra work if streaming is a core product requirement.

Framework version constraints are strict. The laravel/ai package requires illuminate/support ^12.62|^13.15. If your Laravel 12 installation predates the 12.62 patch release, you must update the framework first before the SDK will resolve cleanly. Run composer update laravel/framework before adding laravel/ai if you are on an older patch version.

PHP 8.2 and below are unsupported. The package uses PHP 8.3 features. If you are migrating from Laravel 10 or 11, a PHP upgrade is a prerequisite. Check your hosting environment's available PHP versions before planning the migration.

The AWS SDK conflict. laravel/ai depends on aws/aws-sdk-php ^3.339. If your project pins an older AWS SDK version (for example, because another package requires it), Composer will flag a dependency conflict. Resolve this by updating the AWS SDK across all dependent packages simultaneously, or by using the openrouter driver as a workaround that avoids the AWS dependency.

Summary

The Laravel AI SDK makes multi-provider support a configuration and call-site concern rather than an architectural one. Name your providers in config/ai.php, route to them via using(), catch RateLimitedException and ProviderOverloadedException for failover, and use the built-in fakes for testing. Keep AI work in queued jobs, set maxSteps() on any agents that call tools, and pin to ^0.8 until v1.0 ships.

If you need Laravel development in Mumbai, Mumbai Web Designer builds production-grade Laravel applications.

Top comments (0)