DEV Community

Cover image for Laravel AI Failover Worked. My JSON Contract Didn’t.
Paul-S
Paul-S

Posted on

Laravel AI Failover Worked. My JSON Contract Didn’t.

#laravel #php #ai #testing

A Laravel AI failover can produce a surprisingly successful failure.

09:42:11  primary_provider  429 rate_limit
09:42:12  fallback_provider  200 success
09:42:12  application        Missing key: recommended_action
Enter fullscreen mode Exit fullscreen mode

The primary provider was unavailable. Laravel moved the request to the backup provider, which returned a valid response.

Infrastructure saw success.

The application crashed.

The Prompt Was Pretending to Be a Contract

The agent had been instructed to return JSON:

Return JSON using this structure:

{
  "risk": "low|medium|high",
  "recommended_action": "approve|review|reject",
  "reason": "string"
}
Enter fullscreen mode Exit fullscreen mode

The primary model usually followed that instruction.

The fallback model returned this instead:

{
  "risk_level": "medium",
  "action": "manual_review",
  "explanation": "The account requires additional verification."
}
Enter fullscreen mode Exit fullscreen mode

Nothing about that answer was nonsensical. A person could understand it immediately.

PHP could not.

The parser expected recommended_action. The fallback supplied action. Even its value changed from review to manual_review.

Failover had restored availability without preserving substitutability.

Provider Failover Is Not Contract Failover

The Laravel AI SDK allows an ordered list of providers:

use Laravel\Ai\Enums\Lab;

$response = (new RefundDecision)->prompt(
    'Evaluate refund request RF-1084.',
    provider: [
        Lab::OpenAI,
        Lab::Anthropic,
    ],
);
Enter fullscreen mode Exit fullscreen mode

If the primary provider is rate-limited or unavailable, the SDK can move to the next one.

That answers one question:

Can another provider complete the request?

It does not automatically answer another:

Will every provider behave identically enough for my application?

Providers differ in model behavior, supported tools, configuration options, and structured-output implementations. Laravel hides much of the transport complexity, but the application still owns its business contract.

The Laravel AI SDK’s recent changelog includes provider-specific structured-output fixes. That is a useful reminder that multi-provider compatibility must be tested, not assumed.

Replace JSON Instructions With a Schema

Laravel agents can implement HasStructuredOutput instead of asking for JSON in natural-language instructions.

<?php

namespace App\Ai\Agents;

use Illuminate\Contracts\JsonSchema\JsonSchema;
use Laravel\Ai\Contracts\Agent;
use Laravel\Ai\Contracts\HasStructuredOutput;
use Laravel\Ai\Promptable;

class RefundDecision implements Agent, HasStructuredOutput
{
    use Promptable;

    public function instructions(): string
    {
        return <<<'PROMPT'
        Evaluate the refund request.

        Return a risk level, a recommended action,
        and a short explanation based only on the
        supplied account and policy information.
        PROMPT;
    }

    public function schema(JsonSchema $schema): array
    {
        return [
            'risk' => $schema->string()->required(),
            'recommended_action' => $schema->string()->required(),
            'reason' => $schema->string()->required(),
        ];
    }
}
Enter fullscreen mode Exit fullscreen mode

The response can then be accessed like an array:

$response = (new RefundDecision)->prompt(
    'Evaluate refund request RF-1084.',
    provider: [
        Lab::OpenAI,
        Lab::Anthropic,
    ],
);

$action = $response['recommended_action'];
Enter fullscreen mode Exit fullscreen mode

This is stronger than “please return JSON” because the output structure becomes part of the agent definition.

It still should not be the final validation layer.

Convert Model Output Into a Domain Type

A string can satisfy the schema and still contain an unsupported business value.

The application should reject anything outside its own vocabulary:

enum RefundRisk: string
{
    case Low = 'low';
    case Medium = 'medium';
    case High = 'high';
}

enum RefundAction: string
{
    case Approve = 'approve';
    case Review = 'review';
    case Reject = 'reject';
}

final readonly class RefundResult
{
    public function __construct(
        public RefundRisk $risk,
        public RefundAction $action,
        public string $reason,
    ) {}

    public static function fromAiResponse($response): self
    {
        return new self(
            risk: RefundRisk::from($response['risk']),
            action: RefundAction::from(
                $response['recommended_action']
            ),
            reason: $response['reason'],
        );
    }
}
Enter fullscreen mode Exit fullscreen mode

Now manual_review, approve_now, or an empty value fails before it reaches refund logic.

That failure is useful. It protects the business operation from a response that is valid JSON but invalid for the application.

Test Every Provider as a Primary Provider

A test that always uses the normal provider does not test failover.

Run a small contract suite against each configured provider independently:

$providers = [
    Lab::OpenAI,
    Lab::Anthropic,
];

foreach ($providers as $provider) {
    $response = (new RefundDecision)->prompt(
        'Account is seven days old. Policy requires review.',
        provider: $provider,
    );

    $result = RefundResult::fromAiResponse($response);

    expect($result->risk)->toBeInstanceOf(RefundRisk::class);
    expect($result->action)->toBe(RefundAction::Review);
}
Enter fullscreen mode Exit fullscreen mode

Keep this separate from fast unit tests because it makes real provider calls. Run it in staging, before changing models, and after upgrading the SDK.

Also test what happens when validation fails. The safe behavior might be to retry once, send the request to manual review, or stop the workflow. It should not silently invent a default action.

A Backup Provider Is a Different Dependency

The important lesson is not that Laravel’s failover failed.

It worked exactly where it was supposed to work.

The mistake was treating two probabilistic providers as interchangeable implementations of the same deterministic interface.

If you are evaluating the best Laravel development company for an AI-enabled application, ask how the team tests structured outputs across every fallback provider. At Spaculus Software, provider availability and business-contract validity are treated as separate production concerns.

A 200 OK means the provider answered.

It does not mean your application received an answer it can safely use.

What contract would break first if your Laravel application switched AI providers today?

Top comments (1)

Collapse
 
innokentyb profile image
Kent Bodrov •

I'd add one negative control to the cross-provider suite: a response with all three required keys and valid enum values, but approve for the seven-day-old account that policy says must go to review. It passes the shape and domain-type checks. Does a separate policy gate still stop it before any refund action? Keeping that fixture beside malformed-output cases would distinguish provider compatibility from correctness of the business decision.