Preventing Cascading Failures: Circuit Breakers in Laravel 🛑

#laravel #php #microservices #architecture

The Anatomy of a Cascading Failure

In a distributed microservices architecture, your system is only as resilient as its weakest network link. Imagine your primary Laravel application acting as an API Gateway. When a user requests their dashboard, your Laravel app makes synchronous HTTP calls to three separate microservices: the Billing Service, the Recommendation Service, and the Inventory Service.

What happens if the Recommendation Service goes down or experiences a severe database lock, causing its response time to jump from 50 milliseconds to 30 seconds? Your Laravel application will dutifully wait for 30 seconds. If 1,000 users request their dashboard, you suddenly have 1,000 PHP-FPM workers hanging, completely blocked, waiting for a dead service. Within minutes, your server runs out of available RAM and connections. The entire Laravel gateway crashes. Because the gateway is dead, the Billing and Inventory services are now unreachable as well.

A non-critical failure in a tertiary service (Recommendations) has just caused a catastrophic, system-wide outage. This is known as a Cascading Failure.

At Smart Tech Devs, we build self-healing infrastructure. We isolate network degradation and prevent total system collapse by implementing the Circuit Breaker Pattern for all inter-service communication.

The Philosophy of the Circuit Breaker

Borrowed from electrical engineering, a software Circuit Breaker acts as a state machine that sits between your Laravel application and the external service. It monitors the failure rate of outgoing HTTP requests and transitions between three absolute states:

Closed (Healthy): The network is fine. Requests flow through normally. The breaker counts any timeouts or 500 errors.
Open (Tripped): If the failure rate exceeds a specific threshold (e.g., 5 failures in 10 seconds), the circuit physically "opens." All subsequent requests to the dead service are instantly rejected by the breaker in 1 millisecond. We stop making network requests entirely, giving the downstream service time to recover and protecting our own PHP workers from hanging.
Half-Open (Testing): After a predefined cooldown period (e.g., 30 seconds), the breaker lets a single "probe" request pass through. If it succeeds, the circuit closes. If it fails, it violently snaps open again.

Phase 1: Architecting the Redis-Backed State Machine

To implement this in Laravel, the state of the Circuit Breaker must be shared across all PHP workers concurrently. We architect this using Redis.


namespace App\Services\Resilience;

use Illuminate\Support\Facades\Redis;
use Exception;

class CircuitBreaker
{
    private string $serviceName;
    private int $failureThreshold;
    private int $cooldownSeconds;

    public function __construct(string $serviceName, int $failureThreshold = 5, int $cooldownSeconds = 30)
    {
        $this->serviceName = "circuit_breaker:{$serviceName}";
        $this->failureThreshold = $failureThreshold;
        $this->cooldownSeconds = $cooldownSeconds;
    }

    public function isAvailable(): bool
    {
        $state = Redis::get("{$this->serviceName}:state");

        if ($state === 'OPEN') {
            $lastFailure = Redis::get("{$this->serviceName}:last_failure");
            
            // Check if the cooldown period has expired (Transition to Half-Open)
            if (time() - $lastFailure > $this->cooldownSeconds) {
                Redis::set("{$this->serviceName}:state", 'HALF_OPEN');
                return true; // Allow one probe request through
            }
            return false;
        }

        return true; // CLOSED or HALF_OPEN
    }

    public function recordSuccess(): void
    {
        // Reset the breaker to a pristine state
        Redis::del("{$this->serviceName}:failures");
        Redis::set("{$this->serviceName}:state", 'CLOSED');
    }

    public function recordFailure(): void
    {
        $failures = Redis::incr("{$this->serviceName}:failures");
        Redis::expire("{$this->serviceName}:failures", $this->cooldownSeconds * 2);

        if ($failures >= $this->failureThreshold) {
            // Trip the breaker
            Redis::set("{$this->serviceName}:state", 'OPEN');
            Redis::set("{$this->serviceName}:last_failure", time());
        }
    }
}

Phase 2: The Service Wrapper and Fallbacks

Now, we build a wrapper around Laravel's native HTTP facade. When we call an external microservice, we wrap the execution in our Circuit Breaker logic.

Crucially, when the circuit is OPEN, we do not throw a fatal 500 error to the user. We implement Graceful Degradation by returning a fallback response (e.g., cached data, or simply omitting the recommendation section of the UI).


namespace App\Services;

use App\Services\Resilience\CircuitBreaker;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Log;

class RecommendationService
{
    private CircuitBreaker $breaker;

    public function __construct()
    {
        // Trip if we hit 3 timeouts within the window
        $this->breaker = new CircuitBreaker('recommendation_api', 3, 45); 
    }

    public function getUserRecommendations(int $userId): array
    {
        // 1. Instant Rejection. Protect the Laravel Worker!
        if (!$this->breaker->isAvailable()) {
            Log::warning("Recommendation API Circuit is OPEN. Returning fallback data.");
            return $this->getFallbackRecommendations();
        }

        try {
            // 2. Attempt the network call with a strict timeout
            $response = Http::timeout(2)->get("http://recommendation-service.internal/api/users/{$userId}");

            if ($response->successful()) {
                $this->breaker->recordSuccess();
                return $response->json();
            }

            // 500 level errors from the microservice count as failures
            if ($response->serverError()) {
                $this->breaker->recordFailure();
            }

            return $this->getFallbackRecommendations();

        } catch (\Exception $e) {
            // 3. Network Timeouts violently trigger failure counts
            $this->breaker->recordFailure();
            Log::error("Recommendation API Timeout: " . $e->getMessage());
            
            return $this->getFallbackRecommendations();
        }
    }

    private function getFallbackRecommendations(): array
    {
        // Return generic, pre-computed recommendations so the UI doesn't break
        return [
            ['id' => 101, 'title' => 'Popular Item 1'],
            ['id' => 102, 'title' => 'Trending Now'],
        ];
    }
}

The Engineering ROI and Autonomous Healing

Architecting Circuit Breakers transforms your enterprise infrastructure from a fragile house of cards into an autonomous, self-healing organism. By instantly rejecting traffic to degraded services, you completely isolate failures to their specific domain. Your primary API gateways remain lightning-fast and perfectly stable, ensuring that critical workflows (like processing payments) are entirely immune to the failure of non-critical systems. Paired with graceful UI degradation, your users will rarely even notice that a microservice is currently experiencing an outage in the background.

DEV Community