Mastering Redis Queues: Laravel Horizon Architecture

The Asynchronous Black Box

When building enterprise applications at Smart Tech Devs, you quickly realize that executing everything synchronously is impossible. Generating PDFs, sending bulk emails, and processing Stripe webhooks must be offloaded to background queues.

The standard Laravel queue system (using php artisan queue:work) is powerful, but it operates as a complete black box. When you have 10,000 jobs in the queue, how do you know how fast they are processing? How do you monitor job failure rates in real-time? How do you scale workers based on the queue length without writing custom bash scripts? Operating Redis queues without visibility is a severe operational liability. To gain enterprise-grade control, you must deploy Laravel Horizon.

The Solution: Horizon Dashboard & Auto-Scaling

Laravel Horizon is a stunning, official dashboard and code-driven configuration system for your Redis queues. It completely replaces standard queue workers.

Instead of manually starting individual worker processes via Supervisor, you start a single Horizon master process. Horizon then reads your config/horizon.php file and automatically spawns, monitors, and terminates worker processes based on your exact specifications. It provides a real-time UI to view throughput (jobs per minute), runtime metrics, and failed job stack traces.

Architecting Auto-Scaling Worker Pools

The true power of Horizon is its ability to auto-scale worker processes dynamically based on the length of the queue.

// config/horizon.php

'environments' => [
    'production' => [
        'supervisor-1' => [
            'connection' => 'redis',
            'queue' => ['high', 'default', 'emails'],
            
            // 1. ✅ THE ENTERPRISE PATTERN: Auto-Scaling
            // Horizon will dynamically scale the number of processes between min and max
            'balance' => 'auto',
            'minProcesses' => 1,
            'maxProcesses' => 20,
            
            // 2. Define exactly how many processes each queue gets under load
            'balanceMaxShift' => 1,
            'balanceCooldown' => 3,
            
            'tries' => 3,
        ],
        
        // 3. Isolate heavy, slow jobs into their own dedicated supervisor
        'supervisor-reports' => [
            'connection' => 'redis',
            'queue' => ['heavy-reports'],
            'balance' => 'simple',
            'processes' => 5,
            'timeout' => 300, // 5 minutes max runtime
        ],
    ],
],

Securing the Dashboard

Because Horizon exposes sensitive operational data and allows you to retry failed jobs, it must be secured in production. You define authorization logic in the HorizonServiceProvider.

// app/Providers/HorizonServiceProvider.php

use Illuminate\Support\Facades\Gate;

protected function gate(): void
{
    Gate::define('viewHorizon', function ($user) {
        // Only allow users with the 'admin' role or specific emails to view the dashboard
        return in_array($user->email, [
            '[e-mail]',
            '[e-mail]'
        ]);
    });
}

The Engineering ROI

By migrating your Redis queues to Laravel Horizon, you transform a blind backend process into a highly observable, self-healing system. You eliminate the need for complex Supervisor scaling scripts, drastically reduce infrastructure costs by only spinning up workers when the queue is actually full, and empower your DevOps team to monitor and recover failed jobs directly from a premium UI.