Laravel Horizon Won't Save You From Long Jobs. Use This Splitting Pattern.

• Book: Decoupled PHP — Clean and Hexagonal Architecture for Applications That Outlive the Framework
• Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
• My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
• Me: xgabriel.com | GitHub

---

You ship Horizon. The dashboard is green. Throughput looks fine. Somewhere, a GenerateMonthlyReports job has been running for 47 minutes. Nobody noticed because Horizon doesn't show mid-job progress, only completed jobs. When the worker finally OOMs at minute 51, Horizon marks it failed. You retry. It runs another 47 minutes and dies again.

This isn't a Horizon bug. Horizon was built for a queue of many short jobs. The fix isn't a config flag. It's a pattern that lives in your job class.

What Horizon shows and what it doesn't

Horizon's dashboard, since Laravel 8, shows you:

• Jobs per minute, per supervisor.
• Runtime per job (after the job finishes).
• p95 / p99 wait times.
• Failed jobs with stack traces.
• Memory usage per worker process.

What it doesn't show:

• Progress inside a running job. A job that's been processing for 40 minutes looks identical to a job that started 40 seconds ago.
• Whether a "running" job is making forward progress or stuck on a single slow DB call.
• A meaningful retry estimate for jobs that take longer than your --timeout.

The Pending Jobs widget counts jobs in the queue. It does not count work units inside a single job. A job that internally iterates over 200,000 PDF generations is one row in Horizon. From Horizon's perspective, your queue is healthy.

This blind spot bites the hardest when the job is something like:

• A monthly PDF export for every active customer.
• A CSV import that runs firstOrCreate per row.
• A vector embedding backfill that hits an external API per document.
• Image resizing for an entire S3 bucket.

All of these have the same shape: one job, thousands of internal iterations, hours of wall time, zero visibility.

The 47-minute silent failure pattern

Picture a finance app that generates a PDF statement for every customer at month-end. The naive version looks reasonable:

<?php

namespace App\Jobs;

use App\Models\Customer;
use App\Services\StatementPdfGenerator;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;

class GenerateMonthlyStatements implements ShouldQueue
{
    use Dispatchable, Queueable;

    public int $timeout = 3600; // an hour, should be enough?

    public function handle(StatementPdfGenerator $generator): void
    {
        Customer::active()->chunk(100, function ($customers) use ($generator) {
            foreach ($customers as $customer) {
                $generator->generate($customer, now()->subMonth());
            }
        });
    }
}

There are 180,000 active customers. Each PDF takes ~15ms. That's 45 minutes of work in one job. Horizon's worker hits a memory ceiling somewhere around customer 140,000 because the chunk() closure holds references through DB query log entries that nobody cleared. The job dies. Retry replays from customer 1.

Horizon's UI never told you anything was wrong until the OOM. There's no progress bar. The job's row just said "running" the whole time.

You can patch the symptoms (DB::disableQueryLog(), raise the worker memory, increase the timeout) but the underlying problem is the job is too big a unit of work.

Pattern: split with Bus::batch()

The right unit of work for a queue isn't "everything to do." It's a chunk small enough that a worker can finish it in well under 60 seconds. Laravel's Bus::batch() (since 8.x) gives you exactly that: a parent batch that owns N child jobs, with hooks for completion, failure, and progress.

The dispatcher job becomes trivial. Its only job is to slice the work:

<?php

namespace App\Jobs;

use App\Models\Customer;
use Illuminate\Bus\Batch;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Support\Facades\Bus;
use Illuminate\Support\Facades\Redis;
use Throwable;

class DispatchStatementBatch implements ShouldQueue
{
    use Dispatchable, Queueable;

    public function __construct(public string $period) {}

    public function handle(): void
    {
        $chunkSize = 250;
        $batchKey  = "statements:batch:{$this->period}";

        $jobs = Customer::active()
            ->select('id')
            ->cursor() // not chunk(), we don't want to hold rows
            ->chunk($chunkSize)
            ->map(fn ($ids) => new GenerateStatementChunk(
                $ids->pluck('id')->all(),
                $this->period,
            ));

        $batch = Bus::batch($jobs)
            ->name("statements:{$this->period}")
            ->allowFailures() // one bad customer shouldn't kill the rest
            ->onQueue('statements')
            ->progress(function (Batch $batch) use ($batchKey) {
                // fires after every child job completes
                Redis::set("{$batchKey}:progress", $batch->progress());
                Redis::set("{$batchKey}:processed", $batch->processedJobs());
                Redis::set("{$batchKey}:total", $batch->totalJobs);
            })
            ->then(fn (Batch $batch) => Redis::set("{$batchKey}:status", 'done'))
            ->catch(function (Batch $batch, Throwable $e) use ($batchKey) {
                Redis::set("{$batchKey}:status", 'failed');
                Redis::set("{$batchKey}:error", $e->getMessage());
            })
            ->finally(fn (Batch $batch) => Redis::expire($batchKey, 86400 * 7))
            ->dispatch();

        Redis::set("{$batchKey}:id", $batch->id);
        Redis::set("{$batchKey}:status", 'running');
    }
}

The child job is small and stateless:

<?php

namespace App\Jobs;

use App\Models\Customer;
use App\Services\StatementPdfGenerator;
use Illuminate\Bus\Batchable;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;

class GenerateStatementChunk implements ShouldQueue
{
    use Batchable, Dispatchable, Queueable;

    public int $timeout = 45;
    public int $tries   = 3;
    public int $backoff = 15;

    public function __construct(
        public array  $customerIds,
        public string $period,
    ) {}

    public function handle(StatementPdfGenerator $generator): void
    {
        // bail early if the batch was cancelled from the UI
        if ($this->batch()?->cancelled()) {
            return;
        }

        Customer::query()
            ->whereIn('id', $this->customerIds)
            ->cursor()
            ->each(fn (Customer $c) => $generator->generate($c, $this->period));
    }
}

A few details worth flagging:

• Batchable is the trait that wires $this->batch() and lets Horizon group these in the UI.
• cursor() instead of get() means we don't allocate 250 hydrated models at once. For 250 customers it barely matters; at chunks of 1,000 it stops the OOM.
• allowFailures() is the difference between "one corrupted customer kills the run" and "we generate 179,997 of 180,000 statements and inspect the 3 failures."
• The child timeout is 45 seconds, not 3600. If a chunk legitimately takes longer, the chunk is too big.

Chunk size math: IO-bound vs CPU-bound

The right chunk size depends on what the work is doing. Two rules of thumb that hold up in production:

CPU-bound chunks (PDF rendering with DomPDF, image resizing with Intervention, encryption): target a job runtime around 10 to 20 seconds. PHP's per-request memory grows roughly linearly with iteration count for these. A chunk of 250 PDFs at 15ms each is ~3.7s. Comfortable, leaves margin for slow customers.

IO-bound chunks (external API calls, slow DB queries): target a job runtime around 20 to 40 seconds, with a smaller chunk size. Each iteration's latency is unpredictable. A chunk of 50 OpenAI embedding requests at p95 800ms each is 40s worst-case. Right at the edge. If the API has a rate limit, your chunk size should be smaller than the per-minute quota so the chunk finishes before the bucket refills.

A useful sanity check: divide your job timeout by 4. If a chunk's typical runtime is below that quarter, you have headroom for occasional slow iterations. If it's above, your chunk is too big.

One gotcha: Bus::batch() writes a row to the job_batches table per child job's progress update. At 720 children (180k / 250) this is fine. At 72,000 children it becomes the bottleneck. The table is hot and every progress callback updates it. If your fan-out is >10,000 jobs, either coarsen the chunk size or use the Redis batch driver in Laravel 11+ ('queue.batching.driver' => 'redis') which keeps batch metadata in Redis instead of MySQL.

Progress tracking with Redis + Bus::batch()->progress()

The progress() callback fires after every successful child. That's the hook for a single Redis key the rest of your app can read:

// in a controller, expose progress to the admin UI
public function show(string $period)
{
    $key = "statements:batch:{$period}";

    return [
        'status'    => Redis::get("{$key}:status") ?? 'unknown',
        'processed' => (int) Redis::get("{$key}:processed"),
        'total'     => (int) Redis::get("{$key}:total"),
        'percent'   => (int) Redis::get("{$key}:progress"),
        'error'     => Redis::get("{$key}:error"),
    ];
}

You can render this as a live progress bar in your admin panel. Five lines of Alpine.js polling the endpoint every 2 seconds and you have something Horizon's dashboard can't give you: real progress inside the batch.

$batch->progress() returns an integer percent based on processedJobs / totalJobs. It counts failures as processed (which is what you want; the batch is moving forward, just not perfectly). If you need to distinguish, $batch->failedJobs is also on the object.

The retry semantics change when you split

This is the part that catches teams off-guard. With a single monolithic job, $tries = 3 meant the entire statement generation retried from scratch. With a batch of 720 chunks, $tries = 3 applies per child. Math:

• Single job, 3 tries → 3 full reruns on failure.
• 720 children, 3 tries each → up to 720 × 3 = 2,160 attempts in the queue, but each only retries its own 250 customers.

This is almost always what you want. A transient network blip kills one chunk; that chunk retries; the other 719 are unaffected. But it changes how you think about idempotency. The child job MUST be idempotent. Running GenerateStatementChunk twice for the same [customer_ids, period] pair must produce the same result, not two PDF rows.

The PDF generator does this with an upsert:

public function generate(Customer $customer, string $period): void
{
    $path = "statements/{$period}/{$customer->id}.pdf";

    if (Storage::disk('s3')->exists($path)) {
        return; // already generated, retry is safe
    }

    $pdf = $this->renderer->render($customer, $period);
    Storage::disk('s3')->put($path, $pdf);

    Statement::updateOrCreate(
        ['customer_id' => $customer->id, 'period' => $period],
        ['s3_path' => $path, 'generated_at' => now()],
    );
}

Without that early exists() check, a retry would regenerate every PDF in the chunk. Wasted work but harmless. With it, retries are cheap. Either way, idempotency is now a property of the child job. It wasn't with the monolith because the monolith never partially completed.

Also worth knowing: allowFailures() means a failed child does NOT abort the batch. The batch's then callback only fires if everything succeeded; the catch callback fires once per failure but the batch keeps running other children. If you want hard-fail-on-first-error semantics, drop allowFailures() and the batch cancels remaining children.

Horizon dashboard customisation: a "batches in progress" panel

Horizon's dashboard is a Vue app served from vendor/laravel/horizon. You can't drop new tabs into it without forking. What you can do is mount a separate Livewire or Inertia page at /admin/batches that reads the same data Horizon does (the job_batches table) plus your Redis progress keys.

A minimal Livewire component:

<?php

namespace App\Livewire;

use Illuminate\Support\Facades\Bus;
use Illuminate\Support\Facades\DB;
use Livewire\Attributes\Computed;
use Livewire\Component;

class BatchesPanel extends Component
{
    #[Computed]
    public function batches(): array
    {
        return DB::table('job_batches')
            ->whereNull('finished_at')
            ->orderByDesc('created_at')
            ->limit(20)
            ->get()
            ->map(fn ($row) => [
                'id'        => $row->id,
                'name'      => $row->name,
                'total'     => $row->total_jobs,
                'processed' => $row->total_jobs - $row->pending_jobs,
                'failed'    => $row->failed_jobs,
                'percent'   => $row->total_jobs > 0
                    ? (int) round(
                        ($row->total_jobs - $row->pending_jobs) / $row->total_jobs * 100
                    )
                    : 0,
            ])
            ->toArray();
    }

    public function cancel(string $batchId): void
    {
        Bus::findBatch($batchId)?->cancel();
    }

    public function render()
    {
        return view('livewire.batches-panel');
    }
}

Pair it with a Blade view that renders a table with progress bars and a "Cancel" button per row. Poll every 2 seconds with wire:poll.2s. You now have what Horizon's dashboard refuses to show you: live, mid-run progress per long batch, with a kill switch.

The cancel button matters more than people think. With a monolithic job, cancelling means killing the worker (horizon:terminate) and accepting the dirty state. With a batch, $batch->cancel() flips a flag; the child jobs check $this->batch()?->cancelled() at the top of handle() and bail out cleanly. Half-done batches stay half-done, but the half that's done is real, durable work.

What to take away

Three rules that hold across PHP teams running Laravel queues at scale:

1. No job in your codebase should be allowed to run longer than 60 seconds. Set $timeout = 60 as a hard ceiling. Anything that genuinely takes longer is actually N smaller jobs.
2. Bus::batch() plus Redis progress keys gives you the observability Horizon doesn't. A 50-line Livewire panel covers the gap.
3. Idempotency stops being optional when you split. Every child job runs at least once, sometimes more. Design for that on day one with firstOrCreate, updateOrCreate, S3 exists() checks, not after the first duplicate-charge incident.

Horizon is good at what it was built for. The 47-minute job isn't what it was built for.

---

If this was useful

If your codebase keeps producing jobs that grow until Horizon stops being useful, it's usually a sign the job class is doing too many things at once: orchestration, persistence, external IO, the actual domain work, all in one handle(). Decoupled PHP walks through the architectural layer your codebase reaches for after it outgrows the framework defaults: keeping the domain operation (generate a statement) separate from the orchestration (run it for every customer). The Bus::batch pattern becomes obvious once those two are separate concerns.

What's the longest-running job in your codebase right now, and what would it take to split it into 60-second chunks?

Available on Kindle, Paperback, and Hardcover. English, German, and Japanese editions out now — Portuguese and Spanish coming soon.