Laravel Horizon Queue Balancing: Idle Workers vs. Starved Queues

Every Horizon supervisor makes you choose. Turn balancing on and you pay for a floor of idle workers. Turn it off and a busy queue starves the queues listed after it. The more queues you run, the more each option costs, and many teams find this out only when a flood on one queue stalls all the others.

This article explains where the trade-off comes from, why running many small queues makes it worse, and how Skyline's weighted queues give you a shared pool of workers without starvation.

Key takeaways

• Balancing on ('auto' or 'simple') gives every queue its own worker pool. Nothing starves, but you pay a permanent floor of resident workers that scales with your queue count.
• Balancing off (false) runs one shared pool with no idle floor, but the queue list becomes a strict priority order, so a flood on one queue starves every queue below it.
• Both costs scale with the number of queues, which penalises the teams who split work into many queues to get per-queue pausing, draining and visibility.
• Skyline's queueWeights keeps the shared pool of unbalanced mode but checks queues in proportion to their weights (3 : 2 : 1) rather than in strict order. Important work gets most of the pickups and no queue drops to zero.

Why you end up with thirty queues in the first place

A queue is the smallest unit Horizon lets you observe and operate on, and Laravel's queue:pause and queue:clear commands both act on one queue at a time. So when you want operational control ("show me how exports is doing", "pause webhooks while the downstream API is down", "empty the reindex backlog without touching payments"), you split work into more, narrower queues. (Skyline adds pausing and resuming a single queue from the dashboard rather than the whole supervisor.)

That is how a healthy app ends up with thirty queues: payments, emails, webhooks-stripe, webhooks-github, exports, imports, reindex, thumbnails, and on down the list. Each one gets its own dashboard row and its own pause and drain switch.

Both balancing modes have a cost that grows with the number of queues, so the teams that split work most finely pay the most.

Option A: balance => 'auto' and the idle-worker floor

With balance => 'auto' (or 'simple'), Horizon allocates worker processes per queue. The auto balancer scales each queue's worker count up and down with its workload, but never below minProcesses, which defaults to 1:

'supervisor-1' => [
    'connection'   => 'redis',
    'queue'        => ['payments', 'emails', 'exports', /* ...27 more */],
    'balance'      => 'auto',
    'minProcesses' => 1,
    'maxProcesses' => 40,
],

minProcesses guarantees that every queue always has at least one worker watching it, so a flood on exports can't prevent payments from being serviced. Each queue is isolated, and for many setups that makes this the right default.

The cost is that one minimum worker per queue across thirty queues is a permanent floor of thirty worker processes. They are booted, resident, and idle most of the time, because most queues are empty most of the time. Each PHP worker is a full framework boot holding tens of megabytes of RAM whether or not it ever picks up a job.

The floor also counts against your ceiling. If minProcesses across thirty queues already pins thirty workers and maxProcesses is 40, the auto balancer has only ten processes left for the queue that is busy. Raising maxProcesses gives that queue more room, but the thirty idle workers are still there.

Option B: balance => false and starvation by list order

With balance => false, Horizon runs a single shared pool of workers. Every worker listens to every queue and checks them in the order you listed them. There is no per-queue floor and no idle minimum, and workers go wherever the work is. On paper this is the efficient option:

'supervisor-1' => [
    'connection'   => 'redis',
    'queue'        => ['high', 'default', 'low'],
    'balance'      => false,
    'maxProcesses' => 10,
],

The problem is the list order. Without balancing, the queue list is a strict left-to-right priority. A worker only looks at default when high is empty, and only looks at low when both are empty. Under light load you won't notice. Under sustained load, the order decides who gets served at all:

A flood on a higher-priority queue starves every queue below it until the flood drains.

Push fifty thousand jobs onto high and every worker in the pool works on it. default and low get no workers for as long as high stays non-empty. Dropping the idle-worker floor cost you the isolation that balancing guaranteed.

What is the difference between balance auto, simple and false?

Horizon ships two balancing strategies and an off switch. The two strategies are closer than most people assume: both 'auto' and 'simple' give every queue its own pool of worker processes, and they differ only in whether that allocation can move.

• 'simple' splits the supervisor's processes evenly across the queues in the list and leaves them there. Ten processes across two queues means five each, even if one of them has been empty all day.
• 'auto' starts from the same per-queue pools but continuously reallocates them to match each queue's workload, scaling a busy queue up toward maxProcesses and a quiet one down to minProcesses. (Whether "workload" means queue size or estimated time-to-clear is configurable via autoScalingStrategy.)
• false is not a balancing strategy. There are no per-queue pools: one shared pool of workers listens to every queue and checks them in the order you listed them.

Side by side, with Skyline's weighted mode in the last column:

	'simple'	'auto'	false	false + weights
Worker pools	One per queue, fixed even split	One per queue, resized to load	One shared pool	One shared pool
Idle-worker floor	maxProcesses ÷ queues, always resident	minProcesses per queue (defaults to 1)	None	None
Can a queue starve?	No	No	Yes, strict list order	No, every queue keeps a share
Priority model	None (even split)	None (load-driven)	Strict left-to-right	Proportional (3 : 2 : 1)
Adds capacity to a spike	No	Yes, scales processes up	Shared pool absorbs it	Shared pool absorbs it

The upstream reference for the two strategies is the Horizon balancing documentation.

Why the trade-off gets worse with every queue you add

The two options are mirror images:

• Balancing on gives you isolation (no queue starves) and charges you a per-queue process floor (idle workers, and less of the ceiling left for busy queues).
• Balancing off gives you a shared pool (no idle floor) and charges you starvation (strict priority order means a flood on one queue stops every queue below it).

Both costs scale with queue count, which is why teams with many queues feel them most. More queues means a taller idle floor under Option A, and under Option B a longer priority list with more queues below any flood. The granularity that made thirty queues attractive makes both options expensive.

Skyline's third option: weight the queues

The dilemma assumes that "balancing off" must mean strict priority: all of high before any of default. Skyline keeps the single shared pool of balance => false, with no per-queue minimum and no idle floor, and replaces strict list order with a proportional one via a queueWeights map:

'supervisor-1' => [
    'connection'   => 'redis',
    'queue'        => ['high', 'default', 'low'],
    'balance'      => false,
    'maxProcesses' => 10,
    'queueWeights' => [
        'high'    => 3,
        'default' => 2,
        // 'low' is omitted, so it gets the default weight of 1
    ],
],

Instead of always checking high first, workers check the queues in proportion to their weights, roughly 3 : 2 : 1. About 50% of pickups favour high, 33% default and 17% low. Queues you leave out of the map get a weight of 1, so a queue you forget to list still gets picked up.

Take the same flood of fifty thousand jobs on high. Under strict priority, low would get no workers until high drained. Under weights, most pickups still go to high, which is weighted highest, but one in six reaches low, so low keeps draining instead of stalling.

How weighted queues compare with both balancing modes

• No idle-worker floor. Weights live under balance => false, so there's no per-queue minProcesses. Thirty queues do not mean thirty resident workers. They share one pool, the same as plain unbalanced mode.
• No starvation. Every queue has a weight of at least 1, so every queue is checked on a regular cadence. A flood can reduce a sibling queue's share, but not its throughput to zero.
• Explicit priority. A 3 : 2 : 1 map says how much each queue matters, instead of encoding priority in list position or paying for it with dedicated worker pools.
• Per-queue operations stay. You keep per-queue stats, pause, drain and inspect for all thirty queues without paying the cost of balancing.

When should you still use balance => 'auto'?

Weighted queues don't replace balance => 'auto' everywhere. Auto-balancing's strength is elastic capacity: it starts more workers for a queue that is spiking, stops them afterwards, and scales the total process count to demand. Weights don't add processes. They redistribute a fixed pool across queues. If your bottleneck is throughput on one queue and you have CPU headroom, auto-scaling is still the better fit.

Weighting fits a specific, common setup best: many queues kept for operational granularity, most of them idle most of the time, with occasional floods on one queue that should not hold up the others. That is also where the balance on/off choice costs the most.