I kept adding Horizon workers to a 2 GB Laravel Cloud box until memory was at about 90%. I got to 27.
Then I made the workers share their compiled code and did it again. Same box, same app, same 600 jobs a minute. This time it held 210.
This was a memory test, not a speed test. At that job rate CPU was never the limit. Memory was. Running the same 210 workers stock took a 16 GB box.
I started looking into this because a couple of my servers were sitting in swap, and Horizon was a big part of what filled them.
Every worker boots your whole app
Each queue:work is its own PHP process. So is every Horizon worker, and (this one surprised me) every Horizon supervisor. Each of them boots Laravel and compiles the code it touches into its own private memory.
The Supervisor example in the Laravel docs uses numprocs=8. If you copied it, you have eight processes, each holding its own copy of the same compiled app.
In my Docker benchmark, a stock Horizon worker took about 63 MB. Supervisors are the funny part. A supervisor never runs a job; it only watches its workers. One still took about 56 MB. Six of them came to about 340 MB, before a single job ran.
PHP-FPM solved this years ago
FPM starts one master process, sets up opcache's shared memory there, and then forks its workers. Every FPM worker sees the same shared memory, so code compiled by one worker is reused by all of them.
The master never boots your app. It just owns the cache. That turned out to be the whole trick.
Queue workers get none of this. Supervisord or systemd starts each php artisan queue:work as a separate, brand-new process, and there's no common parent that owns a cache. CLI opcache is off by default. Turn it on with opcache.enable_cli=1 and each process gets its own private cache. Still one copy per worker.
When I posted about this on r/PHP, someone asked about opcache.file_cache, warmed in CI. It saves compile time, but not memory. Every worker still loads its own copy.
Check your own servers
Before you believe any of my numbers, look at yours. Use PSS, not RSS. RSS counts shared memory in full for every process, so adding it up overcounts. PSS splits shared pages between the processes that use them, so the sum is honest.
Run this as the user your workers run as (or with sudo):
for pid in $(pgrep -f '^[^ ]*php[^ ]* .*artisan (queue:work|horizon(:supervisor|:work)?)( |$)'); do
awk '/^Pss:/ {print $2}' "/proc/$pid/smaps_rollup" 2>/dev/null
done | awk '
{ total += $1; count++ }
END {
if (count) printf "%d processes, %.0f MB total\n", count, total / 1024
else print "No matching processes found"
}'
It counts Horizon's master and supervisors too, since they cost memory just the same. If you run several apps on one server, it adds them all up, so narrow the pattern if you need to.
Write the total down. It's your "before". How much of it is code and how much is your jobs' own data depends on your app, and only an "after" can tell you.
I did think about Go
Moving the workers to Go would fix the memory, sure. But in my apps, the jobs are the app. They use the same models, services and helpers as everything else. In one app, 17 job classes pull in about half the codebase, around 167 of its 400 files. Rewriting that means rewriting half the app and then keeping two versions in sync. A composer package was the cheaper fix.
Fork first, then boot
The obvious PHP fix is to boot the app once and fork the workers from it. The trouble is what the children inherit. Each one gets the parent's open database and Redis sockets, so two workers end up talking over one connection. Each one also starts with whatever state the boot left behind, and any random state seeded before the fork can repeat across children.
Someone on r/PHP summed up their own pcntl worker setup as "more issues than worth it". Fair enough.
So queue-prefork does what the FPM master does. It forks before your app exists. A small template process starts with opcache and no app code at all. Each worker is forked from it and then boots your app itself, from the normal artisan entry, like any stock worker. The first worker compiles your code into the template's shared opcache. Every worker after that finds the code already there.
Each worker still opens its own connections and builds its own container and config, as usual. The only thing they share is the compiled code.
For plain queue:work, the process supervisord started stays behind as a tiny /bin/sh stub with the same PID. Your process manager still controls it, and queue:restart works as usual. With Horizon, the master, the supervisors and the workers all become slim forks. The README has the details.
The numbers
I ran two identical Docker stacks side by side on the same seeded job stream: Laravel 13.38 with about 45 production packages, PHP 8.4 and Horizon 5.50.1, on a 2-CPU VM. Memory is the median total PSS, sampled every minute. The prefork numbers include its own templates, stubs and fork servers.
| Setup | Stock | Prefork |
|---|---|---|
8 × queue:work, typical jobs |
645 MB | 192 MB (−70%) |
8 × queue:work, idle |
455 MB | 102 MB (−78%) |
8 × queue:work, heavy jobs |
1,025 MB | 524 MB (−49%) |
| Horizon, 6 autoscaling supervisors | 1,279 MB | 240 MB (−81%) |
Horizon autoscaled in both runs, to a median of 14 workers on stock and 17 on prefork. So prefork carried more workers in less memory. No job failed in any run, and throughput was the same.
CPU went down too, from 21.3% to 11.9% of a core for Horizon with typical jobs, because a new worker doesn't recompile what's already in the cache. It only boots Laravel.
Heavy jobs save less, because their own data is private either way. Stock ran with CLI opcache off, as php.ini-production ships it. On Laravel Cloud it's on, so there each stock worker also keeps its own private cache. The benchmark harness is in the repo if you want to rerun it.
That was a test setup. In real production, one of my Horizon setups went from about 1.2 GB to about 280 MB.
Back to the Cloud box from the start. There, a stock worker cost about 56 MB and a forked one about 7.4 MB. That's how 27 became 210 on a Flex 2 GB instance. I also ran the 210 workers stock, to see what that really takes. They used 11,766 MB. That's too much for an 8 GB instance, so the smallest one that fits is Pro 16 GB: $128 a month instead of $24. In fairness, Pro also has 4 vCPUs to Flex's one, but at this job rate CPU wasn't what ran out.
The four minutes I blamed on my own code
On Cloud it took about four minutes for all 210 workers to show up, and I was sure my fork server was slow. It wasn't. Stock Horizon does exactly the same. It's Horizon's start-up ramp.
A supervisor starts with floor((minProcesses + maxProcesses) / 2) workers. Then it adds balanceMaxShift workers (1 by default) every balanceCooldown seconds (3 by default). And with balance => 'simple', 'processes' => N only sets the maximum. The minimum stays at 1.
So a 150-worker supervisor starts with 75 and spends almost four more minutes on the rest. That's Horizon 5.50.1; I didn't check other versions. If you want the whole pool right away:
'balance' => 'simple',
'processes' => 150,
'minProcesses' => 150,
With auto balancing, raise balanceMaxShift instead.
This works with stock workers too, but then every worker compiles your app at the same moment, which is a lot of CPU at once. With forked workers it's cheap. In a burst test on 2 CPUs, 100 workers were ready about 1.7 seconds after the supervisor came up, against 8 to 14 seconds with stock. I haven't tested that on a big many-core box, where stock should do better.
Trying it
It's for Linux servers with Laravel 11–13, PHP 8.2+ and, if you use it, Horizon 5.23+. It won't help when each container or pod runs a single worker (that includes Laravel Cloud's managed queues and Vapor), because there's nothing to share with. A worker running as PID 1 in Docker stays stock, so give the container an init (init: true).
composer require tevkert/queue-prefork
php artisan queue-prefork:doctor # on the worker server: how your workers would run, and why
php artisan queue-prefork:status # memory right now, your stock baseline
It's on as soon as it's installed, for workers started after that. Deploy, restart your workers with queue:restart or horizon:terminate, let them warm up, and run status again. That's your "after".
If a safety check fails, for example because an extension opened sockets or threads at startup, the workers just run stock and the package logs why. To turn it off, set QUEUE_PREFORK=false, re-cache your config and restart the workers.
On Laravel Cloud, start Horizon as a custom background process with php -d grpc.enable_fork_support=1 artisan horizon.
It's MIT-licensed and on GitHub.
How many queue workers do you run on one server, and what does the script say they cost? Post the total in the comments, with the worker count and what kind of jobs they run. For comparison, that Horizon setup of mine was at about 1.2 GB. I've only measured my own apps, so I don't know what normal looks like.
And if you fork booted Laravel workers with pcntl in production and it works fine, I'd like to hear how.
Top comments (0)