A customer gets charged twice. A report email goes out three times. The logs show App\Jobs\SyncInvoice has been attempted too many times even though the job "only ran once" as far as anyone can tell.
When I dig into these, the code inside handle() is almost never the problem. The problem is three numbers that people treat as interchangeable: retry_after, timeout, and backoff. They live in different files, they're owned by different parts of Laravel, and they mean completely different things.
Here's what each one actually controls, and the single rule that keeps them from fighting each other.
1. retry_after: "I assume this worker died"
retry_after lives on the queue connection in config/queue.php:
'redis' => [
'driver' => 'redis',
'connection' => 'default',
'queue' => env('REDIS_QUEUE', 'default'),
'retry_after' => 90,
'block_for' => null,
],
It is not a retry delay. It's a lease. When a worker reserves a job, the queue gives it retry_after seconds. If the job hasn't been deleted or released by then, the queue assumes the worker crashed and makes the job available again, so another worker can pick it up.
Notice what that means: if your job legitimately takes 120 seconds and retry_after is 90, a second worker will grab the same job at second 90 while the first one is still running. That's your double charge. Each pickup also counts as an attempt, which is how you end up with "attempted too many times" on a job whose business logic never finished.
(SQS doesn't use retry_after; its visibility timeout plays the same role and is configured in AWS.)
2. timeout: "kill this job if it runs too long"
Timeout is owned by the worker process, not the connection. You set it on the command:
php artisan queue:work redis --timeout=60
or per job:
class SyncInvoice implements ShouldQueue
{
public $timeout = 60;
}
When the limit is hit, the worker process is killed (this needs the pcntl PHP extension; without it, timeouts are silently ignored). The job then goes back through the normal attempt rules. If you'd rather a timed-out job fail immediately instead of being retried, set:
public $failOnTimeout = true;
3. backoff: "wait before the next attempt"
backoff is the one people think retry_after is. It's the delay before a job that threw an exception is attempted again:
class SyncInvoice implements ShouldQueue
{
public $tries = 4;
// 10s, then 60s, then 5 minutes
public $backoff = [10, 60, 300];
}
Without a backoff, a job that fails because an external API is down gets retried almost instantly, burns through all its tries in a few seconds, and lands in failed_jobs before the API has even recovered.
The rule: timeout < retry_after
The Laravel docs state it plainly, and it's the one line I'd tattoo on every queue config:
The
--timeoutvalue should always be at least several seconds shorter than yourretry_afterconfiguration value.
That ordering guarantees the worker kills a stuck job before the queue hands it to someone else. Get it backwards and you've built a duplicate-processing machine.
A sane starting point for a job that normally finishes in under 30 seconds:
| Setting | Where | Value |
|---|---|---|
Job $timeout / --timeout
|
job class or worker command | 60 |
retry_after |
config/queue.php connection |
90 |
$backoff |
job class | [10, 60, 300] |
$tries |
job class | 4 |
If you have one genuinely long job (a big export, video processing), don't raise retry_after for everything. Put it on its own connection with a larger retry_after, and run a dedicated worker with a matching larger --timeout.
Horizon users
Horizon sets the worker timeout per supervisor in config/horizon.php:
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['default'],
'timeout' => 60,
'tries' => 3,
],
The same rule applies: that timeout must stay below the retry_after of the connection it uses. It's easy to raise one and forget the other during an incident.
Make the job safe to run twice anyway
Even with perfect numbers, a worker can die after charging the card but before deleting the job. Treat "runs at least once" as the contract and make the side effect idempotent:
public function handle(PaymentGateway $gateway): void
{
$invoice = Invoice::findOrFail($this->invoiceId);
if ($invoice->paid_at !== null) {
return; // already done on a previous attempt
}
$gateway->charge($invoice, idempotencyKey: "invoice-{$invoice->id}");
$invoice->update(['paid_at' => now()]);
}
Most payment providers accept an idempotency key, which makes the gateway itself refuse a second identical charge.
A 2-minute audit
Before you change anything else, check these in order:
- Open
config/queue.phpand write downretry_afterfor the connection you actually use. ConfirmQUEUE_CONNECTIONin.envpoints at that connection. - Find the effective timeout: the job's
$timeoutif set, otherwise the worker's--timeout(default 60), or the Horizon supervisor'stimeout. - Confirm timeout is at least a few seconds lower than
retry_after. - Check that the job has a
$backoffif it calls anything over the network. - Run
php -m | grep pcntlon the worker server. - After changing config, run
php artisan queue:restart(orhorizon:terminate). Long-running workers keep the old config and code in memory until they restart.
That last one catches a lot of people: you fix the numbers, deploy, and nothing changes because the workers never reloaded.
When retries still don't behave
This post covers why jobs run too many times. The opposite problem, a failed job that won't retry at all, has a different set of causes: tries and retryUntil() interacting, exceptions being swallowed inside handle(), queue:retry pushing to a queue no worker is listening to, and Horizon quirks. I wrote that up in detail, with Laravel 11 to 13 differences, here: Laravel Queue Failed Job Retry Not Working: Causes & Fixes.
And if you need to send your .env to a teammate while debugging queue settings, don't paste raw secrets into chat. The free Laravel .env security validator on Blogs World masks passwords and keys in the browser (nothing is uploaded) and flags production risks like APP_DEBUG=true along the way.
What's the worst duplicate-job bug you've hit? I'm curious whether it was the numbers or the missing idempotency check.
Top comments (0)