DEV Community

Blogs World
Blogs World

Posted on

Laravel retry_after vs timeout vs backoff: The Three Numbers Behind Duplicate Queue Jobs

A customer gets charged twice. A report email goes out three times. The logs show App\Jobs\SyncInvoice has been attempted too many times even though the job "only ran once" as far as anyone can tell.

When I dig into these, the code inside handle() is almost never the problem. The problem is three numbers that people treat as interchangeable: retry_after, timeout, and backoff. They live in different files, they're owned by different parts of Laravel, and they mean completely different things.

Here's what each one actually controls, and the single rule that keeps them from fighting each other.

1. retry_after: "I assume this worker died"

retry_after lives on the queue connection in config/queue.php:

'redis' => [
    'driver' => 'redis',
    'connection' => 'default',
    'queue' => env('REDIS_QUEUE', 'default'),
    'retry_after' => 90,
    'block_for' => null,
],
Enter fullscreen mode Exit fullscreen mode

It is not a retry delay. It's a lease. When a worker reserves a job, the queue gives it retry_after seconds. If the job hasn't been deleted or released by then, the queue assumes the worker crashed and makes the job available again, so another worker can pick it up.

Notice what that means: if your job legitimately takes 120 seconds and retry_after is 90, a second worker will grab the same job at second 90 while the first one is still running. That's your double charge. Each pickup also counts as an attempt, which is how you end up with "attempted too many times" on a job whose business logic never finished.

(SQS doesn't use retry_after; its visibility timeout plays the same role and is configured in AWS.)

2. timeout: "kill this job if it runs too long"

Timeout is owned by the worker process, not the connection. You set it on the command:

php artisan queue:work redis --timeout=60
Enter fullscreen mode Exit fullscreen mode

or per job:

class SyncInvoice implements ShouldQueue
{
    public $timeout = 60;
}
Enter fullscreen mode Exit fullscreen mode

When the limit is hit, the worker process is killed (this needs the pcntl PHP extension; without it, timeouts are silently ignored). The job then goes back through the normal attempt rules. If you'd rather a timed-out job fail immediately instead of being retried, set:

public $failOnTimeout = true;
Enter fullscreen mode Exit fullscreen mode

3. backoff: "wait before the next attempt"

backoff is the one people think retry_after is. It's the delay before a job that threw an exception is attempted again:

class SyncInvoice implements ShouldQueue
{
    public $tries = 4;

    // 10s, then 60s, then 5 minutes
    public $backoff = [10, 60, 300];
}
Enter fullscreen mode Exit fullscreen mode

Without a backoff, a job that fails because an external API is down gets retried almost instantly, burns through all its tries in a few seconds, and lands in failed_jobs before the API has even recovered.

The rule: timeout < retry_after

The Laravel docs state it plainly, and it's the one line I'd tattoo on every queue config:

The --timeout value should always be at least several seconds shorter than your retry_after configuration value.

That ordering guarantees the worker kills a stuck job before the queue hands it to someone else. Get it backwards and you've built a duplicate-processing machine.

A sane starting point for a job that normally finishes in under 30 seconds:

Setting Where Value
Job $timeout / --timeout job class or worker command 60
retry_after config/queue.php connection 90
$backoff job class [10, 60, 300]
$tries job class 4

If you have one genuinely long job (a big export, video processing), don't raise retry_after for everything. Put it on its own connection with a larger retry_after, and run a dedicated worker with a matching larger --timeout.

Horizon users

Horizon sets the worker timeout per supervisor in config/horizon.php:

'supervisor-1' => [
    'connection' => 'redis',
    'queue' => ['default'],
    'timeout' => 60,
    'tries' => 3,
],
Enter fullscreen mode Exit fullscreen mode

The same rule applies: that timeout must stay below the retry_after of the connection it uses. It's easy to raise one and forget the other during an incident.

Make the job safe to run twice anyway

Even with perfect numbers, a worker can die after charging the card but before deleting the job. Treat "runs at least once" as the contract and make the side effect idempotent:

public function handle(PaymentGateway $gateway): void
{
    $invoice = Invoice::findOrFail($this->invoiceId);

    if ($invoice->paid_at !== null) {
        return; // already done on a previous attempt
    }

    $gateway->charge($invoice, idempotencyKey: "invoice-{$invoice->id}");

    $invoice->update(['paid_at' => now()]);
}
Enter fullscreen mode Exit fullscreen mode

Most payment providers accept an idempotency key, which makes the gateway itself refuse a second identical charge.

A 2-minute audit

Before you change anything else, check these in order:

  1. Open config/queue.php and write down retry_after for the connection you actually use. Confirm QUEUE_CONNECTION in .env points at that connection.
  2. Find the effective timeout: the job's $timeout if set, otherwise the worker's --timeout (default 60), or the Horizon supervisor's timeout.
  3. Confirm timeout is at least a few seconds lower than retry_after.
  4. Check that the job has a $backoff if it calls anything over the network.
  5. Run php -m | grep pcntl on the worker server.
  6. After changing config, run php artisan queue:restart (or horizon:terminate). Long-running workers keep the old config and code in memory until they restart.

That last one catches a lot of people: you fix the numbers, deploy, and nothing changes because the workers never reloaded.

When retries still don't behave

This post covers why jobs run too many times. The opposite problem, a failed job that won't retry at all, has a different set of causes: tries and retryUntil() interacting, exceptions being swallowed inside handle(), queue:retry pushing to a queue no worker is listening to, and Horizon quirks. I wrote that up in detail, with Laravel 11 to 13 differences, here: Laravel Queue Failed Job Retry Not Working: Causes & Fixes.

And if you need to send your .env to a teammate while debugging queue settings, don't paste raw secrets into chat. The free Laravel .env security validator on Blogs World masks passwords and keys in the browser (nothing is uploaded) and flags production risks like APP_DEBUG=true along the way.

What's the worst duplicate-job bug you've hit? I'm curious whether it was the numbers or the missing idempotency check.

Top comments (0)