DEV Community

Cover image for What Replaced queue:work in My Laravel App (And Why I Stopped Praying to Supervisor at 3 AM)
Hermann D. Schimpf
Hermann D. Schimpf

Posted on Originally published at hds-solutions.net

What Replaced queue:work in My Laravel App (And Why I Stopped Praying to Supervisor at 3 AM)

Versión en español aquí.


Banner Image


In the previous chapter we figured out where our data lives: files in S3, cache and sessions in DynamoDB, and a NoSQL database that speaks Eloquent. But one question was left hanging in the air: if a Lambda function is born, responds, and dies in milliseconds, who stays awake processing jobs in the background?

On a traditional server, the answer is supervisord babysitting a long-running process. In an ephemeral environment where nothing lives longer than 15 minutes, that recipe simply doesn't work.

In this chapter we're going to replace that daemon with SQS + a Lambda worker. We'll do it in three acts: first the setup (surprisingly short), then the lifecycle of a job (where the magic happens), and finally the topic almost nobody explains in full: what happens when a job fails, who retries it, and where it ends up.


1. The Serverless Treasure Map

Quick recap, so we know where we are:

AWS Serverless architecture

We already have Lambda running our Laravel app, CloudFront splitting traffic between S3 (assets) and Lambda (the app), and DynamoDB serving as cache, sessions, and our primary database. Today's agenda:

  • Managing our event queue with SQS,
  • and a second Lambda function for the Laravel Queue worker.

Remember the component table from the second chapter? Today we cross off this row:

Component Dedicated Server AWS Serverless
Event Queue SSH $ php artisan queue:work / maybe supervisor SQS + Lambda 🚀

2. Why queue:work Doesn't Fit in Serverless

a) How queues work on a traditional server

Oversimplifying a bit, php artisan queue:work is a while (true) loop: it asks the queue if there's any work; if there is, it processes it; if not, it sleeps for a few seconds and asks again. A resident process hogging memory and CPU 24/7, even if the queue has been empty for 3 days.

And because it's a long-running process, it needs a babysitter:

  • supervisor (or systemd) to resurrect it when it dies.
  • queue:restart after every deploy, because the process still has the old code loaded in memory.
  • --max-jobs and --max-time to recycle it before memory leaks turn it into a black hole.
  • And to scale? More processes → more RAM → bigger instances → all accompanied by a bigger bill.

queue:work is an employee getting paid to hit F5 on their inbox every three seconds. Eight hours a day. Even when nothing comes in.

b) Why that doesn't exist in Lambda

A Lambda invocation lives, at most, for 15 minutes 1. There is no resident process, no while (true), and nobody asking "got any work?". If you try running queue:work inside a Lambda... it works for 15 minutes, dies, and nobody brings it back. And if you resurrect it with a cron job, you're paying for a Lambda 24/7 just to poll. Congratulations, you just reinvented EC2, only more expensive.

c) Flipping the model: from pull to push

The solution isn't to adapt the worker to Lambda—it's to flip it upside down. Instead of the worker pulling work (pull), the queue invokes the worker when work arrives (push):

  1. Laravel calls dispatch() and pushes the job to SQS.
  2. SQS detects new messages and invokes a Lambda function with those messages.
  3. Lambda processes the jobs, finishes, and disappears.
  4. An empty queue = zero Lambda invocations.

The worker stops being a process that waits and becomes an event that happens.


3. The New Queue Stack: SQS as the Mailbox, Lambda as the Worker

a) SQS in 30 seconds for Laravel devs

Amazon SQS (Simple Queue Service) is a managed message queue 2. The bare minimum you need to know:

  • Message: for us, the serialized JSON payload of the job. The exact same one you'd see in the payload column of the jobs table if you were using the database driver.
  • Visibility timeout: when someone picks up a message, SQS hides it for X seconds. If nobody deletes it within that window, it reappears so another consumer can try again. It's the equivalent of Laravel's retry_after.
  • Dead Letter Queue (DLQ): a second queue where SQS moves messages that have failed too many times. The "lost and found box".
  • Standard vs FIFO: standard queues don't guarantee order (and they're the ones we'll use); FIFO queues do, in exchange for lower throughput.

With the vocabulary out of the way, let's jump into the code.

b) Wiring it up with Serverless Lift

Starting from the serverless.yml we built in the second chapter, we add two environment variables and a Lift construct 3 4:

service: laravel

provider:
  name: aws
  environment:
    # ...
+   QUEUE_CONNECTION: sqs
+   SQS_QUEUE: ${construct:jobs.queueUrl}

functions:
  web:
    handler: Bref\LaravelBridge\Http\OctaneHandler
    runtime: php-85
    # ...

constructs:
  website:
    type: server-side-website
    # ...

+ jobs:
+   type: queue
+   worker:
+     handler: Bref\LaravelBridge\Queue\QueueHandler
+     runtime: php-85
+     timeout: 60  # seconds, the ceiling for the longest job

plugins:
  - ./vendor/bref/bref
  - serverless-lift
Enter fullscreen mode Exit fullscreen mode

QUEUE_CONNECTION: sqs enables the driver Laravel ships with out of the box, and SQS_QUEUE receives the full URL of the queue Lift just created (if you prefer passing only the queue name, you'll also need SQS_PREFIX 3).

Even though on a traditional server the queue runs via CLI with artisan, here we don't use php-85-console. Just like with OctaneHandler, we use the php-85 event runtime with Bref's QueueHandler class.

And what did Lift do behind the scenes with that jobs construct? 4

  • Created the SQS queue.
  • Created a Dead Letter Queue with a 14-day retention period (the maximum SQS allows).
  • Created the worker Lambda function with Bref's handler.
  • Configured the event source mapping: the AWS piece that connects the SQS queue to the Lambda function and invokes it when messages arrive. Our replacement for queue:work.
  • And granted sqs:SendMessage permission to all functions in the stack 5. That's why, unlike the DynamoDB tables from the previous chapter, we didn't have to write a single line of IAM here.

Credentials, just like with DynamoDB, are assigned to each function's execution role. They rotate automatically and nobody ever sees them.

c) New infrastructure, same old code

This is my favorite part. Nothing you've already written has any idea anything changed:

// A job
SendWelcomeEmail::dispatch($user->uuid);

// A queued mailable
Mail::to($user)->queue(new WelcomeMail($user));

// A notification implementing ShouldQueue
$user->notify(new VerifyEmail);
Enter fullscreen mode Exit fullscreen mode

All of that goes to SQS, and a Lambda processes it. ShouldQueue listeners, chained jobs... almost everything works the exact same way. Another point for Laravel's abstraction 💪.

Important: Laravel's SQS driver caps delay() / #[Backoff] at 900 seconds (15 minutes), which is the SQS limit 6.

If you need multiple queues (one for urgent emails, another with lower concurrency to avoid hammering an external API, a FIFO queue for strict ordering), each one is an additional queue construct, and since Laravel 13.26 they can be routed with Queue::forward() 7.


4. What Actually Happens When You Dispatch a Job?

Up to this point it all looks like magic: we added a block in serverless.yml, called dispatch(), and the job ran. But in production, magic only lasts until the first incident. Let's walk step by step through who does what, from the moment a message enters SQS until it disappears.

a) Laravel dispatches, SQS receives

When you call SendWelcomeEmail::dispatch($user->uuid), Laravel serializes the job and the sqs driver makes a SendMessage call to the SQS API to drop it into the queue. And that's it—our web Lambda has done its part and can finish its execution.

The message body is the same old JSON (displayName, job, data, uuid, attempts). To Laravel, SQS is just another mailbox to drop that exact same envelope into.

b) SQS notifies, Lambda wakes up

The event source mapping that Lift created handles long polling for us 8: AWS keeps an internal process polling the queue and, as soon as it detects messages, invokes our worker Lambda function with a batch of up to batchSize messages (Lift defaults this to 1; we'll see why and when to change it shortly). Polling still exists; the difference is that instead of your queue:work process doing it, AWS takes care of it now.

Important: That AWS polling counts as ReceiveMessage calls in SQS, even when the queue is empty. Fortunately, the free tier includes 1 million monthly requests 9: with an idle queue, that constant check generates ~21,000 requests per day (~630,000 per month), so you pay nothing.

c) Bref receives the batch, Laravel runs the job

When Lambda wakes up, it hands the SQS event to Bref's QueueHandler 10. If we peek inside its handleSqs() method (simplified), we can see exactly what happens with each message:

public function handleSqs(SqsEvent $event, Context $context): void
{
    $worker = $this->container->makeWith(Worker::class, [
        'resetScope' => fn () => $this->resetLaravel(),
        // ...
    ]);

    foreach ($event->getRecords() as $sqsRecord) {
        $timeout = $this->calculateJobTimeout($context->getRemainingTimeInMillis());

        $worker->runSqsJob(
            $job = $this->marshalJob($sqsRecord),
            $this->connection,
            $this->gatherWorkerOptions($timeout),
        );

        if (! $job->hasFailed() && ! $job->isDeleted()) {
            $job->delete();
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Three key details in those few lines:

  1. That Worker is Laravel's. It literally extends Illuminate\Queue\Worker 11. Bref didn't reinvent how to run a job: it passes each message to the exact same engine that queue:work uses. Direct consequence: #[Tries], #[Backoff], $timeout, failed(), middleware, ShouldBeUnique, retryUntil(), and the JobProcessing/JobProcessed/JobFailed events... everything works just like on a traditional server. There's no "supported features" table to check.
  2. State is cleaned up between jobs. That resetLaravel() callback flushes log context, forgets scoped container instances, and clears resolved facades before each run. Why? For the exact same reason we saw with Octane in the second chapter: the Lambda instance (and its PHP process) survives from one job to the next. Without that cleanup, the second job would inherit the in-memory leftovers of the first.
  3. Deleting from SQS is explicit. Only if the job finished without failing does Bref call $job->delete() to remove it from the queue. If it failed or was already deleted (or released back to the queue, as we'll see next), it leaves it alone.

d) Lambda's timeout vs. the job's $timeout

Did you spot that calculateJobTimeout($context->getRemainingTimeInMillis()) inside the loop? Bref doesn't use a fixed timeout: it checks how many milliseconds the Lambda invocation has left to live (minus a 1-second safety margin) and assigns that time to the job. With that value, the Worker schedules a pcntl_alarm() 11: if the job runs over, the alarm goes off, the process exits with exit(1), and the invocation fails. The job is only marked as failed (and sent to failed_jobs) if you defined #[FailOnTimeout] or $failOnTimeout = true.

Two practical takeaways:

  • The worker's timeout in serverless.yml is, in practice, the ceiling for the longest job you can process. Set it with your heaviest task in mind.
  • With a batchSize greater than 1, all jobs in the batch share that same clock: if you process long-running tasks, keep the batch small (or leave it at 1).

Watch out for #[Timeout] / $timeout: If you define a custom timeout on the job class, Laravel overrides the value calculated by Bref. If that number exceeds the time remaining in Lambda, AWS will kill the invocation cold before PHP's alarm ever rings. Recommendation: use a value well below the worker's timeout, especially with batchSize greater than 1, where the last jobs in the batch start with less time available.


5. Who Takes Over When Something Blows Up?

We all love watching the happy path play out by the book: the job comes in, runs, and gets deleted from SQS. But in the real world, external APIs go down, validations fail, and processes run out of memory.

You're probably wondering: "If Lift already created a Dead Letter Queue in SQS and Laravel already has its own failed_jobs table, aren't they doing the exact same thing?"

The answer is no. When we combine Laravel's Worker with SQS and Lambda, two retry and failure mechanisms coexist in the same stack: one on our application side and another on the AWS infrastructure side. And even though they sound similar, they tackle completely different problems.

a) Retries are handled by Laravel, not SQS

Let's start on our application side. Picture an ordinary job configured with retries and backoff:

use Illuminate\Queue\Attributes\Backoff;
use Illuminate\Queue\Attributes\Tries;

#[Tries(3)]
#[Backoff(30)]
final class SendWelcomeEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable;

    public function __construct(public readonly string $user_uuid) {}

    public function handle(): void
    {
        // ... if this throws an exception, Laravel retries in 30 seconds, up to 3 times
    }

    public function failed(Throwable $e): void
    {
        // ... once all 3 attempts are exhausted, Laravel calls this
    }
}
Enter fullscreen mode Exit fullscreen mode

When handle() throws an exception, Laravel's Worker does what it always does: if attempts remain, it calls $job->release(30). And here's the trick. Laravel's native SqsJob simply updates the message's visibility timeout in SQS, but Bref overrides it and does something else 12:

public function release($delay = 0)
{
    $this->released = true;

    $payload = $this->payload();
    $payload['attempts'] = ($payload['attempts'] ?? 0) + 1;

    // deletes the original message...
    $this->sqs->deleteMessage([
        'QueueUrl'      => $this->queue,
        'ReceiptHandle' => $this->job['ReceiptHandle'],
    ]);

    // ...and sends a brand-new copy, with the updated attempt counter and backoff delay
    $this->sqs->sendMessage([
        'QueueUrl'     => $this->queue,
        'MessageBody'  => json_encode($payload),
        'DelaySeconds' => $this->secondsUntil($delay),
    ]);
}
Enter fullscreen mode Exit fullscreen mode

It's not "forwarding". It's deleting the email and composing a brand-new one, identical, with [Attempt 2] in the subject line and scheduled to send in 30 seconds.

Why? Because that way the attempt counter travels inside the payload (attempts() reads it from there, not from SQS's ApproximateReceiveCount), and backoff translates into a real DelaySeconds. As a result, #[Tries(3)] and #[Backoff(30)] behave exactly like they do on a server. Bref deliberately chose Laravel's semantics over SQS's 3.

If you don't define #[Tries] / $tries, Bref's handler defaults to 3 13. And if you use exponential backoff (#[Backoff([10, 60, 300])]), that works too: each re-queued message carries the delay corresponding to that attempt.

b) The side effect: SQS never finds out about the failure

When an exception is thrown inside handle(), Laravel's Worker catches it in its own try/catch: it reports it to the logs, decides whether to retry or mark the job as failed, and moves on. Because that exception never bubbles up to handleSqs(), as far as AWS Lambda is concerned, the function succeeded without errors.

This means SQS's native retry mechanism (visibility timeout → maxReceiveCount → DLQ) never finds out that a job failed due to a PHP exception. Not on the first attempt, and not on the third.

Laravel is that teammate who says "all good" in the daily standup while production is on fire. And SQS believes them.

c) First layer: when the job exhausts its attempts in Laravel

When a job fails for the third time and runs out of attempts, Laravel gives up: it calls the class's failed() method and records the failure using the driver configured in config/queue.php (queue.failed).

The problem is that, by default, Laravel tries to save that record in a failed_jobs table inside a SQL database... which we said goodbye to in the previous chapter. Without a relational database, the quick way out is to disable it altogether with QUEUE_FAILED_DRIVER: null 3: the exception still ends up in CloudWatch logs, but the job itself isn't stored anywhere. I don't recommend it.

The good news is that Laravel already includes native support for storing failed_jobs in DynamoDB 14. We only need to tweak config/queue.php:

'failed' => [
    'driver' => env('QUEUE_FAILED_DRIVER', 'database-uuids'),
    // When using driver=dynamodb there's no need to specify 'database'
    // 'database' => env('DB_CONNECTION', 'sqlite'),
    'key'    => env('AWS_ACCESS_KEY_ID'),
    'secret' => env('AWS_SECRET_ACCESS_KEY'),
    'token'  => env('AWS_SESSION_TOKEN'),
    'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
    'table'  => env('DYNAMODB_FAILED_JOBS_TABLE', 'failed_jobs'),
],
Enter fullscreen mode Exit fullscreen mode
provider:
    environment:
        QUEUE_FAILED_DRIVER: dynamodb
        DYNAMODB_FAILED_JOBS_TABLE: !Ref FailedJobsTable
Enter fullscreen mode Exit fullscreen mode

And we declare the table under resources in serverless.yml. Laravel uses application as the partition key (the app name) and uuid as the sort key:

FailedJobsTable:
    Type: AWS::DynamoDB::Table
    Properties:
        TableName: failed_jobs
        BillingMode: PAY_PER_REQUEST
        AttributeDefinitions:
            - { AttributeName: application, AttributeType: S }
            - { AttributeName: uuid,        AttributeType: S }
        KeySchema:
            - { AttributeName: application, KeyType: HASH }
            - { AttributeName: uuid,        KeyType: RANGE }
        TimeToLiveSpecification:
            AttributeName: expires_at
            Enabled: true
Enter fullscreen mode Exit fullscreen mode

Just like we did in the previous chapter for the cache and session tables, we use TimeToLiveSpecification; Laravel saves each failed job with an expiration timestamp (expires_at, 7 days by default 14). This driver doesn't even implement queue:flush: DynamoDB takes care of cleaning up expired records automatically.

Finally, we grant the execution role permissions on the table:

provider:
    iam:
        role:
            statements:
                -   Effect: Allow
                    Resource: !GetAtt FailedJobsTable.Arn
                    Action:
                        - dynamodb:PutItem
                        - dynamodb:Query
                        - dynamodb:GetItem
                        - dynamodb:DeleteItem
Enter fullscreen mode Exit fullscreen mode

To list or retry failed jobs—since there's no SSH in Lambda—we use the bref:cli bridge we saw in the second chapter:

serverless bref:cli --args="queue:failed"
serverless bref:cli --args="queue:retry all"
Enter fullscreen mode Exit fullscreen mode

"Wait, so what is the DLQ that Lift created even for?"

d) Second layer: when the infrastructure fails

A message only reaches the Dead Letter Queue when the entire Lambda invocation fails, without giving Laravel a chance to catch the error. For example:

  • A job exceeded the remaining invocation time (pcntl_alarm went off and execution terminated with exit(1)).
  • It ran out of memory (OOM).
  • A fatal PHP error outside the scope of the Worker.
  • A boot failure: a missing extension, a configuration error, a composer install that left something out.

In any of these scenarios, Lambda dies before Bref can delete the message, so SQS's native mechanism kicks in:

  1. Wait (visibility timeout): the message stays hidden until its visibility timeout expires. Lift automatically sets it to 6 times the worker's timeout following AWS's recommendation 15 (with the 60 seconds from our example, it reappears after 6 minutes).
  2. Retry: once visible again, SQS hands it back to Lambda.
  3. Off to the DLQ: if the invocation fails 3 times in a row (maxRetries by default in Lift), SQS removes the message from the main queue and stores it in the Dead Letter Queue for up to 14 days, preventing it from being lost.

To inspect what landed in the DLQ or re-queue it without opening the AWS console, Lift gives us these commands 16:

serverless jobs:logs --tail      # live worker logs
serverless jobs:failed           # inspect messages in the DLQ
serverless jobs:failed:retry     # move everything back to the main queue
serverless jobs:failed:purge     # delete everything and move on with life
Enter fullscreen mode Exit fullscreen mode

Of course, nobody wants to run serverless jobs:failed every day just in case. If you add the alarm property to the construct, Lift automatically sets up a CloudWatch alarm that sends you an email as soon as a message lands in the DLQ 4:

constructs:
    jobs:
        type: queue
        # ...
        alarm: alerts@my-domain.com  # email alert when the DLQ receives messages
Enter fullscreen mode Exit fullscreen mode

After the first serverless deploy, AWS will send a confirmation email to the address configured in alarm (don't forget to confirm it to activate the alerts).

e) The two layers, side by side

First layer: failed_jobs Second layer: SQS DLQ
What failed? Our code: exception in handle() Infrastructure: timeout, OOM, fatal error
Who retries? Laravel (#[Tries], #[Backoff]) SQS (visibility timeout, maxRetries)
What gets retried? That job, after configured backoff The in-flight job and rest of batch (6× timeout)
Where does it end? failed_jobs table (DynamoDB, 7d) Dead Letter Queue (SQS, 14 days)
How do I retry it? bref:cli --args="queue:retry ..." serverless jobs:failed:retry
How do I find out? failed(), CloudWatch, queue:failed Email alert (alarm:)

Take a look at the What gets retried? row in the second layer. When Lambda dies mid-batch (due to timeout or OOM), the jobs that had already finished were deleted one by one with $job->delete(), so they don't run again. SQS only re-delivers what remained undeleted: the job that was halfway through execution and the ones waiting behind it in the batch.

The danger lies in that half-finished job: when SQS retries it, it will run again from the very beginning. If it had already charged a credit card or sent an email a second before Lambda died and isn't idempotent, it will do it a second time (and you'll get acquainted with the word "refund" much sooner than you'd like).

Golden rule: always design idempotent jobs.


6. Tuning the Worker Knobs for Production

The 8 lines from section 3 are enough to get started, but now that we know what happens under the hood, we can tune the knobs for production. Here's what a battle-tested configuration looks like:

constructs:
    jobs:
        type: queue
        maxRetries: 5       # queue maxReceiveCount: AWS recommends >= 5
        maxConcurrency: 20  # the "clean" brake: Lambda stops polling, messages wait
        worker:
            handler: Bref\LaravelBridge\Queue\QueueHandler
            runtime: php-85
            layers:
                - ${bref-extra:gd-php-85}
            reservedConcurrency: 25  # hard cap & reservation; always >= maxConcurrency
            timeout: 300
        batchSize: 10
Enter fullscreen mode Exit fullscreen mode

a) timeout: the ceiling for the longest job

It defines both the maximum duration of an invocation and the queue's visibility timeout in SQS (which Lift multiplies by 6). In my case I bumped it to 300 seconds because the worker processes heavy images, and resizing a 20 MB photo takes time. If your jobs only send emails or call an API, 30 or 60 seconds is more than enough.

b) batchSize: fewer invocations, more responsibility

Lift leaves it at 1 by default to keep error handling simple 17. Raising it to 10 means a burst of 500 jobs is processed in 50 invocations instead of 500, reducing cold-starts and costs. In exchange, remember the two conditions we covered earlier: all jobs in the batch share the same Lambda clock, and our jobs must be idempotent in case an invocation dies mid-batch.

There's also maxBatchingWindow: how many seconds SQS waits to gather a full batch before invoking Lambda. Zero by default. If you don't mind a job waiting a few seconds for company to arrive, you can raise it.

c) reservedConcurrency vs maxConcurrency

If 1,000 jobs arrive at once, Lambda scales up fast to hundreds of simultaneous invocations. And that sounds wonderful until those 1,000 jobs slam into a rate-limited external API, our SMTP server, or DynamoDB itself all at the same time.

There are two ways to put a ceiling on it:

  • reservedConcurrency (function property): the function never runs more than N simultaneous instances. But it also reserves those N from your account's total concurrency quota, even when sitting idle 18.
  • maxConcurrency (Lift construct property): limits how many concurrent invocations this queue's event source mapping can trigger, without reserving anything 19.

The sweet spot is combining both: maxConcurrency as a clean brake (messages wait in the queue without burning attempts) and reservedConcurrency slightly above it, as a safety net in case a bug dispatches jobs in an infinite loop.

Heads up: if you use only reservedConcurrency, anything above that limit gets throttled, and every rejection counts as a delivery attempt in SQS. With maxRetries: 3 (Lift's default), a traffic spike can send jobs straight to the DLQ without ever having executed once. That's why AWS recommends setting maxRetries to at least 5 20.

d) memorySize: more memory is also more CPU

As we saw in the second chapter, CPU power in Lambda scales proportionally with allocated memory. A heavy job that takes 10 seconds with 512 MB might finish in 3 seconds with 1,769 MB; and since AWS bills per GB-second, giving it more memory often ends up being cheaper. If your worker does CPU-intensive work, you can assign its own memorySize and find the sweet spot with aws-lambda-power-tuning 21.

e) Tailored layers: extensions only where needed

Did you notice the ${bref-extra:gd-php-85} layer in the configuration? My worker needs the GD extension to process images, but the web HTTP function doesn't. Because they are separate Lambda functions, each one loads only the extensions it actually uses: web requests stay lightweight, and the extra weight is carried exclusively by the background worker.

f) Does the worker need a warmer?

In the second chapter we set up a warmer every 5 minutes to avoid cold-starts on web requests. Does the worker need one? Not at all. A 200 ms cold-start on an HTTP route is felt by a user staring at a loading screen; in a background queue, an extra 200 ms won't keep anyone up at night.


7. What Tutorials Don't Tell You (and You Find Out in Production)

Things that aren't in the docs (or were buried in the fine print and I learned the hard way):

a) SerializesModels and DynamoDB composite keys

Did you notice that in SendWelcomeEmail (section 5.a) I passed $user_uuid as a string instead of the User model? That wasn't an accident. SerializesModels stores only getKey() (PK) and tries to restore the model with a traditional SQL query that doesn't include the SK or support DynamoDB.

For your own jobs, the simplest approach is passing the uuid to the constructor and fetching the model inside handle(). However, if you use queued notifications ($user->notify()), Laravel serializes the User model whether you like it or not: in that case, you have no choice but to override getQueueableId() and newQueryForRestoration() on your model to encode and reconstruct PK + SK.

b) Bus::batch() also needs somewhere to store its progress

By default it looks for the relational job_batches table, but Laravel includes native DynamoDB support: you just need to create the job_batches table (partition key application, sort key id), grant IAM permissions, and set queue.batching.driver to dynamodb 22.

c) Your local environment doesn't need to touch AWS

Since our job code doesn't change based on the driver, on your machine you can keep using QUEUE_CONNECTION=database (or sync) with good old queue:work, or emulate SQS locally with tools like MiniStack 23 or LocalStack 24. QUEUE_CONNECTION=sqs pointing to AWS is only needed in serverless.yml.

d) The limit is 1 MiB per message

That's SQS's hard limit 25. Jobs should carry lightweight identifiers, not base64-encoded files or collections of 10,000 items. If you're anywhere near that limit, your design is screaming at you to have the job fetch the data when it runs instead of receiving it in the payload.

e) The AWS SDK cleanup from the second chapter

If you used aws/aws-sdk-php with removeUnusedServices in composer.json to shrink your deploy size, make sure to add Sqs to the list of allowed services:

"extra": {
    "aws/aws-sdk-php": [
        "DynamoDb",
        "S3",
        "Sqs"
    ]
}
Enter fullscreen mode Exit fullscreen mode

Why is it so easy to forget? Because tests run with QUEUE_CONNECTION=sync or Queue::fake(), so CI passes everything in green without ever instantiating SqsClient... until the first real dispatch() in production blows up because the class was stripped out.


8. Costs: The Worker That Only Charges When It Works

Let's update the tab, just like in every chapter.

  • Before (EC2): the worker lived on one of the t3.small instances from the first chapter (~$15/month per instance). A resident queue:work process with reserved RAM and supervisord running 24/7, even when the queue spent 90% of the day empty.
  • Now (SQS + Lambda):
    • SQS: the first million monthly requests are free; after that it costs \$0.40 per million 26. Adding up the ~630,000 requests from idle polling and 3 requests per job (SendMessage, ReceiveMessage, and DeleteMessage), you can process up to ~120,000 jobs per month for $0.00.
    • Lambda: the perpetual free tier includes 1 million invocations and 400,000 GB-seconds per month. With no jobs in the queue, compute costs literally zero.

Let's put numbers on a month with 100,000 jobs (averaging ~500 ms with 512 MB of RAM):

Item Within Free Tier Full Price (no Free Tier)
SQS Polling (~630k req/mo) $0.00 $0.25
SQS Messages (300k req/mo) $0.00 $0.12
Lambda Worker (100k jobs) $0.00 $0.44
Monthly total $0.00 ~$0.81

Even without the free tier, processing 100,000 jobs costs less than a dollar a month!

And most importantly: with DynamoDB on PAY_PER_REQUEST, Lambda billing per millisecond, and SQS billing per request, almost no component in the queue system charges a flat fee just for existing (except for SQS polling, which in my case falls within the free tier).


9. What's Next?

Today we retired supervisord and the 24/7 queue:work process for good: we replaced them with an SQS queue and a Lambda worker that only wakes up when there's work to do. We saw how Bref reuses Laravel's own Worker (so our jobs don't change), how its release() trick preserves retries and backoff, and how failed_jobs in DynamoDB coexists with SQS's Dead Letter Queue.

But there's still a loose end between the previous chapter and this one. In section 6 I mentioned that my worker loads the GD extension to resize images; now, if in the previous chapter we saw that files travel straight from the browser to S3 without touching our application... who tells Laravel that a file finished uploading so it can queue that job?

In the next chapter we'll close that loop with S3 Event Notifications, and we'll see why the solution is not wiring S3 directly to the queue we just built.


A question for you: what are you running your Laravel queues on in production today, or what was the worst stunt a queue worker ever pulled on you at 3 AM? Let me know in the comments: the best production war stories come from there, not from post-mortems.


  1. https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html#function-configuration-deployment-and-execution ↩

  2. https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/welcome.html ↩

  3. https://bref.sh/docs/laravel/queues ↩

  4. https://github.com/getlift/lift/blob/master/docs/queue.md ↩

  5. https://github.com/getlift/lift/blob/master/docs/permissions.md ↩

  6. https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessage.html#SQS-SendMessage-request-DelaySeconds ↩

  7. https://bref.sh/docs/laravel/queues#multiple-queues ↩

  8. https://docs.aws.amazon.com/lambda/latest/dg/services-sqs-scaling.html ↩

  9. https://aws.amazon.com/sqs/pricing/#How_are_Amazon_SQS_requests_priced.3F ↩

  10. https://github.com/brefphp/laravel-bridge/blob/a62196d5cb3efe95861fc6b78f7e5606bd2ad09d/src/Queue/QueueHandler.php#L53-L78 ↩

  11. https://github.com/brefphp/laravel-bridge/blob/a62196d5cb3efe95861fc6b78f7e5606bd2ad09d/src/Queue/Worker.php#L10-L39 ↩

  12. https://github.com/brefphp/laravel-bridge/blob/a62196d5cb3efe95861fc6b78f7e5606bd2ad09d/src/Queue/SqsJob.php#L13-L38 ↩

  13. https://github.com/brefphp/laravel-bridge/blob/a62196d5cb3efe95861fc6b78f7e5606bd2ad09d/src/Queue/QueueHandler.php#L107-L117 ↩

  14. https://laravel.com/framework/docs/13.x/queues#storing-failed-jobs-in-dynamodb ↩

  15. https://github.com/getlift/lift/blob/master/docs/queue.md#retry-delay ↩

  16. https://github.com/getlift/lift/blob/master/docs/queue.md#commands ↩

  17. https://github.com/getlift/lift/blob/master/docs/queue.md#batch-size ↩

  18. https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html ↩

  19. https://github.com/getlift/lift/blob/master/docs/queue.md#max-concurrency ↩

  20. https://docs.aws.amazon.com/lambda/latest/dg/services-sqs-configure.html#events-sqs-queueconfig ↩

  21. https://github.com/alexcasalboni/aws-lambda-power-tuning ↩

  22. https://laravel.com/framework/docs/13.x/queues#storing-batches-in-dynamodb ↩

  23. https://github.com/ministackorg/ministack ↩

  24. https://github.com/localstack/localstack ↩

  25. https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html ↩

  26. https://aws.amazon.com/sqs/pricing/ ↩

Top comments (0)