Updated Queue Rework For Masonite 2.1

Preface

I would like to write this post to demonstrate some of the changes in this release since this is a bit larger than a few line file change.

The queue feature in Masonite allows you to put jobs into a few supported drivers in order to run them at a later date, either asynchronously or through some message broker like RabbitMQ.

The original queue feature was written in about 3 days and put into Masonite and released. It did everything i personally needed it to do and since it scratched my itch then maybe other Masonite developers would find it good too.

During the development of Masonite 2.2 (will be released in June 2019), I wanted to take a few concepts and "rework" them to make them better internally which usually just entails rewriting the implementation from scratch without touching the external interface.

In other words I wanted to do a pretty big rewrite but also make it an non breaking change that can go into the next minor version of the framework.

So in this post I'll talk about all the new features of the Queue Rework.

queue:work Command

There are a few changes to the queue:work command. You can now specify the driver that you want to start a worker for using the -d or --driver option:

$ craft queue:work -d amqp

This will now run a queue worker for the specific driver.

Failed Jobs

Failed jobs are a really awesome new addition to Masonite. Before when a job failed it sort of got stuck in the queue. You would have to purge out the job if it failed.

Now the job inside the queue will try to run 3 times back to back with 1 second in between before finally calling the job a failed job.

If the job fails it will call the failed method on the job. A normal job might now look something like:

from masonite.queues import Queueable
from masonite.request import Request
from masonite import Mail

class SendWelcomeEmail(Queueable):
    def __init__(self, request: Request, mail: Mail):
        self.request = request
        self.mail = mail

    def handle(self):
        return 2/0 # fails

    def failed(self, payload, error):
        self.mail.to('[e-mail]').send('
            The welcome email failed with error {}
        '.format(error))

The first parameter is the payload which tried running which is a dictionary of information that looks like this:

payload == {
    'obj': <class app.jobs.SomeJob>,
    'args': ('some_variables',), 
    'callback': 'handle', 
    'created': '2019-02-08T18:49:59.588474-05:00', 
    'ran': 3
}

and the error may be something like division by zero.

Now you can get a pretty proactive look at your jobs when they fail

Saving Failed Jobs

If a job fails it will sort of just disappear. You cannot run a failed job again.

This is why we now have a table and migration for failed jobs. You can create the table and migrate it into your database by running:

$ craft queue:table
$ craft migrate

This will create a new migration file for you into your migration directory. You can then migrate manually.

Now when a job fails it will actually serialize it and store it in this table. In order to run jobs again you can run

$ craft queue:work --failed

which sends all the jobs into the queue again to be re-ran.

Internal Rewrite

For those developing drivers for Masonite. This feature did to an internal rewrite of how drivers are written. All drivers must now inherit from BaseQueueDriver which gives each driver a few features now to be compliant with Masonite.

Since I know there are no other queue drivers currently written for Masonite, I feel comfortable making this change.

The contract for queues now has more methods in it. Meaning more methods are required to create a queue driver that is in compliance with the queue contract.

The queue:work command is now queue driver agnostic when previously it was only for the amqp driver. As long as your driver inherits from the QueueContract then you will be fine to add a driver to Masonite.

---

If you would like to contribute or support Masonite's development, please head over to the Masonite Github and give it a star!