DEV Community

Paul Chin Jr.
Paul Chin Jr.

Posted on

Serverless background tasks with OpenJS Architect - Part 3

Queues example

@queues are very similar to @events because they also allow for asynchronous message processing. @queues will provision an AWS SQS queue and register a Lambda function to handle messages that are sent to the queue. There are notable differences between @queues and @events. While @events push messages to all of its subscribers, @queues will poll for messages. Queues work on the first message in the queue before moving onto the next. Queues will also keep retrying until it is delivered for up to 4 days.

Let's make an example message queue by starting with a fresh Architect project

npm init @architect ./arc-queues-app
cd arc-queues-app
npm install @architect/functions
Enter fullscreen mode Exit fullscreen mode

Open up the app.arc file and modify the manifest to include our @queues function as follows:

# app.arc
@app
arc-queues-app

@http
get /

@queues
account-signup
Enter fullscreen mode Exit fullscreen mode

After modifying the app.arc file you can run arc init from the project root to scaffold the function folders with Architect.

The queue function uses SQS as an event source. You can use this pattern to move data between Lambda functions and using the queue as a temporary data store. To write a queue function, make a new file in /src/queues

// src/queues/account-signup/index.js
exports.handler = async function queue (event) {
  event.Records.forEach(record => {
    const { body } = record;
    console.log(body);
  });
  return {};
}
Enter fullscreen mode Exit fullscreen mode

Now we can modify our get-index function to publish a message as follows:

// src/http/get-index/index.js
let arc = require('@architect/functions')

exports.handler = async function http(req) {
  let name = 'account-signup'
  let payload = {body: req.body, text: 'yolo'}
  await arc.queues.publish({name, payload})
  return {statusCode: 201}
}
Enter fullscreen mode Exit fullscreen mode

Run npm start from the root of the project to start Sandbox. Navigate to http://localhost:3333 and take a look in the terminal for our output.

You should see the @queue event object being logged in the console.

@queue {
  "name": "account-signup",
  "payload": {
    "text": "yolo"
  }
}
Enter fullscreen mode Exit fullscreen mode

This event has name and payload keys which are reflected in the records when the queue is polled by the Lambda. You should also see the output from the account-signup function handler:

{"text":"yolo"}
Enter fullscreen mode Exit fullscreen mode

Let's follow how Architect implements @queues by default with CloudFormation. Run arc deploy --dry-run and open up sam.json. You should see that AccountSignupQueueLambda has an Events source specified by type of "SQS":

"Events": {
          "AccountSignupQueueEvent": {
            "Type": "SQS",
            "Properties": {
              "Queue": {
                "Fn::GetAtt": [
                  "AccountSignupQueue",
                  "Arn"
                ]
              }
            }
          }
        }
Enter fullscreen mode Exit fullscreen mode

Architect also creates an SQS Queue with CloudFormation. Notice below that the queue is set to FIFO, instead of standard. This means this Lambda will try to ensure that it handles all messages in order as they come in batches.

"AccountSignupQueue": {
      "Type": "AWS::SQS::Queue",
      "Properties": {
        "VisibilityTimeout": 5,
        "FifoQueue": true,
        "ContentBasedDeduplication": true
      }
    },
Enter fullscreen mode Exit fullscreen mode

The @architect/functions library has publish() and subscribe() methods that wrap the JSON payload with a compatible Lambda function signature.

Top comments (0)