Invoke HTTP from AWS Lambda without Waiting

What is the fastest way to invoke a HTTP/REST URL from an AWS Lambda?

The answer is pretty straight forward if you need a response.
But what if you don't want to wait for a reply?

Non-Waiting HTTP

Consider the case of a Lambda function that after processing, accumulates some metrics and then emits those back to a REST aggregator service for processing. If the metrics are counters, it may not matter if the metrics request fails occasionally, as the next request will send the updated counter totals. In cases like these, it will be faster not to wait for the HTTP response.

So how would we do this in NodeJS?

Let's look first at a naive example request (without error handling) where we wait for the response.

Just for this example, we'll create an async "request" node function that returns a promise that we can wait on.

const https = require('https')
const URL = require('url')

async function request(url, data) {
    return new Promise((resolve, reject) => {
        let req = https.request(URL.parse(url), function (res) {
            let body = ''
            res.on('data', (chunk) => { body += chunk })
            res.on('end', () => { resolve(body) })
        })
        req.write(data)
        req.end()
    })
}

Our Lambda would then look like:

exports.handler = async (event, context) {
    ...
    let result = await request('https://metric-service.com', metrics)
    return result
}

To invoke our HTTP request inside a Lambda, we use "await" which issues the request and then waits for the response.

If the metric aggregator service takes 950 milliseconds to process the request and return a status code we will be billed for an additional second on every invocation. During this wait time, our Lambda function is asleep, but AWS is still billing us for that time. With AWS Lambda, you are billed for elapsed time, not for utilized CPU time. While Lambda is extraordinarily cheap, with high transaction volumes, these short waits of 950 milliseconds can add up to a significant bill.

Don't Wait

So what happens if we simply do not call "await" and thus not wait on the response from our HTTP request?

exports.handler = async (event, context) {
    /* nowait */ request('https://example.com', metrics)
    return 'done'
}

Strange things happen.

Sometimes the request is sent and sometimes the request is not.

Sometimes the request is received immediately by the metrics aggregator and sometimes the request is received after our Lambda next runs. What is happening?

Freezing Lambda Containers

Lambda functions run inside an AWS Firecracker container. When you return from your Lambda function, AWS immediately freezes the container and its global state. When the Lambda is next invoked, it will thaw the container to service the new invocation.

If we send a HTTP request and that request is not fully sent over the network, if we return from the Lambda function, AWS will immediately suspend our Lambda container AND the partially sent request will be suspended as well. The request will remain frozen until the Lambda is next invoked and the Node event loop will then resume processing and the request will be fully transmitted.

How to Solve?

We could try a short sleep to give time for the request to be sent? But how long should we wait.

exports.handler = async (event, context) {
    /* nowait */ request('https://example.com', metrics)
    await sleep(100)
    return 'done'
}

This hardly seems reliable.

Wait for Request Transmission

The correct solution is to use the Node req.end(,,callback) API and wait until the request is fully sent, but not wait for the response to be received. Here is a sample:

const https = require('https')
const URL = require('url')

async function request(url, data) {
    return new Promise((resolve, reject) => {
        let req = https.request(URL.parse(url))
        req.write(data)
        req.end(null, null, () => {
            /* Request has been fully sent */
            resolve(req)
        })
    })
}

Notice that the request is resolved via the end callback on the "req" object and not on the "res" object in the previous example.

This modified request function should be invoked by our Lambda with "await". In this case, we are not waiting for the HTTP response, but rather for the request to be fully sent. This is much faster than the 750 milliseconds to receive our metrics response, typically under 20 milliseconds.

exports.handler = async (event, context) {
    await request('https://example.com', metrics)
    return 'done'
}

Alternative Design Patterns

There are many other excellent ways to avoid waiting and blocking in Lambda functions such as using Step functions, SQS queues and directly invoking Lambda functions as Events without waiting. Consider the best approach for your app, but if you must use HTTP and you don't need to wait for a response, consider the technique above to lower your wait time and AWS bill.

SenseDeep

Our SenseDeep serverless studio uses this technique in a Watcher Lambda that monitors your Lambdas, runs alarms and ingests log data. We needed the Watcher to be exceptionally fast and not wait for any REST/HTTP API requests. So the Watcher uses this non-waiting technique when sending status back to the SenseDeep service.

To learn more about the SenseDeep Serverless Troubleshooting platform and try it for free, please go to: https://www.sensedeep.com.

Please let me know if you have any comments using similar or different techniques. [e-mail].

References

Here are some other good reads about Lambda and AWS asynchronous programming.

• Node event loop freezing:
  https://levelup.gitconnected.com/avoiding-the-pitfalls-of-async-node-js-functions-in-aws-lambda-941220582e7a

• Lambda Billing:
  https://www.jeremydaly.com/serverless-tip-dont-overpay-when-waiting-on-remote-api-calls/

• Async programming:
  https://read.acloud.guru/save-time-and-money-with-aws-lambda-using-asynchronous-programming-3548ea65f751

• Perils of step functions:
  https://blog.scottlogic.com/2018/06/19/step-functions.html

• Invoke Lambda async:
  https://docs.aws.amazon.com/lambda/latest/dg/invocation-async.html

• Lambda invocation overview:
  https://docs.aws.amazon.com/lambda/latest/dg/lambda-invocation.html