DEV Community: Dan Farrelly

What is waitUntil (Vercel, Cloudflare) and when should I use it?

Dan Farrelly — Wed, 22 May 2024 17:06:55 +0000

Recently, Vercel announced that their waitUntil utility method is now available for all serverless functions. Now, this useful feature is available for both Vercel's Node.js and edge functions. This utility is not unique to Vercel -- Cloudflare Workers also offers a similar one.

This is a very useful tool so let's dive into what waitUntil is and what it does, when to use it, and when it's better not to.

What does `waitUntil` do

To understand what waitUntil does and why it exists, let's talk about how serverless functions work.

Serverless functions are designed to be stateless and short-lived. Unlike traditional servers, serverless functions are not kept alive between requests. Instead, they are spun up on-demand to handle incoming requests and then shut down once the response is sent.

With JavaScript serverless functions, you have to await all asynchronous operations to ensure that the function doesn't return before the asynchronous operation is complete. If you don't await an asynchronous operation, the serverless function might be shut down before the operation is complete. If this happens, the operation might be canceled or fail which results in unpredictable behavior. This is why it's important to await all asynchronous operations in serverless functions.

// ❌ BAD: If sendMetrics is an async function (a Promise), there is no guarantee it will succeed
async function POST(request) {
  const data = await createPost(request.body)
  sendMetrics({ metric: 'posts.created', increment: 1 })
  return new Response(data)
}
// ✅ GOOD: Awaiting will ensure it completes before the function returns
async function POST(request) {
  const data = await createPost(request.body)
  await sendMetrics({ metric: 'posts.created', increment: 1 })
  return new Response(data)
}

The downside of using await is that it can slow down the response time of your serverless functions. You ideally want to keep the critical path of your request minimal so you return the fastest response possible for your users. This is where waitUntil comes in.

waitUntil allows you to ensure that an async function (a Promise) completes before the serverless function is shut down. As the example code above doesn't require any data returned from sendMetrics, we can use waitUntil to ensure that the async function completes in the background without blocking the response from returning to the user.

// ✅ GOOD: The sendMetrics function will execute in the background,
// but not block the response from returning to the user
async function POST(request) {
  const data = await createPost(request.body)
  waitUntil(sendMetrics({ metric: 'posts.created', increment: 1 }))
  return new Response(data)
}

So, is waitUntil a solution to all possible problems? Well, it is a very useful tool and it ensures completion. However, it does not ensure success. For example, if the promise passed to waitUntil fails, the serverless function will still complete and return a response. So when should we use it?

When to use `waitUntil`

waitUntil is useful when you need to perform some asynchronous work that is not critical to the response of the serverless function. This is useful for things like sending metrics, logging, cache control, or other things that should not block the user's response. The user should not pay the penalty for these operations, so they can be done in the background.

When not to use `waitUntil`

As mentioned above, waitUntil does not ensure success. If the promise passed to waitUntil fails, the serverless function will still complete and return a response. This is important to keep in mind when deciding when to use waitUntil.

In other words, waitUntil is not the tool for the job if the asynchronous operation affects critical business logic in your application (which is precisely what makes it great for logging and metrics!). If the operation fails, it's not the end of the world. But if the operation is critical to the response of the serverless function, await is the way to go.

Can I run background jobs with `waitUntil`?

Short answer: No.

Long answer: waitUntil is not designed for running background jobs. There are no retries or tools for handling failures. Also, waitUntil will only run for as long as your function's timeout is set for, meaning if something takes longer than your timeout, it will be canceled. For running background jobs, you should use a system that handles queueing, retries, and logs failures.

Inngest is designed to handle background jobs reliably. It automatically handles retries, logs, and failures. If you need to run background jobs, you should consider using Inngest instead of waitUntil.

Should I use `waitUntil` with Inngest?

Inngest functions are truly async, background jobs. To trigger Inngest functions, you use inngest.send() which sends the data to the Inngest Event API. Inngest then triggers the function and handles retries on it's end. The inngest.send() function returns a Promise so it is possible to use with waitUntil. Should you use it though?

import { inngest } from "src/inngest";

async function POST(request) {
  const data = await createPost(request.body)
  waitUntil(inngest.send({ name: "post.created", data: data }))
  return new Response(data)
}

Typically, Inngest functions contain core business logic, so you need to ensure that the functions are triggered successfully. The key question is the reliability of waitUntil and the function it calls. Let's break down some things to consider:

The Inngest Event API has very high uptime and is designed to be reliable.
The inngest.send() function is very simple - mostly just an POST request to the Event API.
The waitUntil utility from Vercel and Cloudflare is designed to be reliable.
Networking issues or blips can always cause issues with outbound requests from any platform.
waitUntil will only last as long as the timeout for the function.

Given these points, it is safe to use waitUntil with inngest.send() with the caveat that if there is a networking issue, the event may not reach Inngest. While this is a rare occurrence, it is a risk to be aware of and is a trade off that you need to decide for yourself.

If you don't want to take this risk, you can use inngest.send() without waitUntil, but it will be blocking:

import { inngest } from "src/inngest";

async function POST(request) {
  const data = await createPost(request.body)
  await inngest.send({ name: "post.created", data: data })
  return new Response(data)
}

If you want the best of both worlds, you might determine a recovery strategy for when the event doesn't reach Inngest. This could be as simple as logging the failure and retrying the event later. Below is an incomplete example of how you might handle a failure and do something with the event. Not included is any logic you might need for retrying the event later using whatever logging backend or similar that you use.

import { inngest } from "src/inngest";

async function POST(request) {
  const data = await createPost(request.body)
  waitUntil(
    inngest.send({ name: "post.created", data: data })})
      .catch((error) => {
        // Log the event somewhere where you could read and re-send it later
      })
  )
  return new Response(data)
}

Conclusion

waitUntil is a highly useful tool to have in your serverless toolkit. It can help keep async code out of the critical path of the request to return the response to the user as fast as possible. It is very important to understand when to use it and when not to use it. It's also important to understand the risks of using it with other services.

Ideally, in the future, these services might offer some sort of out-of-the-box logging and retry mechanism for waitUntil. Until then, you should be aware of the risks and decide if it's right for your use case.

Debounce messages in queueing systems: How to do it with Postgres

Dan Farrelly — Fri, 05 Apr 2024 16:23:09 +0000

When developers talk about debouncing, it's most often discussed as a front-end development technique to handle issues in user interfaces. It's also quite a useful technique to apply to backend message queuing systems. In this post we'll cover:

what debouncing is,
when to use debouncing with queues,
how to implement this in a simple Posgtres queue,
and how you can use this feature with Inngest.

What is debouncing

Debouncing is used to avoid unnecessary executions. At its core, it is a technique to ensure a particular task or operation is executed only once after a specified delay since its last invocation, even if triggered multiple times within that delay.

A classic front-end example involves an auto-complete input where a user's actions, like typing, could trigger duplicate work by fetching search results. The goal is to send a single action, such as an HTTP request, only after the user has finished typing.

When to use debouncing in queuing systems

Debouncing within queueing systems is most important when you're aiming to improve efficiency of execution. It might be a user that triggers a job that's costly to run or some event that triggers a larger job to execute that is wasteful to have to re-run. Here are couple of specific situations where you may want to add debouncing into your queue and worker system:

Cost efficiency with AI / LLMs. Leveraging any sort of AI is expensive. Whether you're using OpenAI's APIs or running your own models on GPU-optimized instances, leveraging AI can be costly. Depending on what is triggering your AI code to run (for example, a user saves data in a form) it may be triggered twice within a short time window. Debouncing this trigger can help ensure there is no wasted re-work with AI.
Data synchronization and integrity. In systems involving data synchronization or batch processing, it's often essential to ensure that operations are performed only when all relevant changes have been completed. Debounce can facilitate this by delaying execution until a predefined idle period has passed, ensuring data integrity and consistency.
User interface interactions and auto-save. Similar to the example mentioned above, background jobs are often triggered by user actions that save data. If a user saves the same data in quick succession, or perhaps the front-end implements auto-save, it is likely that multiple background jobs will be triggered. Adding a debounce delay to your background job can make your system more efficient.
Deduplicating chatty webhook events. While webhook events are great to integrate with third-party systems, some webhook providers, for example Intercom, may trigger multiple events in short succession for the same or similar actions that you only need to react to once. Also, often when building webhook integrations, you'd design for eventual consistency, so debounce delays are acceptable.

While debouncing is traditionally associated with frontend development and user interface interactions, its principles and techniques can also be applied effectively in backend software development to optimize resource utilization, mitigate performance bottlenecks, and enhance the scalability and reliability of distributed systems and event-driven architectures.

How to implement debouncing in a message queue

Depending on what backing queue you use, implementing debounce can be simple or difficult. At the basic level, your system needs to be able to schedule a given message at a timestamp in the future (the delay) and when the same message is sent, the currently scheduled message's timestamp is “pushed” back by the delay amount again. Your queue then must only pick up messages that are ready (the current timestamp).

Here we'll cover a basic implementation of a Postgres queue and look at how to add debounce. There are more advanced techniques for queuing in Postgres, but we will not cover that here.

First, we'll create a simple debounce_job_queue table in our database. It will have a job id , a job_type , a current status for the job, a payload for relevant job data, and a scheduled_at timestamp. Lastly, we'll add a debounce_key which will be what we'll use to de-duplicate and delay work.



CREATE TABLE debounce_job_queue (
  id SERIAL PRIMARY KEY,
  job_type VARCHAR(64),
  status VARCHAR(64),
  payload JSONB,
  scheduled_at TIMESTAMP,
  debounce_key VARCHAR(64)
);

As we want to de-duplicate jobs waiting the queue, we'll add a unique index to ensure that there is only one matching job that is 'QUEUED' at a given time.



CREATE UNIQUE INDEX debounce_key_status
ON debounce_job_queue(debounce_key, status)
WHERE status = 'QUEUED';

Creating a new job in this queue is a relatively straightforward insert, but we add a few things that help us implement debounce:

Create a deterministic debounce_key using a hash of the job's type and payload JSON data. Alternatively, this could be any unique piece of data like a user's ID.
Allow the calling function to specify a delay when setting the scheduled_at.
Handle conflicts with existing queued items with an UPDATE to the existing item's scheduled_at with the specific delay. This is where the debouncing happens.

We'll use the @vercel/postgres library's simple sql here, but your favorite library will do just fine.



import { crypto } from "node:crypto";
import { sql } from "@vercel/postgres";
async function enqueue(jobType, payload, delay = 0) {
  const json = JSON.stringify(payload);
  const debounceKey = crypto
    .createHash("sha256")
    .update(jobType + json)
    .digest("hex");
  return await sql`
        INSERT INTO debounce_job_queue (
            job_type, status, payload, scheduled_at, debounce_key
        )
        VALUES (
            '${jobType}', 'QUEUED', '${json}',
            now() + INTERVAL '${delay} seconds', ${debounceKey}
        )
        -- If the item is already QUEUED, update the scheduled at w/ the delay
        ON CONFLICT (debounce_key, status) WHERE status = 'QUEUED'
        DO UPDATE SET scheduled_at = now() + INTERVAL '${delay} seconds';
    `;
}

The code above may appear complex at first, but the magic happens in the ON CONFLICT statement. Our unique index ensures that two identical jobs cannot be 'QUEUED' at the same time. When the query attempts to insert a job and there is an identical job that is QUEUED, the conflict statement is executed and the existing job's scheduled at is updated with our delay to push it back.

To query for available jobs, we'll also now use our scheduled_at timestamp to ensure that our function only fetches jobs that are ready to be processed. When jobs are debounced and the scheduled_at is pushed into the future, that prevents our query from fetching that job.

In our query, we'll also update the status to PROCESSING to mark that this job should not be picked up by any other worker and that it shouldn't be debounced again as it's being executed. In our code, we'll omit the worker polling logic and focus on the query. We'll use SKIP LOCKED here to prevent only query for rows that are locked due to being updated elsewhere.



async function getQueueItem(jobType) {
  return await sql`
    UPDATE debounce_job_queue
    SET status = 'PROCESSING'
    WHERE id = (
      SELECT id FROM debounce_job_queue
      WHERE status = 'QUEUED' and scheduled_at <= now() + INTERVAL '2 minute'
      ORDER BY scheduled_at ASC
      FOR UPDATE SKIP LOCKED
      LIMIT 1
    )
    RETURNING id, payload;
  `;
}

That's it! In your own worker code you'll want to also DELETE this item from the queue table when it's done processing (or similar). If you are implementing a debounce queue in Postgres, you may want to additionally consider:

Adding other indexes to your table for performance.
Adding a job expiration timestamp and clearing of expired jobs.
Adding a column to track the number of attempts to process a given item and handling a maximum number of retries.
Implementing a dead letter queue.
Learning more about using SKIP LOCKED .

Next, we'll compare this to implementing debounce with Inngest.

How to debounce with Inngest functions

Inngest's architecture combines an event stream, queues, and durable execution into a single reliability layer for your application. With Inngest's internal queueing system, you can easily debounce jobs.

The main differences to the above example is that with Inngest:

You send events instead of messages.
You define functions which each has their own queue(s) created automatically for you.
Inngest calls your function for you, so you don't need to implement polling or similar in a worker.
You can define your debounce key using any piece of data within your event's payload.

Configuring debouncing for an Inngest function is a couple of lines of code. In our example below, we'll create a function that generates suggested titles for a blog post every time a user saves a draft. We'll add a debounce period of one minute and we'll set a debounce key to only debounce based off the given blog_post_id . Setting the key will make sure that each unique blog post will be separately debounced.



const generateIdeasFunction = inngest.createFunction(
  {
    id: "generate-ideas-via-open-ai",
    debounce: {
      period: "1m",
      key: "event.data.blog_post_id",
    },
  },
  { event: "blog/draft.saved" },
  async ({ event, step }) => {
    /* call OpenAI */
  },
);

For more information on this, check out our guide on debouncing or check out our quick start tutorial to learn the basics of Inngest.

Conclusion

Debouncing is a powerful and highly useful technique in many queueing use cases. This is why we built it right into Inngest so any developer can use it with any function with ease. We'd love to hear what other use cases where you use debouncing and any other features that you have added!

If you found this blog post interesting, check out another one on how we've built our multi-tenant queuing system.

Run Next.js functions in the background with events and cron schedules

Dan Farrelly — Thu, 12 May 2022 13:48:48 +0000

There comes a point in time when your application gets too complex to be completely comprised of synchronous API endpoints. APIs are fantastic for CRUD actions that your users are taking that need an immediate response, but there are use cases when you want to do some work in your application in the background:

Non-critical path tasks - When your API needs to do some work that isn't truly crucial, you should move it off of the critical path of the request to ensure a timely response time for your users. Your users shouldn't have to wait longer for a response because your app needs to send an email, update a CRM or perform some data aggregation to store in your database.
Scheduled tasks - You need to run something periodically like send a weekly digest email to your users or do a periodic data cleanup.

To accomplish these types of things, you usually have options that require more infrastructure to be setup like message queues, pub sub, or deploying your code to another platform that does scheduling. There are two considerations for this:

Time & Complexity: It requires building out some of your own infrastructure which is time consuming and, if you're not familiar with these architectures, you learn the issues and pitfalls and the hard way.
Logs & Observability: You don't get logs, metrics, or traces about whats happening in the background without even more configuration of other services and instrumentation of your code.

The Inngest platform can do this for you with full observability and audit trails without having to configure new infrastructure. Let's see how we can get this done with your existing Next.js project running on Vercel or Netlify in just a couple of minutes.

Schedule functions

If you wanted to send a weekly email digest to your users, you'll want to run this once a week at a particular time. Maybe you've written the code to query your database for users, query their weekly product usage, and format a well-designed email to send them. It's a fantastic feature and user retention tool. Here's some hypothetical code in a Next.js api route:



/* /pages/api/sendWeeklyDigests.js */
export default async function handler(req, res) {
  const results = await sendWeeklyDigestEmailsToAllUsers();
  res.status(200).json({
    message: `Weekly digests sent to ${results.sent} users successfully`,
  });
}

To schedule this on Inngest is just a couple quick steps:

Install the inngest cli:



$ curl -sfL https://raw.githubusercontent.com/inngest/inngest-cli/main/install.sh | sh && \
  sudo mv ./inngest /usr/local/bin/inngest

Navigate to your project's repo and initialize a new function. Here you get to choose your schedule - you can change this later (Crontab Guru is useful for generating a schedule). Select “Call a URL” - this will be the URL of your endpoint that you'd like to request. Here we'll use once a week on Monday at 12pm (0 12 * * 1):



$ inngest login
$ inngest deploy

Creating a scheduled function in the Inngest web app

You can also quickly create a scheduled function in the Inngest web app. From the "Functions" tab, click the "New Function" button and select "Call an existing HTTP endpoint." At the right, you'll be able to click the function "Trigger" drop down and select "Run on a schedule." You then can choose to "Run" to test your function or "Deploy" to deploy it make it live!

Background jobs and event-driven functions

Moving code out of the critical path of a request can give you several benefits in your API backend. If you haven't done this before, some advantages to moving some code to a background job or function:

Decouples key logic which you can re-use
Ensures the initial API request stays fast as possible
Provides an audit-trail of what jobs got triggered by who and when

When you implement this pattern, your initial endpoint shoots a message off to Inngest and immediately responds, allowing you to return a response to the user. Inngest logs the message (aka event) then dispatches a request to the endpoint of your choosing and waits for the response, logging the response code and body.

The first step is choosing your event message to send to your background function. This is comprised of an event name and any relevant data about that event and the associated user. For this example, our event name will be user.signup and we'll pass the user's email and their role on their team captured in your signup survey. Our event will look like this:



{
  name: "user.signup",
  data: {
    signupReason: "some string"
  },
  user: {
    email: "the.user.email@example.com"
  }
}

Let's define the function that will be called:

Install the inngest cli



$ curl -sfL https://raw.githubusercontent.com/inngest/inngest-cli/main/install.sh | sh && \
  sudo mv ./inngest /usr/local/bin/inngest

Navigate to your project's repo and initialize a new function. Here you get to choose your an event as the trigger and enter the new user.signup. Select “Call a URL” - this will be the URL of your endpoint that you'd like to send the request, e.g. https://myapp.com/api/sendWelcomeEmail.



$ inngest init

Let's get you set up with a new serverless function.
Answer these questions to get started.

1. Function name: Send Welcome Email
2. Function trigger: Event based
3. Function type: Call a URL
4. Event trigger: user.signup

🎉 Done!  Your project has been created in ./send-welcome-email
For more information, read our documentation at https://www.inngest.com/docs

Using the inngest JavaScript library, we can send this event in our API's signup function (You can create as many unique source keys as you want):



/* /pages/api/signup.js */
import { Inngest } from "inngest";

export default async function handler(req, res) {
  const { email, password, signupReason } = req.body;
  const result = await createNewUser(email, password);
  const inngest = new Inngest(process.env.INNGEST_SOURCE_KEY);
  await inngest.send({
    name: "user.signup",
    data: { signupReason },
    user: { email },
  });
  res.status(200).json({ success: true });
}

Our background job function will receive this event as the req.body, so, for this example, our function could look like this:



/* /pages/api/sendWelcomeEmail.js */
export default async function handler(req, res) {
  const { event } = req.body;
  const result = await sendEmail({
    template: "welcome-email",
    to: event.user.email,
    data: {
      // The template will use this to show useful content to our new user
      signupReason: event.user.data.signupReason,
    },
  });
  const messasge = result.ok ? "Successfully sent" : result.error;
  res.status(result.ok ? 200 : 500).json({ message });
}

Now we have everything in place and you can deploy your function to Inngest:



$ inngest login
$ inngest deploy

When you next deploy your Next.js app you'll now start offloading work to your background job! Remember to add the Inngest Source Key to your environment variables first 😉.

Creating a background job or event-driven function in the Inngest web app

Very similar to creating a scheduled function above, you can create a function in our web app that is triggered by an event. Clicking on the default event trigger will allow you to select a new event. You can create a new event of your own and name it whatever you'd like. You can also easily edit the test event paylod and click "Run" to send the request to your server. When you're happy with what you've got, click "Deploy Funciton" to make it live!

Viewing event and function history

When you deploy either a scheduled function or a background job, you'll get full history of the event messages that your app has sent and the responses that your background function has sent.

Now go ship something!

Congrats! You now know how you can easily move key logic in your app to be asynchronous and out of the critical path of a request. Background jobs are important for any scaling and performant application that you have so we think this will really help you grow your amazing products.

PS - If you need to run background jobs for event longer (up to 15 minutes!), you can deploy your functions directly to Inngest: Here's a quick start guide to create and deploy a TypeScript function

DEV Community: Dan Farrelly

What is waitUntil (Vercel, Cloudflare) and when should I use it?

What does waitUntil do

When to use waitUntil

When not to use waitUntil

Can I run background jobs with waitUntil?

Should I use waitUntil with Inngest?

Conclusion

Debounce messages in queueing systems: How to do it with Postgres

What is debouncing

When to use debouncing in queuing systems

How to implement debouncing in a message queue

How to debounce with Inngest functions

Conclusion

Run Next.js functions in the background with events and cron schedules

Schedule functions

Creating a scheduled function in the Inngest web app

Background jobs and event-driven functions

Creating a background job or event-driven function in the Inngest web app

Viewing event and function history

Now go ship something!

What does `waitUntil` do

When to use `waitUntil`

When not to use `waitUntil`

Can I run background jobs with `waitUntil`?

Should I use `waitUntil` with Inngest?