Serverless Redis with Cloudflare Workers & Pulumi

Cloudflare Workers are an exciting function as a service offering. They are built on the Service Worker web standard and are globally deployed on Cloudflare's edge network. This edge deployment makes them a low latency solution, so it comes naturally to use them with a low-latency database like Redis.

The problem is, Cloudflare Workers only allow requests to HTTP and HTTPS ports, so connecting to a Redis instance with a regular client isn't possible.

Luckily Upstash, a serverless Redis hosting provider, has now added a REST API to their product. This way, we can profit from the low latency of Redis inside a Cloudflare Worker.

Example Project

I created an example project with Pulumi, infrastructure as code tool like the AWS CDK, but it works with non-AWS cloud providers, like Cloudflare, too!

You can find the complete code on GitHub.

It creates a Worker script and links it up with a Worker route. This way, the script can be accessed from a custom domain.

Prerequisites

• Node.js & NPM installation
• Pulumi installation and Pulumi account
• Cloudflare account
• Upstash account

Setup

When you clone the project from GitHub, you have to login into your Pulumi account and provide the project with Cloudflare and Upstash credentials.

Cloudflare Credentials

First, we need to gather the Cloudflare credentials.

The API token can be created in the Cloudflare Dashboard.

Click on "Create Token" and then on "Edit Cloudflare Workers." This way, the token only has permissions to mess with Workers and not other Cloudflare resources.

If you click through that wizard, you will be presented with the token at the end, copy it, and use the following Pulumi CLI command inside the project directory:

$ pulumi set config cloudflare:apiToken <YOUR_API_TOKEN> --secret

Next, you need to get your Cloudflare account ID. You can find it in the Cloudflare Dashboard when you click on your domain. Scroll down, and on the right side is the account ID together with the zone ID, which we will need next.

$ pulumi set config cloudflare:accountId <YOUR_ACCOUNT_ID>

Then we need the zone ID from Cloudflare; otherwise, we can't use a custom domain for the Worker; it's right above the account ID we copied in the last step.

Open the index.ts file and replace <CLOUDFLARE_ZONE_ID> with your zone id.

new cloudflare.WorkerRoute("blog-route", {
  zoneId: "<CLOUDFLARE_ZONE_ID>",
  pattern: "example.com/posts",
  scriptName: blogScript.name,
});

You also have to replace the domain in the pattern with your domain; example.com won't work.

Upstash Credentials

Next, we need the Upstash credentials; otherwise, our Cloudflare Workers can't talk to the database.

First, we have to create a new database in the Upstash console.

Click on "Create New Database," give it a name, and choose a region. Then you can click on the freshly created database to get the credentials for the REST API.

There is a REST API button that displays an example request with cURL.

curl <YOUR_DB_URL>/set/foo/bar \
  -H "Authorization: Bearer <YOUR_DB_TOKEN>"

We need two parts here. The API token and the URL of the database.

The URL is everything between curl and /set/foo/bar, and the token is everything after Bearer (without the last quotation mark).

These two strings need to be added to the index.ts; this time, we need to replace the config for the WorkerScript.

const blogScript = new cloudflare.WorkerScript("blog-script", {
  name: "blog-script",
  content: workerCode,
  plainTextBindings: [{ name: "DB_URL", text: "<UPSTASH_DB_URL>" }],
  secretTextBindings: [{ name: "DB_TOKEN", text: "<UPSTASH_DB_TOKEN>" }],
});

The bindings we define make new global variables available inside the worker script. Namely DB_URL and DB_TOKEN.

Deployment

To deploy the whole thing, we have to run pulumi up. Pulumi will take care of provisioning the Cloudflare resources for us.

$ pulumi up

Usage

If everything went correctly, you should have a /posts endpoint on your domain that accepts GET, POST, and DELETE requests.

http://example.com/posts

How to Call Redis with REST?

Upstash has excellent documentation for their new REST API feature if you want to dive deeper, but let's look at some examples.

The Redis REST API is called like this:

GET https://us1-merry-cat-32748.upstash.io/redis-command/arg1/arg2/...?_token=...

To call the API directly in JavaScript could look like this:

const response = await fetch("https://us1-merry-cat-32748.upstash.io/incr/post:id?_token=...");
const { result } = await response.json();
const newPostId = result;

First, we send a request with fetch to the correct database endpoint. It includes the Redis command, the args, and the token.

Then we convert the response to a JavaScript object and extract the result property from it.

In the worker.js of the example project, I wrote a straightforward client for the REST API.

async function callRestApi(command, ...args) {
  args = Array.isArray(args[0]) ? args[0] : args;
  const response = await fetch(
    `${DB_URL}/${command}/${args.join("/")}?_token=${DB_TOKEN}`
  );
  const data = await response.json();
  return data.result;
}

The client uses the two global variables, the DB_URL and the DB_TOKEN, which we defined as bindings for our Worker script and a Redis command plus its arguments.

This utility function simplifies the calls to the database a bit. The example call from above would now look like this:

const newPostId = await callRestApi("incr", "post:id");

The API client can also be used with an array to execute a command with a dynamic amount of arguments.

A GET request to our Cloudflare Worker would execute the following code:

async function listPosts() {
  const postKeys = await callRestApi("smembers", "posts");
  const posts = await callRestApi("mget", postKeys);
  return new Response(
    JSON.stringify(posts.map((text, i) => ({ id: postKeys[i], text })))
  );
}

First, it gets all postKeys from a key called posts. The command is smembers, because the posts key is a Redis Set.

Then it gets all the posts with mget, the multi-get command of Redis, that accepts multiple keys as arguments.

In this example, the posts are just strings, not a complex object, so in the end, we use map to relate every text to its key.

Conclusion

Cloudflare Workers is a low-latency FaaS product, so using a low-latency database is the obvious choice here.

The port restrictions for Cloudflare Workers rule out some database choices, but Upstash gets around with its new REST API. It's as simple as you expect a Redis API to be, so not much new is to learn.

If you want to learn what Upstash brings that Workers KV doesn't, there is also a new article explaining all this.