DEV Community: Tony Holdstock-Brown

Modern serverless typescript job queues in >= 2022

Tony Holdstock-Brown — Wed, 21 Sep 2022 19:09:24 +0000

We’re closer to Jan 2023 than 2022. Serverless has taken over our stack, but we’re still struggling with stateful job queues for background work & scheduled jobs. We have to configure Redis, configure our queues, set up long-lived workers, and then handle observability, capacity, and failures ourself.

Why is this important? You need background jobs to have a reliable and fast system. But...

Background jobs aren’t modern, and it’s making our lives harder than it needs to be.

In this post, we’ll talk about what a modern job system looks like, how you’d use it, and then show a library that implements everything already 😎

What would a modern TypeScript based job queue look like?

Truly serverless: we shouldn’t have to configure redis, servers, capacity, etc.
Zero config: we don’t want to have to configure each queue before we need it.
Fan out (event driven): you should be able to trigger > 1 job from a single call
Fully typed: you should always know the data available to each function with full types
Reliable: failures should be retried automatically
Able to handle complexity: jobs are complex, and you should be able to specify a function as many individual steps for parallelism and reliability
Kitchen sink: a good platform should also handle things like idempotency, throttling, backoffs, and scaling for you

The world has gone serverless. We shouldn’t have to worry about scaling, failures, and stateful servers. We shouldn’t also have to worry about configuring queues, or jobs failing because the data you pushed onto a queue was invalid. And we should be able to run more than one function at once!

Alongside serverless, the secret sauce is: job queues should be event-driven. Node has an event emitter built in, and here’s the basics: you write functions that listen for specific events. When you send those events, the relevant functions run. Simple! This should be the way background jobs work, too.

Designing the API

Creating serverless background jobs should be simple, with zero setup:

import { createFunction } from "inngest";

export default createFunction(
  // Function name, for observability, logs, and metrics
  "Send welcome email",
  // Event name that runs the function
  "app/user.created",
  // The function code
  ({ event }) => {
    sendEmailTo(event.data.id, "Welcome!");
  }
);

And running them should be easy:

// Send events
import { Inngest } from "inngest";
const inngest = new Inngest({ name: "My App" });

// Run the function above automatically, in the background
inngest.send("app/user.created", { data: { id: 123 } });

Events trigger any number of functions automatically, in parallel, in the background. Inngest also stores a history of all events for observability, testing, and replay.

This solves all of our problems. We can use any framework (like NextJS) to build serverless functions that deploy to Vercel, Netlify, or Cloudflare, then run reliable background functions that do anything you need:

Sending emails
Calling external APIs
Run functions on a schedule to generate reports
Anything that shouldn't happen in your core API to make your API faster and more reliable

You should never have to set up Redis, configure queues, set up SQS or SNS on AWS to get this functionality out of the box.

We've released our SDK that allows you to do this in any javascript or typescript project today, for free: https://github.com/inngest/inngest-js. You can read the docs here: https://www.inngest.com/docs

I'd love to break down how this works under the hood and this post is already quite long. Let me know your thoughts and if you want to see the architecture and system design!

Building infrastructure for an open-source programmable zapier

Tony Holdstock-Brown — Fri, 19 Aug 2022 18:30:00 +0000

Hi community! Looking forward to hearing your thoughts about this.

Inngest was started out of the pain of building products that required complex integrations and background jobs — especially those driven by user interactivity. We’ve built the infrastructure so many times that we decided to open-source it for other people — so that everyone can have access to world class tooling, instead of just MANGA.

We’ve created a single API that lets you record what’s happening in your app, then lets you run serverless functions in response to these events. It’s a developer’s Zapier.

It’s more than a job queue or serverless

From our past experience, we know that building out reliable products requires more than a job queue, Kafka, or Lambda, and off-the-shelf products don't cut it for the customization us developers need.

We believe that it should be accessible to everyone — not $just 1B+ companies who have hundreds of people to configure and maintain infra.

Even for something as basic as reliably handling a webhook you’ll need some sort of queue, retries and logging which can take weeks to build, even with off the shelf systems in clouds. Here’s our plan to give this to everyone, from jamstack APIs all the way to realtime elixir webapps:

Event-driven infrastructure foundations

Using a single API to record what’s happening in your app provides rich functionality like:

Running step functions in real time, whenever specific things happen, automatically. This includes off-the-shelf, repeatable functions, like sending emails, forwarding events to analytics, or common APIs.
Historic replay and retries — by saving events we allow you to replay and retry past data locally on your own machine if there are errors, or retry your functions if there are errors.
Personalized debugging — when your users do things in your app, you can assign the events that are generated to that user. This lets you see every function that ran for each of your users, making debugging and insight super easy
Coding best practices. Sending an event describing what’s happening then reacting to the event reduces coupling and makes your critical path faster.
No servers or job queues. You can make everything serverless by scheduling functions to run in the future when an event comes in, making your app entirely serverless
Monitoring and visibility, allowing you to see what’s happening in your app and the functions that are running
Complex coordinated functionality, such as “on cart added, wait 24 hours for the checkout event — and only run the next step if the user didn’t check out”. This is annoying to build manually, and usually requires complex cron jobs, messy database state, or external (expensive) services like braze.com

Local development first

We also think that local development should be a first class part of the process, without spending days installing and configuring services locally. The open source CLI already does the following:

Makes it easy to create functions in any language, either from a blank slate or using quick starts for common use cases
Locally run functions with a single command — using the same execution engine deployed to your cloud, with a single command. This makes sure you have local <> production parity
Start a fully functional local test environment in a single command — which accepts events then runs functions locally
CI/CD out of the box, using things like github actions
Replay real production events locally for debugging & testing.

We want everyone to have world-class infrastructure to build the complex parts of their app — the stuff that’s beyond basic API routes. This includes things you’d only dream of at fortune 500s: serverless functions, event schema generation, event and function versioning, blue-green deploys, throttling, idempotency, and local debugging.

Even for basic use cases like handling webhooks, this makes your life far easier. And we want this for everyone, available locally, with the source freely available to hack on.

Status & next steps

We’ve already got our execution engine and CLI that does the above. We’re planning webassembly support, step-over debuggers, and a ton more. What would you like to see? Do you have job queues or events in your system? What would your ideal ‘programmable zapier’ do?

Feel free to comment here or on Github at https://github.com/inngest/inngest.

Build a realtime websocket UI using SvelteKit

Tony Holdstock-Brown — Tue, 22 Feb 2022 20:11:39 +0000

We used SvelteKit to make https://typedwebhook.tools - which allows you to test HTTP requests and automatically generates typescript types for the body. Inspired by that, let's run through how to make a realtime websocket UI using SvelteKit.

If you want a sneak preview, the code for typed webhooks is open source: https://github.com/inngest/typedwebhook.tools

Why SvelteKit?

Using Vite, it builds fast in dev. And it has HMR with state persistence out of the box. Somehow this is consistently broken in every react app, no matter what kit you use :D
It has SSR out of the box. It’s built for progressive enhancement, and configuring pre-rendering is the easiest I’ve seen
State management is easy. It’s easy to work with stores. You can (broadly speaking) use the stores from anywhere: no top-level context necessary (ahem, hello websockets!)
SvelteKit comes with a standard way of doing things (CSS, JS, forms, state, routing), so it’s easy to work with and it’s easy to share amongst devs. It’s easy to get set up and running with your entire framework — think a mixture of NextJS and create-react-app for Svelte.

Plus, it's amazingly developer friendly

Getting started

Make sure that you have Node & NPM installed, then run:

npm init svelte@next

It'll run you through a guide to set up your base project. Here's how we answered those questions:

✔ Where should we create your project?
  (leave blank to use current directory) … realtime-demo
✔ Which Svelte app template? › Skeleton project
✔ Use TypeScript? … Yes
✔ Add ESLint for code linting? … Yes
✔ Add Prettier for code formatting? … Yes

Let's go to that directory and run the dev server:

cd ./realtime-demo && yarn dev

If you go to localhost:3000 you should see Svelte up and running!

The current code lives in ./src. The routes folder (./src/routes) acts as a router: ./src/routes/index.svelte is the index page rendered by default, and ./src/routes/about.svelte is rendered when you navigate to /about.

You might be asking yourself "where do shared components go?". They go in ./src/lib which isn't made by default.

Let's jump to real-time state - the meat of what we're building.

Real-time state

State is saved in stores. A store is similar to react's useState value, but way more powerful. We're going to be creating a store that records websocket responses.

Let's make a file for our store in the shared directory: ./src/lib/state.ts.

Inside it, we're going to use Svelte's writeable stores:

// Import the function which initializes a new mutable store.
import { writable } from 'svelte/store';

type Item = {
  id: string;
  content: string;
};

// Our store will record an object containing an array of
// items produced by the websocket.
type State = {
  items: Array<Item>;
  error?: string;
};

// That's it;  state is now usable!  Components can subscribe
// to state changes, and we can mutate the store easily.
//
// Note that this is a singleton.
export const state = writable<State>({
  items: []
});

// We also want to connect to websockets.  Svelte does
// server-side rendering _really well_ out of the box, so
// we will export a function that can be called by our root
// component after mounting to connnect
export const connect = (socketURL: string) => {
  const ws = new WebSocket(`wss://${socketURL}`);
  if (!ws) {
    // Store an error in our state.  The function will be
    // called with the current state;  this only adds the
    // error.
    state.update((s: State) => return {...s, error: "Unable to connect" });
    return;
  }

  ws.addEventListener('open', () => {
    // TODO: Set up ping/pong, etc.
  });

  ws.addEventListener('message', (message: any) => {
    const data: Item = JSON.parse(message.data);
    // Mutate state by prepending the new data to the array.
    state.update((state) => ({ ...state, items: [data].concat(state.items) }));
  });

  ws.addEventListener('close', (_message: any) => {
    // TODO: Handle close
  });
}

We can now use this in our index page, ./src/routes/index.svelte:

<script context="module" lang="ts">
  export const prerender = true;
</script>

<script lang="ts">
  import { onMount } from 'svelte';
  // $lib auto-resolves to ./src/lib in Svelte.
  import { state, connect } from '$lib/state';

  onMount(async () => {
    connect();
  });
</script>

<!--
  We haven't defined the ItemList component (which should
  go in ./src/lib too), but this shows how you can auto-
  subscribe to the store using `$state`.  Every time state
  updates, $state.items changes and this will re-render
-->
<ItemList items={$state.items} />

This shows the power of Svelte, SvelteKit's routing, and state management. You can access state from anywhere in your app - no component hierarchy needed - and it's super easy to use within your components.

Svelte is incredibly powerful and developer efficient. Give it at try!