DEV Community: Svix

APIs are a Contract. Don't Break Them.

Svix — Tue, 06 Feb 2024 13:00:00 +0000

APIs are an unbreakable contract between your service and its customers. While humans can (reluctantly) adjust to drastic UI redesigns, API clients will just stop working with even the slightest change. This is why it's important for API providers and API consumers to agree on how to interact.

Because this contract can't change unilaterally, being precise and strict is extremely important. Undefined behavior is de-facto defined behavior, and if you are too lenient with what you accept, you will forever be stuck supporting all the unpredictable ways people use your APIs.

When talking about APIs I often hear people quoting the robustness principle: "be conservative in what you send, be liberal in what you accept". I strongly disagree with this principle, as it leads to fragile, insecure, and inefficient systems.

For example, consider an API endpoint that expects a query parameter called sort which according to the docs accepts the values asc for ascending, and desc for descending.

Here is an example psuedo-code implementation:

function list_items(sort: string) {
    if (sort === "asc") {
        return fetch_sorted_asc(...);
    } else {
        return fetch_sorted_desc(...);
    }
}

This function, as you can see accepts sort as a string, and then checks the value to sort the data accordingly. The problem is that the input is not validated to be one of two values, so in practice Desc, descending, and dscnding will all lead to the input correctly being sorted in a descending manner. Which in turn means that anyone that uses this API incorrectly like this will see it working correctly and therefore will start relying on this behavior.

This means we are now stuck supporting endless variations of desc until the end of time without even realizing. Which most likely also means we are going to break some implementations by accident when we change this code without even realizing it.

Additionally, it's easy to become more lenient later, but it's impossible to become more strict. So if you start more strict you keep your options open, if you start more lenient you don't.

Formulating the contract

As I said above, APIs are a contract, which is why it's important to formulate that contract as well as humanly (computerly?) possible. Therefore having an OpenAPI spec for your API goes a long way. Preferably also having validation that the actual implementation is compatible with the spec.

You should also be as specific as you can with both the definition, and therefore the validation of the fields. For example, the Svix user defined IDs (uids) follow a specific pattern: they have to be between 1 and 256 characters long, and match this regex ^[a-zA-Z0-9\-_.]+$. This is both documented in the spec and validated at runtime.

The advantage of having it documented in the spec, is that you can generate rich and specific documentation for your customers. Your customers don't need to guess what values are allowed for a uid, they can see it right there.

Validating the spec at runtime, when API calls are made, means that even if someone made a mistake and accidentally passed the wrong value, this will be caught immediately and they would be able to rectify the mistake.

Another advantage of having an OpenAPI spec, is that your API consumers can also generate validation on their end. Having schemas type checked on their end at compile time, so issues never hit production.

Tagging IDs with their type

Many APIs use uuids internally as their ID representation, which they then expose in their APIs. Those uuids usually look something like this: 12455207-ab83-5602-8d93-67999e204cff. First of all, I think the base62 representation (not a typo, it's 62) is much nicer: 2bd6zcCxa8v3fx1nj9XVlQp3WR3. It's more concise and more aesthetically pleasing, but to each their own.

The problem with using uuids, whether in their standard string representation or the base62 one, is that they are just generic IDs. This means that you can accidentally use an ID in the wrong place (which will usually return a 404) and be very confused on why it doesn't work.

A much better system is to tag IDs with their type, so the example above when tagged would look like this: app_2bd6zcCxa8v3fx1nj9XVlQp3WR3 for an "application" and msg_2bd6zcCxa8v3fx1nj9XVlQp3WR3 for a message.

Tagged IDs mean that you can have strict validation in your API and great examples in your docs that show exactly the kind of ID that's expected. Passing the wrong ID will no longer result in a 404, but rather with a validation error that explains exactly what's going on.

This makes debugging much much easier, but it also makes support much easier. Because it makes it very obvious when someone is accidentally passing the wrong IDs.

It's a wrap

This post was cross-posted from the the 5x9s newsletter. Subscribe for more content like this sent directly to your inbox.

For more content like this, make sure to follow us on Twitter, Github or RSS for the latest updates for the Svix webhook service, or join the discussion on our community Slack.

The State of Webhooks 2023

Svix — Wed, 18 Oct 2023 12:00:00 +0000

As software continues to eat the world, an increasing emphasis on real-time information, seamless integrations, and automation has propelled webhooks to the forefront of modern application architectures.
Webhooks, are now a core component in driving real-time event based workflows across platforms. You can check out the State of Webhooks here.

Special thanks to Ojus Save from Zoom, Judy Vander Sluis from Intuit, Sharon Yelenik from Cloudinary, Sarah Edwards from Github, and Kanad Gupta from ReadMe for collaborating with us on our webhook documentation reviews.
It really helped us understand how people think about documenting their webook APIs and informed a lot of our decisions when creating this report.

In this comprehensive report on the State of Webhooks in 2023, we looked at over 100 of the top API providers, examining how they've embraced and implemented webhooks, and analyzing the diverse ways these implementations have taken form.
Are most leading API providers on board with best practices?
Beyond mere adoption, how have they optimized, secured, and enriched their webhook offerings to cater to the needs of today’s developers and businesses?

Recognizing that ground truths often emerge in real-world use-cases, we've tapped into our own customer base, compiling an intriguing cache of statistics that shed light on the realities of webhook deliveries.
How often do they falter in the wild? How many retries do these messages generally need before they successfully land in their destined applications?
These firsthand statistics not only offer a clear picture of current delivery success rates but also demonstrate the value of implementing best practices.

Webhooks are more than mere HTTP callbacks; they embody a commitment to proactive, real-time communication in a world that's growing ever more interconnected.
With this report, we invite you to explore the current state of webhooks, and hope to spur continued adoption of webhook best practices.

You can get access to the full report here.

Strong static typing, a hill I'm willing to die on...

Svix — Wed, 04 Oct 2023 12:00:00 +0000

I've been writing software for over 20 years, and with every day that goes by I grow more certain that strong static typing is not just a good idea, but also almost always the right choice.

There are definitely uses for untyped languages (or language variants), for example they are much nicer when using a REPL, or for throwaway scripts in environments that are already hopelessly untyped (e.g. the shell). In almost every other case, however, strong typing is strongly preferred.

There are advantages to not using types, such as a faster development speed, but they pale in comparison to all of the advantages. To that I say:

Writing software without types lets you go at full speed. Full speed towards the cliff.

The question around strong static typing is simple: would you rather work a bit more and get invariants checked at compile-time (or type-checking time for non-compiled languages), or work a bit less and have them be enforced at runtime, or even worse not enforced even at runtime (JavaScript, I'm looking at you... 1 + "2" == 12).

Getting errors at runtime is a terrible idea. First, it means that you won't always catch them during development. Second, when you do catch them, it will happen in a customer facing manner. Yes, tests help, but writing tests for every possible mistyped function parameter is impossible given the endless possibilities. Even if you could, having types is much easier than testing for wrong types.

Types lead to less bugs

Types also offer annotations to code that benefit both humans and machines. Having types is a way to more strictly define the contract between different pieces of code.

Consider the following four examples. They all do exactly the same thing just with varying level of contract definition.

// Params: Name (a string) and age (a number).
function birthdayGreeting1(...params) {
    return `${params[0]} is ${params[1]}!`;
}

// Params: Name (a string) and age (a number).
function birthdayGreeting2(name, age) {
    return `${name} is ${age}!`;
}

function birthdayGreeting3(name: string, age: number): string {
    return `${name} is ${age}!`;
}

The first one doesn't even define the number of parameters, so it's hard to know what it does without reading the docs. I believe most people will agree the first one is an abomination and wouldn't write code like that. Though it's a very similar idea to typing, it's about defining the contract between the caller and the callee.

As for the second and the third, because of the typing, the third will need less documentation. The code is simpler, but admittedly, the advantages are fairly limited. Well, until you actually change this function...

In both the second and the third functions, the author assumes the age is a number. So it is absolutely fine to change the code as below:

// Params: Name (a string) and age (a number).
function birthdayGreeting2(name, age) {
    return `${name} will turn ${age + 1} next year!`;
}

function birthdayGreeting3(name: string, age: number): string {
    return `${name} will turn ${age + 1} next year!`;
}

The problem is that some of the places that use this code accept user input which was collected from an HTML input (so always a string). Which will result in:

> birthdayGreeting2("John", "20")
"John will turn 201 next year!"

While the typed version will correctly fail to compile because this function excepts age to be a number, not a string.

Having the contract between a caller and callee is important for a codebase, so that callers can know when callees change. This is especially important for an open source library, where the callers and the callees are not written by the same group of people. Without this contract it's impossible to know how things change when they do.

Types lead to a better development experience

Typing can also be used by IDEs and other development tools to vastly improve the development experience. You get notified as you code if any of your expectations are wrong. This significantly reduces cognitive load. You no longer need to remember the types of all the variables and the function in the context. The compiler will be there with you and tell you when something is wrong.

This also leads to a very nice additional benefit: easier refactoring. You can trust the compiler to let you know whether a change you make (e.g. the change in our example above) will break assumptions made elsewhere in the code or not.

Types also make it much easier to onboard new engineers to a codebase or library:

They can follow the type definitions to understand where things are used.
It's much easier to tinker with things as changes will trigger a compile error.

Let's consider the following changes to our above code:

class Person {
  name: string;
  age: number;
}

function birthdayGreeting2(person) {
    return `${person.name} will turn ${person.age + 1} next year!`;
}

function birthdayGreeting3(person: Person): string {
    return `${person.name} will turn ${person.age + 1} next year!`;
}

function main() {
  const person: Person = { name: "Hello", age: 12 };

  birthdayGreeting2(person);

  birthdayGreeting3(person);
}

It's very easy to see (or use your IDE to find) all the places where Person is used. You can see it's initiated in main and you can see it's used by birthdayGreeting3. However, in order to know it's used in birthdayGreeting2, you'd need to read the entire codebase.

The flip side of this is also that when looking at birthdayGreeting2, it's hard to know that it expects a Person as a parameter. Some of these things can be solved by exhaustive documentation, but: (1) why bother if you can achieve more with types? (2) documentation goes stale, here the code is the documentation.

It's very similar to how you wouldn't write code like:

// a is a person
function birthdayGreeting2(a) {
    b = a.name;
    c = a.age;
    return `${b} will turn ${c + 1} next year!`;
}

You would want to use useful variable names. Typing is the same, it's just variable names on steriods.

We encode everything in the type system

At Svix we love types. In fact, we try to encode as much information as we possibly can in the type system, so that all of the errors that can be caught at compile time will be caught at compile time; and also to squeeze that extra mileage of developer experience improvements.

For example, Redis is a string based protocol with no inherent typing. We use Redis for caching (among other things). The problem is that all of our nice typing benefits will be lost at the Redis layer, and bugs can happen.

Consider the following piece of code:

pub struct Person {
    pub id: String,
    pub name: String,
    pub age: u16,
}

pub struct Pet {
    pub id: String,
    pub owner: String,
}


let id = "p123";
let person = Person::new("John", 20);
cache.set(format!("person-{id}"), person);
// ...
let pet: Pet = cache.get(format!("preson-{id}"));

There are a couple of bugs in the snippet:

There's a typo in the second key name.
We are trying to load a person into a pet type.

To avoid such issues we do two things at Svix. The first is that we require the key to be of a certain type (not a generic string), and to create this type you need to call a specific fuction. The second thing we do, is force pairing a key to a value.

So the above example would look something like:

pub struct PersonCacheKey(String);

impl PersonCacheKey {
    fn new(id: &str) -> Self { ... }
}

pub struct Person {
    pub id: String,
    pub name: String,
    pub age: u16,
}

pub struct PetCacheKey;

pub struct Pet {
    pub id: String,
    pub owner: String,
}


let id = "p123";
let person = Person::new(id, "John", 20);
cache.set(PersonCacheKey::new(id), person);
// ...
// Compilation will fail on the next line
let pet: Pet = cache.get(PersonCacheKey::new(id));

This is already much better, and makes it impossible to get either of the previously mentioned bugs. Though we can do even better!

Consider the following function:

pub fn do_something(id: String) {
    let person: Person = cache.get(PersonCacheKey::new(id));
    // ...
}

There are a couple of problems with it. The first is that it's not very clear which id it should be for. Is it a person? A pet? It's very easy to accidentally call it with the wrong one, like in the following example:

let pet = ...;
do_something(pet.id); // <-- should be pet.owner!

The second is that we are losing the discoverability. It's a bit hard to know that a Pet has a relationship to a Person.

So at Svix, we have a special type for each id to ensure that there are no mistakes. The adjusted code looks something like:

pub struct PersonId(String);
pub struct PetId(String);

pub struct Person {
    pub id: PersonId,
    pub name: String,
    pub age: u16,
}

pub struct Pet {
    pub id: PetId,
    pub owner: PersonId,
}

This is indeed much better than our previous example.

There is still one issue. If we accept the ids from the API, how do we know that they are valid? All of the pet ids in Svix, for example, are prefixed with pet_ and are then followed by a Ksuid like so: pet_25SVqQSCVpGZh5SmuV0A7X0E3rw.

We want to be able to tell our customers that they pass the wrong id in the API, e.g. they pass a person id when a pet one is expected. One simple solution for this is to validate this (duh...) but it can be easy to forget to validate it everywhere that it's used.

So we enforce that a PetId can never be created without first being validated. This way we know that all of the code paths that create a PetId first make sure it's valid. This means that when we return a 404 Not Found to a customer because a pet isn't found in the database, we can be sure it was actually a valid id that wasn't found in the database. If it wasn't a valid id, we would have already returned a 422 or 400 when it was passed to the API handlers.

So why doesn't everyone like types?

The main topic of argument against types are:

Development speed
Learning curve and types complexity
The amount of effort and boilerplate required

First of all, I'd argue that even if all of the above were true, the advantages mentioned above are well worth the trouble. Though I also don't agree with all of the above.

The first one is development speed. Prototyping without types is definitely much faster. You can comment out pieces of the code and won't have a compiler complain to you. You can set the wrong values for some of the fields until you're ready to figure out the right ones, etc.

Though like I said above: "Writing software without types lets you go at full speed. Full speed towards the cliff." The problem is that this is just aggressive and unnecessary technical debt. You'll pay it mulitple times over when you need to debug why your code doesn't work (either locally, in the test suite, or in production).

As for the learning curve: Yes, learning more things takes time. Though I'd say that most people don't need to be typing experts. They can just get by with very simple type expressions, and ask if they ever hit a wall. However, if you keep things simple, you'll probably rarely if ever hit one.

Additionally, people are already required to learn how to code, learn frameworks (React, Axum, etc.), and so many other things. I don't think the learning burden is as significant as it's made out to be.

Last, but not least, regarding the learning curve: I strongly believe that the benefits of a lessened learning curve by not having to know types, are much less than the benefits of using the type script to onboard on a specific codebase. Especially since learning types is a one time cost.

The last point is about the amount of effort and boilerplate required to use types in your codebase. I strongly believe that the amount of effort is actually less than the amount of effort required by not writing types.

Not using types requires significant documentation and testing in order to reach even a basic level of sanity. Documentation can go stale, and so can testing; and either way they require more effort than just adding the right types. Reading code with types is also easier, because you get the types inline instead of in the function documentation where it's in an inconsistent format with a lot of added noise.

Yes, typing can be a pain in languages that don't support inference, e.g Java can be tedious:

Edit: It seems that Java has type inference nowadays. Thanks pron for the correction on HN.

Person person1 = newPerson();
Person person2 = newPerson();
Person child = makeChild(person1, person2);

while other languages that have inference (like Rust), are much nicer:

let person1 = new_person();
let person2 = new_person();
let child = make_child(person1, person2);

So having the right tools definitely helps.

Speaking of tools, in order to reap the benefits of your typing, you probably need to use a code editor (or IDE) that supports modern code completion that is language aware.

Closing words

I can see both side of the arguments on many topics, such as vim vs. emacs, tabs vs. spaces, and even much more controversial ones. Though in this case, the costs are so low compared to the benefits that I just don't understand why anyone would ever choose not to use types.

I'd love to know what I'm missing, but until then: Strong typing is a hill I'm willing to die on.

For more content like this, make sure to follow us on Twitter, Github or RSS for the latest updates for the Svix webhook service, or join the discussion on our community Slack.

Fixing Webhooks With New Funding From Andreessen Horowitz

Svix — Wed, 22 Feb 2023 12:00:00 +0000

We are very excited to announce that we have raised a new round of funding led by Andreessen Horowitz (a16z) with participation from Y Combinator Continuity and existing investors.

This fundraise comes at the back of our impressive growth, as we are now sending billions of webhooks a year for some of the most sophisticated engineering teams at companies both large (Fortune 500) and small (startups). Companies like Brex, Lob, Benchling, Bizzabo, and LTSE Equity, all use Svix to power their webhooks.

We have gotten to know the a16z and YC Continuity teams well over the last few months, and we are excited to be working with them for years to come, as well as continuing working with our amazing existing investors: Aleph, Y Combinator, Andrew Miklas, Yuri Sagalov (Wayfinder), Kevin Mahaffey (SNR), Christopher Golda (Rogue Capital), Jason Warner, Ian Storm Taylor, Kurt Mackey, Hana Mohan, Holly Dunlap, and Michael Orland.

Today is an important milestone towards our mission of making server-to-server communication easy, secure, and reliable. So I want to take this opportunity to thank the team for getting us to where we are today. I love waking up every day and being surrounded by such smart and passionate individuals, serving amazing customers, and working with the best investors out there!

What is Svix?

To talk about Svix, we first need to talk about webhooks. Webhooks are how services communicate; you can think of them as sort of a reverse API, or as the API equivalent of push notifications. They power most of the products that we all use on a daily basis. For example: when you make a payment online, a webhook is sent from the payment provider to the merchant; when you push a commit to Github, a webhook is sent to the CI providers to trigger a build; and when you use a workflow automation tool like Zapier, the workflows are almost always triggered by webhooks.

Webhooks have also become an increasingly common customer demand, both by API providers and more importantly API consumers. We see this trend across all industries, where customers expect webhooks in order to be able to take action on events without having to poll for updates. It makes sense, as webhooks enable building workflows on top of products, turning them from useful into critical infrastructure, increasing both stickiness and usefulness.

The problem, however, is that building a reliable webhook service requires a lot more engineering time, resources, and ongoing maintenance than you first expect. Which is especially painful, as it diverts valuable resources from improving the product, to reinventing infrastructure. We believe webhooks will follow a similar development pattern to other internet infrastructure—like email, web servers, and CMS—which only a few teams build on their own.

That's where Svix comes in. With Svix, companies can build a secure, reliable, and scalable webhooks solution in minutes. Saving them a lot of time, effort, and money, while focusing on their business and increasing team happiness.

Looking back

Milestones are always a great time to reflect on the past. Looking back, I'm really proud of the product we have built, and how we've been supporting our customers. It never gets old to hear "wow, that was really fast" when we release a new feature shortly after discussing it with a customer. We deserve some credit for that (and we work hard to earn it), but a lot of the credit also belongs to our customers. We are very lucky to have such engaged customers, and doubly lucky to have them be developers. We often get very detailed feedback, specced out feature requests, and of course pull requests for both the code and the docs.

I think this has also been the secret to why our customers love the product so much. It's because they helped us build it every step of the way, and they still do to this day. The same way it never gets old to hear about how fast we move with product development, it also never gets old to hear about how people use the product and how much they love it.

{/* prettier-ignore */} This is by far one of the best products I've ever worked with. It does one thing but it does it so well that I can't imagine ever doing my own webhooks solution. — Senior Engineering Manager at Bizzabo

One super important but often under appreciated aspect of what we do, is the relentless focus on stability and scalability. We spend a lot of time and work really hard to ensure that we provide our customers with a robust service at the scale they need. Whether it's the quality of the code, the test suite, the infrastructure, or our SDLC, we always hold ourselves to the highest standards. It's one of those things that no one notices when you do your job well, but are essential for any infrastructure service.

At the end of the day, however, it's not about what we have built, but rather how we are affecting the lives of our customers. Thanks to Svix our customers can offer a state of the art webhooks solution to their customers at a fraction of the time and cost it would take to build it themselves, with no ongoing maintenance, while getting constant improvements and automatic scaling. All of this while freeing their engineering teams to work on their core business. For more on that, you can see what Lob had to say about their experience switching to Svix.

Looking forward

We have identified an important (and growing) problem, built a great product to solve it, and started a whole new vertical along the way (webhooks as a service). This is why our customers, our investors, and the Svix team are so excited about webhooks and what we are building.

We have an opportunity to become an industry defining company, which is why we have decided to raise this round of funding, and partner with some of the best investors out there. We are going to use the money and the support of our new investors, to further expand the team and improve the product to keep up with the demand and our ever evolving customer needs. We have a lot of exciting stuff lined up, and we can't wait to share them!

If you are passionate about developer tools, webhooks, and server-to-server communication and you think you may be a good fit, please check out our careers page. If your company is looking to start sending webhooks, or upgrade their existing legacy solution, try us out or check out our docs.

💬 Join our Slack https://www.svix.com/slack

📄 Check out our docs https://docs.svix.com/

🪝 Start sending webhooks in minutes https://www.svix.com

The What, Why, and How of Payload Transformations

Svix — Wed, 08 Feb 2023 00:00:00 +0000

If you don't already know, we've had a new feature available in beta for some time now called
Transformations. I was one of the primary contributors to this feature, and am hoping to explain
to you what they are, why they're useful, and how to use them.

Later, in another post, I'll be doing a deeper dive into how transformations work under the hood for
those of you interested in the implementation.

A scenario to start

Say you're a Svix customer who offers an e-commerce software suite. Your customers are small, online
businesses who sell and send out physical goods. You use Svix to notify these customers of important
events: a new order has been made, a support ticket has been sent, a charge-back has occurred, etc.

All of these events need to be routed to various places in specific formats. This can be done by
defining event type filters per endpoint. Then you, the company offering this e-commerce system, can
dispatch an order.created event, a ticket.created event, or an order.chargeback event.

However, what do you do when your customers need the same event sent in different formats? For
example, your customer may need the order.created event sent to several places.

They may have an application tracking inventory and another application keeping statistics on the
types of products sold.

But, they might also need its details sent via email, in a human readable format, to their
warehouse, such that the product is shipped promptly.

The naive solution is to offer more event types. Even then, the default webhook format may not match
your customers' exact needs. There could be too many combinations of formats and events to manage as
a Svix customer. And then, you may not want to create event types just for one of your customers who
wants to do something a bit different.

With transformations, it's so much easier for your customers to make these customizations.

Introducing transformations

In brief, transformations allow your consumers to apply changes to messages before they're
dispatched to any given endpoint. These changes -- or transformations -- are defined via JavaScript
code that runs per message dispatched for any endpoint where the feature is enabled.

In the scenario presented above, a customer can very easily alter the content of each
order.created message per destination using a short JS function. Say that the email dispatch of
order details can be made with an HTTP request: then your customer, with no work on your end, can
just define a transformation to reorganize the payload into that of the email service's required
schema.

Let's get this out of the way -- the cost

Good news: Transformations are free and unlimited.

Bad news: Transformations are free and unlimited for now.

Since the feature is in beta, we don't want to charge any additional costs for something that may
not work entirely as expected. But because this is quite a computationally expensive feature to
offer, transformations will incur some additional cost once we're confident the feature is
production-ready. The pricing has not been finalized, but we intend for it to be very reasonable.

Limitations

We have added a few restrictions in order to ensure that all customer data is secure and isolated,
as well as some limitations that ensure that everyone can use the feature.

For one, precautions exist so you cannot access any:

Files
Environment variables
Networking
Generally any other I/O

This shouldn't come up in non-malicious use of the feature very often, but it's definitely a needed
restriction.

Additionally, there are hard limits on RAM use and processing time, so no loops that run a million
times. We wanted to make sure nobody can starve other customers' access to this feature.

Finally, you may not use async JavaScript in these scripts. No changes you make should be IO
bound, though, so we hope that's not too much of a problem.

In the future these restrictions may be eased, but only after I can ensure nothing is easily abused.

Beside these three points, you can manipulate the webhook dispatched in practically any way you'd
ever want.

How to use them

You can see the docs on this feature here for a more
concise explanation, but I have some examples that may help you understand the feature.

First of all, transformations have to be explicitly enabled per environment. You can get there via
the Settings page on the dashboard. Under the Settings section, enter the General Settings
page. From there, you can simply flip the Enable Transformations switch to turn it on or off.

From the app portal, assuming your organization allows transformations as mentioned above, your
customers can enter the Endpoints section, click on any endpoint, then enter the Advanced
tab. In this tab, you can enable and edit a transformation for that one endpoint.

This brings up an entire JavaScript editor with validation and test events so consumers can ensure
their transformations work just how they want the transformations to work.

From this editor you are given the following template:

/**
 * @param webhook the webhook object
 * @param webhook.method destination method. Allowed values: "POST", "PUT"
 * @param webhook.url current destination address
 * @param webhook.eventType current webhook Event Type
 * @param webhook.payload JSON payload
 */
function handler(webhook) {
  // modify the webhook object...

  // and return it
  return webhook
}

Every time a webhook is sent through an endpoint which has a script defined, the handler function
in the script is called. This function is expected to take one argument -- called webhook in the
example above. As it states in the comment, this webhook variable has the following members:

method -- The HTTP method the webhook should be sent with. It can be either POST or PUT, but this defaults to POST.
url -- The URL to send the webhook too. This can be any arbitrary URL that meets the requirements of the environment (so no plain HTTP requests if HTTPS Only Endpoints is enabled in the general settings page).
eventType -- If you're familiar at all with Svix, you know that each message is associated with an "event type" -- an arbitrary identifier that is used to mark the type of webhook. You can change it here to further modify the final payload.
payload -- Finally, the actual message contents can be programmatically manipulated. The payload, however, be a valid JSON value both before and after transformations are applied.

The webhook value given to the handler is mutable, and it should be returned after the requisite
alterations have been made.

Examples

Example 1

Say you're a client of the hypothetical e-commerce service mentioned earlier. You want to send a
notification to some service every time an order has been paid for so you can know what product to
package and ship.

But there's a catch: this service expects that the product ID be encoded in the URL's path instead
of in the payload.

Well, luckily, with transformations, you don't need your own server to forward requests that need
this tiny adjustment.

function handler(webhook) {
    if (webhook.eventType === "invoice.paid") {
        const product = webhook.payload.product_id;
        delete webhook.payload.product_id;

        webhook.url = 'https://a-valid.url/product/' + product + '/purchased/';
    }

    return webhook
]

Example 2

Need more invasive manipulations to the payload? Consider this example: You're again the client of
the hypothetical e-commerce service. This time, you want a message sent via a company-wide chat app
to a channel that sends a message every time you get an order, but it also has to scrub PII from the
data.

The format this chat application requires may be vastly different from the schema of the webhook the
e-commerce service sends. Say the default webhook looks something like:

{
  "product_id": "prod_abc123",
  "quantity": 7,
  "customer": {
    "name": "Jane Doe",
    "address_1": "123 Fake St",
    "address_2": null,
    "city": "RealCityName",
    "state": "IAmRunningOutOfFakeNames",
    "zip_code": "12345-6789"
  }
}

But then the chat application requires this format:

{
  "channel": "some-id",
  "message": "some-message"
}

Transformations are to the rescue again! Combined with our custom headers feature allowing you to
send any API tokens needed with the request, you could use the following code:

function handler(webhook) {
  if (webhook.eventType === 'invoice.paid') {
    const quantity = webhook.payload.quantity
    const first_name = webhook.payload.customer.name.split(' ')[0]

    webhook.payload = {
      channel: 'we-got-an-order',
      message: first_name + ' just ordered ' + quantity + ' products!',
    }
  }

  return webhook
}

TL;DR

Transformations allow your customers to modify select aspects of a webhook, such as the payload and
URL, before it's dispatched by writing short JavaScript functions,which can be written in the app
portal or updated with the Svix API.

Transformations are an excellent feature for interfacing with different applications, enabling your
customers to send events directly to an external service's API in whatever format the service
requires.

This can vastly speed up your customers' workflow. No longer do they have to hack together a
functional web server just to receive a webhook and forward it after making minute alterations.
Instead, they can just write one function and forget about the rest.

Zero Downtime Secret Rotation for Webhooks

Svix — Thu, 05 Jan 2023 00:00:00 +0000

Before we talk about zero downtime secret rotations for webhooks, we need to explain what are webhook secrets and what they are used for.

Because of the way webhooks work, attackers can impersonate services by simply sending a fake webhook to an endpoint. Think about it: it's just an HTTP POST from an unknown source. In order to prevent it, Svix, and many popular webhook providers sign every webhook and its metadata with a unique secret key. This is called a webhook secret.

You can read more on how Svix signs webhooks in the verifying webhooks section of the docs, and we also previously wrote a post about securely verifying signature hashes in case you find this interesting.

Secret rotation

The secret, as the name implies, needs to be secret in order to work effectively. If the secret is ever compromised, or suspected to be compromised, it needs to be rotated (replaced by a new one) in order to ensure the security of the signature scheme.

Rotating the webhook secret is a simple operation, usually it can be done by just a click of a button. The problem is that the moment you rotate the key, webhooks will be signed with the new key, which means that your endpoints verifying the signature using the old key will fail to verify the webhooks. This is because the key used to sign and the key used to verify are now different.

This is easy to solve, you just need to update the verifying endpoint to use the new key and everything will work again. The problem is that webhooks will fail from when you rotate, until the secret is updated.

Zero downtime secret rotation

The solution to this problem is simple: sign the webhooks with both the old key, and the new key for a set period of time to give your webhook consumers enough time to update their endpoints to use the new key.

You may ask yourself: isn't this a security issue? The key is compromised, an evil attacker may have access to it. While that's true, and why you need to rotate the secret in the first place, signing the payload with the compromised secret is not actually a security issue. This is because it doesn't give an attacker any advantage. The compromise of a key can let an attacker create fake payloads and sign them to trick the verifiers, but having a legitimate payload signed doesn't cause an issue.

OK, so how do we do it? As we said above the solution is just to support signing the same message with multiple keys. This can be done in many different ways, but the easiest way (and what Svix does), is to pass multiple space-delimited signatures in the webhook-signature header like so:

v1,g0hM9SsE+OTPJTGt/tmIKtSyZlE3uFJELVlNIOLJ1OE= v1,bm9ldHUjKzFob2VudXRob2VodWUzMjRvdWVvdW9ldQo=

Then, your consumer can just try and match each signature and as long as one of them matches, consider it a success. Here's an example JavaScript snippet that does it:

const signatures = signatureHeader.split(' ')

for (const signature of signatures) {
  if (timingSafeEqual(signature, expectedSignature)) {
    return true // Verification succeeded
  }

  return false // Verification failed
}

As we saw, making sure your webhook secret can be safely rotated with zero downtime is fairly easy, you just need to make sure to design your signature scheme to support it in the first place. Alternatively, you can just use the Svix service which takes care of everything for you, or use any of the Svix libraries to implement secure signing and verification.

For more content like this, make sure to follow us on Twitter, Github or RSS for the latest updates for the Svix webhook service, or join the discussion on our community Slack.

Open Source Webhooks as a Service Written in Rust

Svix — Thu, 29 Dec 2022 18:05:56 +0000

Sending one webhook is trivial. It's just a POST request to an API endpoint. But sending webhooks at scale is so much harder. At Svix we make it easy for anyone to offer a world class webhook experience.

https://github.com/svix/svix-webhooks

Using ngrok's new built-in Svix support

Svix — Mon, 07 Nov 2022 00:00:00 +0000

ngrok is a staple tool for many developers that creates tunnels between networks. It is often used to expose a port on localhost to the public Internet, but with ngrok Cloud Edge it can be also be used to secure traffic from the Internet to production cloud environments.

In this tutorial, we will learn how to verify Svix webhook requests using ngrok both for local development and on ngrok Cloud Edge.

This tutorial assumes you are already familiar with the Svix webhooks service and ngrok. If this is your first time using Svix,
we recommend you first check out our quickstart documentaion. We also recommend checking out ngrok's documentation on Webhook Verification.

Verify Svix webhooks locally with ngrok CLI

Install ngrok

If you haven't already, install the ngrok CLI.

Create a tunnel

First, create an endpoint in the Consumer App Portal, and copy the endpoint's Signing Signature. Then, run the following ngrok command on your computer's terminal, replacing SIGNATURE with the Signing Signature you just copied:

ngrok http 3000 --verify-webhook=svix --verify-webhook-secret=SIGNATURE

When you run the command, ngrok should generate URL that looks like https://d7f4c8296c55.ngrok.io. Copy that URL, and set it as the URL for the Svix endpoint you previously created in the Consumer App Portal.

Assuming you are running a service on port 3000, all Svix webhooks to your endpoint will now be forwarded to that local service by ngrok. And because you configured the --verify-webhook and --verify-webhook-secret options, ngrok will only forward verified Svix webhooks.

Verify Svix Webhooks on ngrok Cloud Edge

Create a Svix endpoint

Create an endpoint in the consumer application portal. You'll need the newly-created endpoint's Signing Signature later on.

Sign up for ngrok

Create an account on ngrok.com.

Create an ngrok Edge

Login to the ngrok dashboard. Using the menu on the left, expand "Cloud Edge" and choose "Edges." Create an edge by clicking the "New edge" button and choose "HTTPS Edge".

Configure ngrok Edge

On the Edge configuration page, find and click the "Webhooks Verification" menu item, and click "Begin Setup." Choose Svix as your webhook provider:

And paste your endpoint's Signing Signature from Svix as the webhook signing key:

Why Your Product Needs Webhooks

Svix — Tue, 01 Nov 2022 00:00:00 +0000

Why Your Product Needs Webhooks

Every product gets to the point where they’re missing a key element to continue their growth trajectory. Integrations.

Integrations with other popular services helps you acquire customers as well as retain them as your product becomes embedded in their workflows. However, creating these integrations consumes valuable engineering resources. It’s especially painful when you need to keep diverting engineering time to the next integration (and the next, and the next, and…).

Enter webhooks.

Webhooks, also known as http callbacks, are a way for APIs to notify applications that a specific event has occurred without receiving a request. Webhooks allow users to specify an endpoint URL where they can receive real time event updates, eliminating the need to send constant requests to their API providers looking for updates.

For example, Zapier, a popular low/no code integration platform, hates API polling:

Polling is the process of repeatedly hitting the same endpoint looking for new data. We don't like doing this (it's wasteful), vendors don't like us doing it (again, it's wasteful) and users dislike it (they have to wait a maximum interval to trigger on new data). However, it is the one method that is ubiquitous, so we support it.

Many products have APIs with a well documented webhook service that make receiving real time updates a breeze for their users.

Why webhooks

Here are 4 reasons why your procuct should offer a webhooks feature and stop relying on API polling:

Easy Integration

Webhooks enable both custom integrations as well as listing on no-code integration platforms.

Zapier, IFTTT, and Make (formerly Integromat) are very common ways for products to tap into a network of app integrations. All can work using API polling and webhooks (and we already know how we feel about polling).

Integrating with these workflow automation platforms has many benefits:

Integrate with 3000+ apps currently available in the marketplace
Continued integration with any new apps added
Increased visibility to 3.5M+ users

In addition to no code platforms, custom integrations become more straightforward and repeatable. For example, Hubspot has a public API with full documentation on how to set up a webhook listener to receive event notifications from them so that anyone can integrate with their platform.

Resource Efficiency

Generally, a webhook system will be much less resource intensive than a polling system. This is true for both the API provider and consumer.

For example, if 1,000 users poll the API every 5 seconds, your API will have to be able to handle up to 1,000 requests per second depending on the timing of the requests.

Zapier estimates that only 1.5% of polling requests find an update. That implies that with a webhook solution, you would only need to send up to 15 responses per second while your users don’t have to send any requests at all. That’s < 1% of the resources needed to sustain the polling solution (1000 requests + 1000 responses)

Real Time Updates

Our example scenario only accounts for 1 request every 5 seconds. What if customers want updates in real time? They would need to send requests at least every second if not more. This increases the load on your system and increases operational complexity.

If the API provider has rate limits, you could end up blocked from sending requests due to exceeding the specified rate limit. If the customer has implemented polling at frequencies that do not exceed the rate limit, they end up polling so infrequently that their updates are stale. This can decrease customer satisfaction and lead to churn as your API could no longer be a valid solution.

Webhooks get around this issue by sending updates right as events happen.

Customer Experience

Webhooks also offer a better customer experience. Forcing your users to integrate by constantly polling your API for updates means they need to do the heavy lifting of maintaining and comparing state to determine if there have been any changes and what actually changed.

Think about the common situation of a family on a road trip. Polling is equivalent to the kids constantly asking “Are we there yet?” and the parents having to say “no” over and over again. Wouldn’t you prefer the webhook scenario where the kids are quietly reading a book or watching a movie while the parents get to drive in peace?

Developers Expect Webhooks

Major API providers like Stripe and Plaid have very well documented webhook features and they are becoming commonplace with the most popular APIs. That means developers are used to working with the webhook pattern as they are fairly simple for customers to ingest. The more others start offering webhooks, the more developers will expect webhooks to integrate with.

Webhooks also improve the developer experience. In fact, a poll conducted by Wufoo (now part of SurveyMonkey) found that 82% of developers preferred working with webhooks over API polling.

So Why Aren’t More People Offering Webhooks?

There are some cases where polling would be a preferable solution to webhooks. When updates are more frequent than the polling interval, it’s more efficient to get the updates in batches via polling. If you also don’t need the updates in real time, polling is a great solution.

The main thing blocking most people from offering webhooks is that they’re really hard to implement at scale. Unreliable user endpoints means you need automatic retries to ensure deliverability. Monitoring deliverability and notifying customers when their endpoints are broken is critical. There are many potential security vulnerabilites like server-side request forgery (SSRF) and replay attacks that you need to defend against. You need a way for your users to authenticate webhook events to ensure they are coming from you. You need a UI for your users to test, debug, and monitor their implementations.

You can see all the potential components and services needed to build a secure, reliable and scalable webhook system in this architecture diagram we created for Svix:

Even a well funded Silicon Valley startup with a strong engineering culture like Lob recently revamped their system using our webhook service. They were able to successfully simplify the design to make their engineers’ lives easier, increase scalability, add features to their dashboard, and make their cost structure more predictable.

Summary

Webhooks are becoming a must have feature for products that want to integrate with other systems. They’re very efficient for providing real time updates, provide an excellent developer experience and enable a lot of possible integrations through no/low code platforms like Zapier and Make.

Although they can be difficult to implement at scale, there are open source and hosted solutions that make it simple to offer a great webhook experience for your users.

Securely Verifying Signature Hashes

Svix — Mon, 24 Oct 2022 00:00:00 +0000

A friend of mine shared this link with me the other day, about CVE-2022-43412. The issue disclosed in that CVE is that Jenkins Generic Webhook Trigger Plugin was using a non-constant time comparison function when checking whether the provided and expected webhook token are equal. In addition to the aforementioned CVE, there's also CVE-2022-43411 which is the same issue, but with the Jenkins Gitlab plugin.

In this blog post we will explain how the attack works, how to avoid similar problems, and what we do at Svix to protect our customers.

What is the issue?

The issue described in this CVE uses a timing-based side-channel to execute an oracle attack against the hash verification. Or in more plain words: this attack uses the fact that the comparison may take different times based on the content, to construct a valid signature even without knowing the key.

I know this may sound a bit like magic, because it kinda is. Though let's demystify it by using a more simple example, and go from there.

What's a timing based oracle attack?

Let's assume it's not a webhook, but rather a website, and the website is password protect. The developer who built this website is new to security, so they implemented the security as follows:

const password = 'Secret123'
if (user_provided_password != password) {
  return 'Wrong password!'
}

Now let's take a deeper look at the comparison in the condition. Because we used a high level function we can just use != between strings, but what goes on behind the scenes is that it actually compares them character by character (well, it's actually more complex than that, but let's assume that for simplicity).

So a naive implementation may look like this:

// This function assumes the strings are the same length for simplicity
function is_equal(a, b) {
    for (let i = 0; i < a.length; i++) {
        if (a[i] != b[i]) {
            // Can stop iteration, strings are not equal!
            return false;
        }
    }

    return true;
}

For the sake of example, let's assume our computer is very slow, and doing a[i] != b[i] takes a second each time.

Looking at the above then, we can see that if the user passes Bad123123 as the password, it'll try to compare the first characters (B vs S), and because they are unequal, the comparison will stop early so the whole function will take 1 second. Though if we pass Secret123 the whole function will take 9 seconds, as it compared all of the characters.

This means that we found an observable change of behavior based on the password. Now let's use that to our advantage.

We know that this function is slower the more characters are correct. So we can just start feeding it with characters one after the other and measure the time it took. We try A and get 1 second, B and get one second, all the way to S, where it all of a sudden takes two seconds. This lets us find the first character.

Now we can do the same for the second character. We pass Sa which takes 2 seconds, and then Sb, which takes two seconds, all the way to Se which then takes 3 seconds, so our password starts with Se!

We can repeat this process until we found the full password.

What's the correct way to solve it then? Well to make comparison constant time!

// This function assumes the strings are the same length for simplicity
function is_equal_constant(a, b) {
    let ret = true;

    for (let i = 0; i < a.length; i++) {
        // Putting the && ret at the end, to avoid short-circuiting
        ret = (a[i] != b[i]) && ret;
    }

    return true;
}

With this function the comparison always takes the same time, which protects us from timing attacks!

Note: do not use the function above in production as it may not work as expected. Optimizing compilers may change the behavior, so it's important to use purpose-built comparison functions that are provided by your system libraries which already protect against that.

How does it apply to webhooks?

Webhooks don't really have a password, but they have signatures, which in a way are dynamic passwords. With webhooks you pass a specific payload, e.g:

{
  "foo": "bar"
}

and sign it with a secret key in a way that only someone with the key could have signed it. The signature (e.g. Vh082qUqkhY7WiBWktRKRbP6/c+EYoqtHi/dMULaTKc=) is then passed with the payload, and the receiver can verify the sender has the secret key, and thus that the webhook can be trusted.

An attacker may want to send a malicious webhook, but without the secret key, the signature will not match, and the webhook will be rejected.

The aforementioned timing attack can't help us with recovering the secret key used for the signature, but it can help us find a valid signature!

So similarly to how we managed to find the password, we can find a valid signature for the malicious payload, and then just use this valid signature as a proof we know the key and get the webhook validated. Even though we never actually learned the key!

How does Svix solve it?

Security is hard. The reason for that is that security issues are not always obvious. If you have a bug in your code, you will usually notice it while testing it. Though with security, you don't always know what to test for.

This issue, unfortunately can't be mitigated on the sender side (Svix), it has to be mitigated on the client side. So we can't blanket solve it on our end.

We have however created a set of open source webhook signature libraries, that make it easy both to sign and verify webhook signatures correctly. Svix customers can offer them to their customers to ensure they have a great developer experience and stay secure. Additionally, non Svix customers can also reuse them, both for signing, and verification to ensure that they offer their customers an easy and secure experience.

For more content like this, make sure to follow us on Twitter, Github or RSS for the latest updates for the Svix webhook service, or join the discussion on our community Slack.

Comparing Popular Webhook Providers

Svix — Tue, 18 Oct 2022 00:00:00 +0000

After reviewing a few of the most popular webhook providers, we found some interesting trends about which features are most and least common.

Comparing the most popular webhook providers

We’ve published a series of articles reviewing various webhook providers and have noticed some interesting commonalities and differences between them. In each of the articles, we take a look through the webhook provider’s documentation to see if and how they implement 7 key features we believe help make a great webhook experience: automatic retries, exponential backoff, signature verification, manual retries, event types, multiple endpoint support, and log visibility. We tried to select the most popular webhooks by looking at google search volumes. Here is the summary table of our reviews:

Provider	Automatic Retries	Exponential Backoff	Signature Verification	Manual Retries	Event Types	Multiple Endpoints	Log Visibility
Shopify	✅	✅	✅	⬜	✅	✅	✅
Stripe	✅	✅	✅	✅	✅	✅	✅
PagerDuty	⬜	⬜	✅	⬜	✅	✅	⬜
BitBucket	⬜	⬜	⬜	⬜	✅	✅	✅
DataDog	✅	⬜	⬜	⬜	⬜	✅	⬜
CircleCI	⬜	⬜	✅	⬜	✅	✅	⬜
GitLab	✅	✅	✅	✅	✅	✅	✅
Svix	✅	✅	✅	✅	✅	✅	✅

Only Stripe, GitLab, and Svix check all 7 boxes (since Svix is a webhook service, all Svix customers check all the boxes as well 😉). Shopify came close with 6 boxes but the rest have at best 3. A round of applause for Stripe, GitLab, and Shopify please!

All 7 offer multiple endpoint support so it seems we’re past the days of having to receive all your webhooks on one endpoint. Event Types were offered by 6 of the companies and DataDog had a unique design for filtering which events you receive via custom variables. This combination of features can make webhook consumers’ lives much easier by allowing them to have separate endpoints for each Event Type. It makes debugging failing endpoints much easier and adds resilience to their system so one failing endpoint doesn’t block all webhook messages from being received.

Signature verification was offered by most companies which was a relief as its a critical component to offering a secure webhook service. We suspect this will become table stakes for any webhook provider very soon.

Manual retries were only offered by Stripe and GitLab and was the only feature not offered by Shopify. This suggests it's the most “nice to have” feature on our list. It does make developers’ lives a lot easier when they’re debugging a failing endpoint and can trigger a retry manually instead of having to wait for the retry schedule.

We were most surprised by the retry policies (combination of having automatic retries sent with an exponential backoff schedule). A few of the companies that we didn’t give checkmarks to actually do offer retries but the number of retries are very low. It felt like they were just trying to check a box instead of focusing on how a retry policy can improve the developer experience. We believe having a retry policy that sends retries over at least a 24h span is necessary to give developers time to fix their failing endpoints before they get disabled.

Overall, we were very impressed with some webhook services and a bit disappointed by others. This just goes to show how difficult webhooks can be to implement at the scale some of these products have acheived.

Introducing React Hooks in the Svix React library

Svix — Tue, 27 Sep 2022 00:00:00 +0000

The Svix React library now ships with custom React Hooks to make building a custom Portal easier than ever.

At Svix, we value developer experience. That's why we release our client library across a wide range of languages, including JavaScript/TypeScript. And today, we're bolstering our tooling in one of the most popular JavaScript frontend frameworks: React.

Many Svix users are interested in building custom, branded Consumer App Portals for their customers, usually by theming and embedding our hosted Consumer App Portal as an iFrame. This works great for most use-cases, but sometimes our users need more granular control over structure and appearance, and decide to build their own Portal as a part of their existing application. We're excited to offer a new, powerful tool to help ease development in this area: React Hooks, available in the latest version of our svix-react library.

The hooks are an abstraction over our standard JavaScript client library, and cover all the core functionality required to replicate the Consumer App Portal. They have been designed to manage concerns like pagination and reloading, so you can just focus on building an awesome UI.

Getting started is easy. Install or update the svix-react library (and svix, which is a peer dependency)...

npm i svix-react@latest svix

...and wrap your app in the SvixProvider component:

import { SvixProvider } from 'svix-react'

export default function App() {
  return (
    <SvixProvider token={token} appId={appId}>
      {/** your app's components **/}
    </SvixProvider>
  )
}

Great. Now, you can use our hooks for things like listing event types, rotating an endpoint key, and examining message attempts. Here's an example of using the hooks to list endpoints:

import { useEndpoints } from 'svix-react'

export default function ListEndpoints() {
  const endpoints = useEndpoints()

  return (
    <div>
      {endpoints.error && <div>An error has occurred</div>}
      {endpoints.loading && <div>Loading...</div>}
      <ul>
        {endpoints.data?.map((endpoint) => (
          <li>{endpoint.url}</li>
        ))}
      </ul>
      <button disabled={!endpoints.hasPrevPage} onClick={endpoints.prevPage}>
        Previous Page
      </button>
      <button disabled={!endpoints.hasNextPage} onClick={endpoints.nextPage}>
        Next Page
      </button>
    </div>
  )
}

As you can see, the useEndpoints hook handles fetching the data and maintaining various state, including pagination, errors, and loading.

Under the hood, our hooks are strongly typed using TypeScript. The useEndpoints hook returns a generic interface that's reused by other similar hooks, so you can expect the same properties and functionality across all hooks returning list data.

In the same way, hooks fetching a single item also share a common interface. Here's an example of the useEndpoint hook:

import { useEndpoint } from 'svix-react'

export default function Endpoint({ endpointId }) {
  const endpoint = useEndpoint(endpointId)

  return (
    <div>
      {endpoint.error && <div>An error has occurred</div>}
      {endpoint.loading && <div>Loading...</div>}
      {endpoint.data && (
        <div>
          <div>{endpoint.data.url}</div>
          <div>{endpoint.data.description}</div>
        </div>
      )}
    </div>
  )
}

All of the new hooks, along with their input options and response objects, are documented in the svix-react library's README. If your app requires functionality not provided by one of the hooks, there's an escape hatch:

const { svix } = useSvix()

The useSvix hook gives you access to the underlying Svix instance being used to make API calls, as if you were using the svix library directly. The SvixProvider component has already configured it with your token and appId, so it's ready for use immediately.

This update to our React library will make custom Consumer App Portal development easier and quicker than ever. We can't wait to see what you build with it!