DEV Community: Stigg

A practical guide for working with entitlements

Stigg — Sun, 18 Jun 2023 15:46:00 +0000

This is part 3 out of a 3-part series about entitlements, first published on the Stigg blog, here. If you're new to the concept of entitlements, check out part 1 first: "Entitlements untangled. The modern way to software monetization" yet, you can check it out here. And if you're already thinking of alternative ways to gate feature access in your application, check out part 2: "The shortcomings of gating feature access using feature flags, authorization, or plan identifiers". Hope you enjoy reading it!

Entitlements are a crucial aspect of any SaaS pricing & packaging. We’ve summarized a few tips we found crucial when working with entitlements.

1. Don’t nest entitlement checks

As you work with entitlements, one key principle to follow is to avoid nesting entitlement checks within your codebase. Nesting checks can lead to complex logic that is hard to follow, manage, and debug. Moreover, it can create interdependencies, where changes in one entitlement can unintentionally affect others.

Instead of nested checks, we recommend introducing a new entitlement, if possible, instead of depending on many entitlements. If that's not an option, at least you can introduce an abstraction that wraps those entitlement checks as a single function to better deal with this kind of coupling, and make it easier to change in the future. A singular check streamlines your code and makes it easier to understand and maintain.

Example of alternatives to nested entitlement checks:

Link to code

2. Never assume new entitlements are ‘there’

Due to the dynamic nature of entitlements, and the fact that they can be moved between plans, there is a chance an entitlement won't be present at all anymore.
Never assume that there are “mandatory” entitlements to ensure the application can continue to work properly. This will make your application less fragile and less subject to errors due to miss-configuration of entitlements.

Let’s say, you’ve added a new entitlement to a new version of your Free plan in your development environment. You can’t expect that this entitlement will be available until you’ll finish migrating all of your free customers to this new plan version. In the meantime, if the new entitlement check was already deployed - it must accommodate the fact that the entitlement is missing, or else the application will break.

3. Embrace the “fail open” principle

Entitlements are a critical component of any infrastructure. It's crucial to account for potential service disruptions. The "fail open" principle states that in the event of a failure, default access permissions should grant the user as much access as possible without breaking the application invariants. This ensures that users can continue to access necessary services, even when your entitlement service encounters an issue.

4. Create a centralized module in the code that evaluates the entitlements

In general, it’s good practice to keep the entitlement checks at bay by introducing a shared module or a class that encapsulates the checks for different features. This type of will make it much easier to track and change entitlement-related logic. It also helps define a more “business centric” terminology, when using entitlements in different areas of the application.

Think about this scenario. You have an entitlement ‘seats’. Inside your application, there is a members page for your customers to invite more team members. The invite button is enabled if the number of members is below the entitlement limit. Introducing a canInviteMembers(numOfNewMembers) function that checks for the seats entitlement against the current usage of the entitlement can help simplify the access check logic and re-use it throughout other places in the application.

Link to code

This is just a recommendation. You’ll need to adjust it to your project’s coding standards.

5. Less is more

This might sound obvious, but way too often, this gets out of hand… Just like feature flags, if you don’t keep your entitlements as clean and simple as possible, you’ll end up with messy code. If an entitlement does not exist anymore, first remove the entitlement checks in all the places from the codebase, and then the corresponding entitlement.

Building an entitlement service:

Once your product is mature enough you’ll probably need an entitlement service.

Here are a few points that you should take into consideration before attempting to build one yourself:

1. Prepare for network latency and downtime

Every network call adds latency to the overall system response time. Entitlements are a critical function in the service you provide. You don’t want your customers to face long response times when trying to access a feature inside your product. Always strive to optimize your entitlement service for low latency and scale, or at least make sure you have a good caching mechanism in-place.

‍2. Communicate usage and limits

Especially in self-service products, your customers need to be aware of their current usage and the limits of their subscription, before they are being denied access to functionalities or service. An effective entitlement service should be able to track and communicate this information clearly to users. That can be through dashboards, a self-service customer portal, or regular in-app / email notifications.

3. Determine your source of truth

You’ll need a source of truth. Deciding between your billing provider and your entitlement service is a significant consideration. This decision will influence how you synchronize data between the two systems. If your entitlement service includes complex logic like complex trials, scheduled downgrades, or grandfathering, this will be more suitable as your source of truth. Conversely, if your billing system is more sophisticated, then it should be the primary data source. The key is to identify which system contains the most comprehensive and accurate representation of customer data and use it to drive your entitlement decisions.

To wrap up

Operating in highly dynamic markets, SaaS companies have much less chances of success when compromising on rigid pricing models managed in the codebase.

Entitlements are the base for a truly flexible taxonomy and speed in go to market, allowing developers to quickly build and update pricing and giving GTM teams the opportunity to experiment with pricing & packaging, without code.

We made it our mission at Stigg to solve entitlements for SaaS. Our API gives you a flexible and scalable infrastructure, including metering, built-in experiments, a bunch of snap-in widgets, and native billing integrations, out of the box, and provides the automation to orchestrate everything about your pricing & packaging. If you consider buying over building, check out our Free plan. You can use one of our sandboxes to see all functionalities of Stigg, even before integrating it.

How to gate end-user access to features: Shortcomings of plan identifiers, authorization & feature flags

Stigg — Sun, 18 Jun 2023 15:45:58 +0000

This is part 2 out of a 3-part series about entitlements, first published on the Stigg blog, here. If you're new to the concept of entitlements or haven't read part 1: "Entitlements untangled. The modern way to software monetization" yet, you can check it out here. Hope you enjoy reading it!

Today, you might already use a number of concepts, which may seem like a proper solution to solve monetization challenges. Let’s discuss a few alternatives to entitlements, and the limitations and friction points you should be aware of.

Entitlements vs Plan identifiers

A common approach, especially in early-stage SaaS startups, is to tie access control to plan identifiers, that is, checking the user's subscription plan to determine access to specific features.

For example, a structure like this can be found in the early days of a company:

Link to code

While this might seem like a straightforward solution, it has significant downsides that can lead to complexity and rigidity over time.

Brittleness of plan identifiers
Plan identifiers can make your code brittle. The major drawback lies in the tight coupling it creates between your pricing model and the application codebase. Any change in the pricing plan, from changing the name of a plan, to changing the price or modifying the packaging, implies changes to the product behavior, which in turn necessitates updates to the code to accommodate it.

Code maintenance
Coupling the feature access logic with the pricing model will require changing the application code every time you want to adjust or change anything. That’s a lot of maintenance work you can save by decoupling plans and features. Your entitlement checks should only be concerned with whether a customer has access to a feature, not why they have access. This separation of concerns is vital for maintaining clean, maintainable, and flexible code.

Changes and migrations
Another aspect to consider is customer migration between plans. Say, for example, your business introduces a new plan or updates an existing one, and you wish to move customers to this plan. With plan identifiers, you would need to sift through the entire codebase to update the plan reference, even if you only want to make a change in the entitlement configuration. A task prone to errors and bugs.

Custom settings for enterprise plans
Enterprises often require custom settings that differ from standard plans, thus necessitating a unique plan. With plan identifiers, you'd need to create a new plan for each custom case, leading to a multitude of plans and identifiers that are difficult to manage and reason about.

Custom code for usage and limits
When you use plan identifiers, tracking feature usage or metering requires implementing usage limits in your codebase. This adds another layer of complexity, leading to a potential maintenance burden over time.

Entitlements vs Authorization

Authorization is a method of restricting system access based on certain rules, such as role-based (RBAC), policy-based (PBAC), or attribute-based (ABAC) access controls, of individual users within an application.

While entitlements serve monetization, typically on account level, and authorization serve access control, typically on user level, it's possible to model entitlements using an authorization concept by mapping plans to roles, attributes or policies. Here is an example, the free and premium plans are mapped as roles, with each role having a set of entitlements modeled as permissions:

Link to code

While this approach may work for less complex pricing models for a while, here are a few downsides you should be aware of.

User access vs account access
Authorization follows the principle of "least privilege," providing only the minimum access rights needed for a user to fulfill their role. Authorization isn't tied to the business model of a product, but rather to the organizational structure within which a product or system is used. For example, in a typical SaaS product, a user with the Admin role may have access to invite add or remove team members, while a user with the Member role may not.

No support of usage limits
Usage limits are not natively supported in authorization. While some advanced authorization policy engines can support this use-case, it's not a first-class citizen in most authorization implementations.

An example of how to implement entitlements using the Oso policy engine (example from Oso's docs):

This approach will only work for monolithic applications, where the policy is evaluated against a single database. In distributed applications or microservices, where each service owns a subset of the policy’s data, accessing the usage can become even more challenging.

Entitlements and JWT
Some might consider storing customer entitlements on authentication tokens, like JWT, by writing custom claims or permissions as part of the token’s payload. Take into account that this method won’t be accurate enough for most monetization use cases. JWT tokens are stateless and are not typically updated until they're refreshed.

This can be problematic for several reasons:

Usage limits won’t be accurately updated. If a user purchased another seat - the entitlement limit should be increased immediately.
If you enrich the JWT payload with current usage, it won’t be accurately tracked. For example, a user exceeds their limit, but it won’t be reflected or blocked in time.
If a user upgrades or downgrades to a different plan, they won’t have instant access to the new entitlements.
If the company wants to grant a promotional entitlement to an account or change a user’s plan, those changes won’t be reflected in the user’s token immediately.
This can lead to inaccurate or outdated permissions being enforced.

Entitlements vs Feature Flags

Feature flags, also known as feature toggles, are a technique used in software development to turn certain features on or off at runtime, without needing to change or redeploy code. They are originally intended to be short lived and removed from the codebase, when not in use anymore.

While both feature flags and entitlements provide control over software functionality, entitlements offer more sophisticated controls such as usage limits and tiered access levels that are automatically updated when a plan changes.

An example of an entitlement check can be performed using a LaunchDarkly SDK:

Link to code

The evolution of Permissioning Toggles
Feature flags allow developers to turn features on or off, providing a way to test and release features gradually, perform A/B testing, and more. When considering entitlements in comparison, we could view them as an evolution of Permissioning Toggles. However, they specifically cater to marketable features sold under various subscription packages, extending beyond the binary on/off use-case of traditional feature flags.

When compared to other types of toggles, the nature of entitlements is relatively high dynamism (i.e. when the toggle value changes) as they are tied to the pricing model and customer subscription level, and a longevity (i.e. how long the toggle exists in the codebase) that can span from at least a few months to a few years.

You can read more about the different types of feature toggles in this excellent article by Pete Hodgson that was published on Martin Fowler’s website.

If you choose to use a feature flagging system to manage your entitlements, here are a few things you need to consider.

Deterministic changes in entitlements
Feature flags are not part of the customer's contract and don’t have any financial impact. These flags can be used to test new features or variations with a subset of users, hence, they can be altered on the fly. Entitlements, on the other hand, are charged for and are bound by contract terms with a customer. Unlike feature flags that can be altered on the fly, entitlements change whenever a customer’s subscription or contract changes. Changes to entitlements usually require advance notice, and the current entitlements are typically only revoked after a sufficient “grace period” has expired. They are deterministic and should not be assigned ad-hoc, except in explicitly intended circumstances like for enterprise customers.

‍Targeting rules
The core of a feature flagging system is targeting rules among users (or customers), segments (or subscription tiers), and features. This could mean building custom automation workflows to keep everything synchronized when a plan or subscription changes. Entitlements, on the other hand, are natively tied to the customer subscription, automatically updating any changes.

Ease of use
Feature flagging tools are typically more technical and primarily targeted at engineers and product managers. This means Customer Success and Sales teams often find them difficult to navigate and understand, reducing their overall effectiveness for those teams.

Again, custom code for usage and limits
Feature flags don't inherently account for feature usage or metering. As a result, to track feature limits, you might need to define those limits into your codebase and automate toggling of flags when those limits are reached.

Entitlements, Feature Flags, and Authorization: Stitching it all together

While it’s possible to duct-tape a solution to support your pricing model with one of the alternatives listed above, none of them will be sufficient enough on their own. Authorization, RBAC, and Entitlements each serve different purposes, are operated by different departments, and play a part in significantly different processes. To decide which tools you’ll need in your own stack, here’s an overview.

Feature Flags: Primarily managed by Engineering and Product teams. They form part of the software delivery and product management process. A good use case for feature flags is, if you have a user that you want to include into the release of a beta feature and you’re not yet ready to roll it out across your entire user base.

Entitlements: Managed by Product, Sales, Marketing, and/or Growth teams. They form part of the pricing and packaging strategies, and customer contract management processes. A good use case is, if you have a feature you’re ready to monetize, and you want to roll it out to all existing and/or new customers.

Permissions (Authz): Usually managed by the application administrators. They form part of the data security and compliance processes. A good use case is, if you want to enable customers to set different permissions for different users within a given account.

A typical sequence of access checks in the code would start with a feature flag check at the user level. This determines whether the feature is enabled for this particular user or organization.

If the feature is enabled, the system then checks the entitlements at the organization level to confirm that the customer's subscription includes access to this feature.

Finally, the system checks the user's role-based permissions to ensure that the user is authorized to perform the requested operation within the feature.

If we’ll imagine this sequence in code within a React application, it will look similar to this:

Link to code

Scatter/Gather
To stitch these three systems together and make sense of them, one practical approach is to create a centralized place for access checks, such as an API gateway. This gateway can take on the responsibility of authorizing operations using a "scatter-gather" pattern on the backend.
It "scatters" the authorization checks across the feature flag, entitlement, and permission systems, then "gathers" the results to make an overall decision. It can also pass this context on the request to downstream services so they won't have to fetch this data again in case they need it for their internal logic.

This approach encapsulates the complexity of the authorization checks and provides a unified interface to the rest of the application. It also allows for easier updates and changes as the underlying systems evolve.

An example of a “stitched” user data including feature flags, permissions and entitlements:

Link to code

In Part 3 of this technical deep dive

...we're sharing a curated list of best practices when working with entitlements. We've made it our company's mission to solve this complex issue for engineering teams and have learned a lot building Stigg. Whether you decide to look for a solution or build a service yourself, this list will help you make informed decisions. Check out our best practices here.

To wrap up

Operating in highly dynamic markets, SaaS companies have much less chances of success when compromising on rigid pricing models managed in the codebase.

Entitlements untangled: The modern way to software monetization

Stigg — Sun, 18 Jun 2023 15:45:54 +0000

This technical deep-dive was initially published on the Stigg blog.

What are entitlements? Why should engineering teams care? And how are they different from feature flags or authorization tools? Our CTO Anton takes you into a technical deep dive into entitlements for SaaS monetization.

How many times did you change something in your pricing?

If you’ve been around SaaS for a while, you know how dynamic this landscape is, forcing companies to continuously update their pricing and packaging to stay competitive.

Yet, every change entails a corresponding change in the codebase to accommodate the new packaging behavior, inherently becoming more complex and time-consuming than any engineering team appreciates.

Tight coupling between pricing models and application logic can create a significant overhead which usually means the pricing model won’t be changed often… and it might sound ridiculous, but many organizations ignore this problem and continue to suffer from an outdated and anchored pricing model because of bad software design choices in the early days of the company.

This article introduces entitlements, an elegant solution to a complex problem in software monetization, and what engineering teams need to know to utilize it effectively.

So, what are entitlements?

By focusing on features instead of pricing concepts, we can decouple the price model from the application logic. The concept of Entitlements encapsulates the feature access settings under various product variants (aka Pricing Plans or Packages), bridging the gap between how a product is sold and how it behaves for a variant.

In essence, entitlements are a set of permissions defining what a customer (paying or non-paying) can do with your software application. For example, an entitlement can grant the customer access to the “Audit logs” feature or limit their account to “5 seats” within a given plan.

In essence, entitlements are a set of permissions defining what a customer (paying or non-paying) can do with your software application.

Entitlements provide a clean separation of concerns, effectively streamlining the process of updating product features in response to changes in the pricing model.

The 3 types of entitlements and how to control access using entitlement checks

Entitlement checks are vital to accurately control and monitor the access and usage of features by customers. They ensure that customers are granted access to the features they’ve paid for and help prevent overuse or misuse of resources.

To have some common ground on how are the checks are evaluated, lets present an example of a basic entitlements data structure granted to a customer:

Link to code

These are the 3 most common types of entitlements in SaaS and how entitlement checks work for each of them:

Feature gates

Sometimes referred to as binary flags, these types of entitlements control whether a customer has access to a specific feature or not. For instance, a customer on a free plan might not have access to advanced analytics, but upgrading to a paid plan will enable this feature. These binary flags simply gate the feature—either the customer has access, or they don't.

Feature access is controlled by evaluating a condition at runtime based on the customer and feature identifiers. This check verifies whether the customer is entitled to access the particular feature or not.

Link to code

Configuration

This type of entitlement changes the behavior of a feature within the application, depending on the package or plan that a customer is on. For instance, a customer on a free plan of an observability tool might have the data retention policy set to 3 days, while a customer on a paid plan might have a longer retention of 14 days. The entitlement does not just control access, but also how the feature behaves within the application.

For configuration entitlements, the check doesn't just involve whether or not the feature is present but also retrieves a configuration value that drives application behavior.

Link to code

Soft/Hard limits

These entitlements cap the metered usage of certain features. In case of hard limits, access is fully cut off, once the limit is reached. For example, a customer has access to 5 seats. Once they reach 5 seats, they can’t invite more users. In case of soft limits, customers are alerted or throttled once they reach a certain usage level. For example, a customer may be limited to 1000 API calls per day under a free plan. Once the customer crosses the limit a "Maximum capacity reached" email is automatically sent, and a significantly lower API rate limit (or a grace period) is applied before the API starts rejecting new data.

In the case of metered entitlements that have soft or hard limits, these checks are performed against the expected usage (= current usage + new usage) in a per-request context. For hard limit checks, it’s crucial to prevent overages, so real-time data is essential for accuracy.

Link to code

For soft limit checks, where preventing an overage is less important, there’s some tolerance for eventual consistency. This means you can compare against cached or slightly stale usage, offering a balance between system performance and accurate usage tracking.

What entitlements are not

To fully understand what entitlements are, let’s cover some misconceptions about them as well.

Firstly, entitlements are not a customer setting. Entitlements are typically part of a contractual agreement between you and your customer, i.e. the plan the customer chose. Customer settings are individual configurations a customer can make within their account. For example, a customer pays for 1GB of log data per day. One of their applications in their development environment is spammy. The customer setting allows them to reduce the log data for this specific environment to 256mb, while the account is still entitled to 1GB in total.

Secondly, entitlements are not synonymous with plans, e.g. "Basic", "Pro", and "Enterprise". Entitlements are, at their core, marketable features. Different pricing plans may grant access to these features with various configurations, but an entitlement does not inherently represent a particular product level.

Why should you care about entitlements?

Managing pricing changes at the code level, without appropriate abstraction mechanisms, can quickly devolve into a tangled mess, resulting in more complexities than anticipated.

With entitlements, you can cleanly decouple the pricing model from the application logic, enabling swifter and smoother changes to your offerings. Any packaging change will take a fraction of the time you’d spend with hardcoded checks.

Besides taking much less of your time, you will make your organization that much more flexible in the way they go to market. You can introduce new features or tweak existing ones without having to overhaul the entire system. You’ll be able to quickly adapt to changes in the market, customer needs, or business strategies. This agility not only gives you a competitive edge but also makes for a better user experience as you can tailor your offerings to better suit your customers' needs.

On top of that, entitlements enable GTM teams to A/B test and experiment with pricing & packaging, without talking to developers at all. As outlined at the beginning of the article, SaaS pricing & packaging has to change regularly to keep up with the market dynamics. As an abstraction on top of your codebase, those changes will no longer require you to write code.

As an abstraction on top of your codebase, those changes will no longer require you to write code.

Adopting entitlements can save you tons of development hours, fewer bugs, and increase your chances of success in a rapidly changing market environment.

Fusing entitlements and metering

Metering becomes crucial, once your pricing model includes configuration or soft/hard limit entitlements. Say you limit your customers by the number of users they can add to their team, the amount of storage space they get access to, or the number of API calls they can make within a certain timeframe. Without measuring a customer’s usage of an entitlement, you won’t be able to enforce limits in the entitlement service, and the enforcement will have to leak into the application code. For example, a customer’s plan includes 5 seats. Without metering, you won’t know how many seats they actually use. Your customer could invite 30 other users, without being limited - or ever having a reason to upgrade.

Without measuring a customer’s usage of an entitlement, you won’t be able to enforce limits in the entitlement service, and the enforcement will have to leak into the application code.
‍
Metering software is designed to calculate feature usage, based on raw measurements that are often stored in an append-only system. This way, it is possible to maintain a historical record and allows querying for historical usage between different timeframes.

If you don't have a metering solution in place, you'll need to calculate the usage yourself. For simple counting, this is fairly straightforward – you just query your database to figure out how many instances of a feature or resource exist and compare it with the limit set by the entitlement.

However, when your usage is interval-based (such as a limit on API calls per month), it becomes more tricky. You need to design a system that can aggregate usage data for every passing interval, and that can correctly reset or roll-over counts when the interval ends.

For example, to support the entitlement, you’ll need to track and count the number of API calls:

Link to code

Hard stops

For entitlements with hard limits, once the limit is reached, access to the feature must be revoked to prevent any overuse. This is especially relevant for free plans. Hard stops also introduce perfect upsell opportunities, whether you support a self-service or sales-led model. For self-service products, present a paywall for your customer to allow them to upgrade and overcome the limit they’ve just experienced. For sales-led products, you can trigger a notification to your sales team to reach out to their customer.

Optimizing plans for success

Metering isn't just about enforcing entitlements. It's also a powerful analytical tool. By analyzing usage patterns, you can gain insights into how different features or resources are being used. This can inform decisions about how to optimize entitlement limits for future pricing packages and enables modern ways to go to market, such as hybrid pricing (i.e. combining fixed costs with limits and variable costs into a new pricing model).

Pro Tip: When talking about optimizing pricing plans, GTM teams will need a way to experiment with different versions of a plan. A good entitlement service should allow changing the upsell interface (e.g. paywalls or trial banners) dynamically to easily accommodate those experiments without additional coding.

Entitlements vs Billing

Billing software has a very specific role in managing financial transactions, invoicing, payment scheduling, and processing. Simply put, they handle everything when a customer swipes their credit card. In pricing & packaging, especially in self-service products, a lot is happening before and after that swipe, though. Many times, users first access your product through a free plan or trial period. Complex upgrade & downgrade flows require very smooth controls over who has access to what feature.

The key distinction between the two systems is that while billing software is focused on the commercial aspect, entitlements are all about provisioning and access control.

The key distinction between the two systems is that while billing software is focused on the commercial aspect, entitlements are all about provisioning and access control.

Why does your entitlement service have to be separated from your billing?

Bridging between application and billing
An entitlement service can be seen as a bridge between your application behavior and your billing process, and can become the source of truth for both pricing and packaging. Besides acting as an “anti-corruption layer” to shield potential integration errors with the billing software, it also provides a fail-safe layer in case the billing system is unreachable. If the need rises, this can allow you to switch between billing solutions in the future easier with much less coupling.

Dealing with legacy plans
Billing systems typically do not handle the concept of "grandfathering" (i.e. allowing legacy customers to stay on their plan, while updating the plan for new customers), or gradually transitioning customers from legacy plans to the new ones (e.g. when you add a new feature to an existing plan). There is also no native support for scheduling migrations of customer segments, so moving e.g. your legacy customers to the new plan at the end of their billing period will be pretty hard doing it, involving custom scripts and programmatically scheduling the transition. Entitlement services can automate this process, reducing the manual effort and ensuring a smoother transition for customers.

Centralized entitlement management
The main benefit of an entitlement service is that it allows you to manage each feature's business logic centrally. It's an independent system that decides who has access to what based on the user's subscription status. Billing software, on the other hand, isn't designed to carry out these checks or grant access to features. It's solely concerned with financial transactions.

Scalability and low latency
Entitlement checks are crucial for ensuring that applications operate reliably. These checks need to be made quickly and efficiently, which requires a system designed with low latency and scalability in mind. Billing systems typically don't offer this kind of performance because they delegate these concerns to the integrating system.

Flexible access management
An entitlement service allows for a much greater level of flexibility when it comes to managing user access. It can handle multiple sources of access rights, such as subscription plans, add-ons, and ad-hoc features (like promotions) granted to a customer. Billing systems are far more limited, as they’re focused on pricing and payment, not on deciding who gets access to what.

Decoupling pricing and access control
By separating entitlements from billing, you can decouple pricing from access control. With this separation, you can give users access to features independently of their billing status, offering more flexibility and control for use-cases like free & reverse trials, and customisations for enterprise customers.

Enforcing plan immutability
Billing systems usually don’t have safeguards to control changes of an existing plan. If one of your team members makes a change unintentionally, this will affect all your existing customers immediately. An entitlement service ensures that plan details are immutable, providing peace of mind for both operators and customers by preventing unexpected changes.

Development, Staging and Production environments
Billing systems often lack support for different environments or the development release lifecycle. For changes in your pricing model, whether you add or remove a feature, update the price or overhaul your entire model, you’ll probably want to build in development, test in staging, and only then move to production. Billing systems offer limited support for change management, delegating the tedious tasks of seeding data and syncing it between multiple development accounts to you. An entitlement service should support multiple environments and allow automatically promoting changes between different development, staging, and production in order to prevent potential drift.

In Part 2 of this technical deep-dive

...we're going to discuss different approaches to gating feature access for your end users. We're taking a closer look at authorization, feature flags, and plan identifiers, how to gate features with each of these approaches and the downsides you need to be aware of, before choosing any of these methods. Check out the top shortcomings we've identified here.

In Part 3 of this technical deep-dive

To wrap up

Operating in highly dynamic markets, SaaS companies have much less chances of success when compromising on rigid pricing models managed in the codebase.

Best practices I wish we knew, when introducing Stripe webhooks

Stigg — Sun, 11 Jun 2023 09:07:37 +0000

In this article we will describe what are the best practices to create the webhook integration with Stripe. We will be focusing only on one side of our two-way integration. The side that is listening to webhooks and acting on them.

Why are we so obsessed with Stripe incoming webhooks in Stigg?

One of the value propositions of our product is our no-code Stripe integration. When integrating with Stigg we take care of everything related to interacting with Stripe for our customers. We sync all the plans and add-ons, customers, and subscriptions and manage the state of those entities in Stigg and propagate changes to Stripe if needed. We are also listening to webhooks from Stripe that indicate that something changed to the synced entities on Stripe's side.

One of the most important flows in Stigg is the Create Subscription flow. When having a Stripe integration and a paid subscription is created we are waiting for an incoming webhook from Stripe to acknowledge that the payment was successfully made in order to make the subscription active. Failing to create subscriptions for any reason can lead to our customers losing money and therefore this has to be a well-tested, highly resilient, and easy to scale.

In this article, we will describe what are the best practices to create the webhook integration with Stripe. We will be focusing only on one side of our two-way integration. The side that is listening to webhooks and acting on them.

‍

What’s the problem with spinning up an API to listen to Stripe webhooks and act on them?

It is fairly easy to just listen to incoming webhooks with a service and act upon them and that's how we started when developing the proof of concept for this feature.

It could be something along those lines:

For us, this was great for the proof of concept but not good enough to go to prod.

There are a few issues with this design that we, in the engineering team, spotted and wanted to address in order to be confident about this new part of the integration:

What happens if we are down and webhooks are not being processed?
What happens if our processing fails silently?
How can we monitor failures properly?
How can we be sure that we can reprocess failed events on our own when we fix a potential bug on our side?
If Stripe bombards us with events, how do we scale our processing speed without losing data?

We had to build something way different to answer these questions. And with the complexity of the design, we found out that there are many caveats in the way Stripe is implementing their webhooks. Some of those limitations we found the hard way, so reading this can actually save you a lot of pain trying to build your webhook integration.

So here are a few limitations we had to take into account when designing the new solution for reliable Stripe webhooks processing and what it meant for us:

- “Stripe does not guarantee delivery of events in the order in which they are generated” ~Stripe docs. That means that we need to be very careful processing an event and we can’t rely on a customer.subscription.created event to arrive before the customer.subscription.updated event. It’s awesome to allow us to process in parallel as much as we want. But really not awesome when we want to first create a subscription and then update it as it happens naturally.
- “Webhook endpoints might occasionally receive the same event more than once” ~Stripe docs. This is a good one. Anything you build relying on Stripe webhook has to be Idempotence. Don’t miss this one!
- You can’t persist the webhook and reprocess them later. Stripe is limiting us to verify the signature of the webhook within 5 minutes to avoid replay attacks. Failing to do so, will make the event fail with an error. While it increases the security, it doesn’t allow us to propose a design where we persist all webhooks and process them later. It’s not possible because the first step of processing a webhook is unpacking it with stripe.webhooks.constructEvent which verifies the signature and would fail if invoked more than 5 minutes after the webhooked arrived.
- “Stripe will attempt to notify you of a misconfigured endpoint via email if an endpoint has not responded with a 2xx HTTP status code for multiple days in a row” ~Stripe docs. If webhooks start failing, don’t rely on Stripe to let you know there is an issue. It will take a few days until they will let you know something is off.

We are addressing all the Stripe limitations and our requirements both on the code application level and the way we build our architecture.
‍

How do we mitigate limitations 1 and 2?

We are mitigating those two limitations on the application level and not on architecture level.

Limitation 1: Stripe does not guarantee delivery of events in the order in which they are generated

This is a very interesting topic that caused us quite a lot of trouble in the beginning when we just started. For us, the key was to listen to the right events for different use cases and fetch data from Stripe when missing. Let’s look at a concrete example. When creating a subscription we are listening to a subscription.created event in order to activate a subscription after the payment was successful. We also listen to subscription.updated webhook to sync subscription status. In some cases when we just started we received subscription.updated events before subscription.created events (exactly as Stripe docs are promising).It caused our lambda to start throwing exceptions as the webhooks arrived in the wrong order. The subscription.updated handle expected that we already processed a subscription.created webhook and created a subscription.

The solution for us was to fetch the subscription and create it if it’s not yet created in our database. The code could look something like:

‍module.exports = leftpad;

function leftpad(str, len, ch) {
  str = String(str);
  var i = -1;

  if (!ch && ch !== 0) ch = ' ';

  len = len - str.length;

  while (i++ < len) {
    str = ch + str;
  }
  return str;
}

We always keep this limitation in mind when we grow and listen and process more events. We should always think about that in our designs and implementations.

Limitation 2: Webhook endpoints might occasionally receive the same event more than once.

We didn’t experience that one too often but we did build our processing mechanism in a way that we are working in an idempotent manner. We can always run the same message through our system and the result will always be the same.

The last two limitations are mitigated on the infrastructure level and therefore let’s try to get a better understanding of how to design a Stripe webhook integration.
‍

Overview:

AWS API Gateway to take the hit from Stripe.
Lambda function Gateway integration that verifies the signature of the webhooks and persists them on a queue.
AWS SQS with DLQ for reprocessing,
AWS Lambda function that will read and handle the event.

And now let's review each infra item and explain how it answers all our tech requirements.
‍

AWS API Gateway to take the hit from Stripe. (limitation 3)

Our requirement here was to build an API to receive the webhooks from Stripe and persist them on a queue without having extra logic that can become a point of failure and process them with a Lambda function later.

We knew that API Gateway has native integrations with SQS (and Lambda).

This wouldn’t be sufficient for us as we had to have some code verifying and constructing the messages at the moment they arrive synchronously (limitation No 3).

In our implementation, we had to extract the Stripe signature from the headers to use it to verify the event's authenticity and the Stripe event itself. We achieved that by using Mapping Template on the Integration Request. Essentially it allows us to remap the http requests properties to a JSON that would be easily processed by the lambda. In our case, we are mapping the Stripe Signature from the header of the webhook to a stripeSignature parameter and the body of the webhook to a stripeEvent parameter. The mapping code:

import { Detail } from "@raycast/api";

export default function Command() {
  return <Detail markdown="Hello World" />;
}

‍

Lambda function Gateway integration (limitation No 4)

We ended up choosing to have a native API Gateway Lambda integration that would be super simple, doing as little processing as possible. This is important for us as it is the first component that the external stripe webhook hits when entering our cloud. Failing to persist an event here would cause a failure to act upon that webhook and its data will be lost forever.

import { Detail } from "@raycast/api";

export default function Command() {
  return <Detail markdown="Hello World" />;
}

This mapping code should be planted in the API Gateway in the Integration requests:

‍

Moreover, we are very carefully monitoring this lambda for errors to actively know if we are missing any webhooks immediately. According to limitation No 4, it will take Stripe a few days to let us know if there is a problem with the webhooks. As Stripe webhooks are so crucial to our business, we can’t afford not knowing that something is wrong with them only in a few days.

AWS SQS with DLQ for reprocessing

We already rely heavily in Stigg on having SQS with DLQs deployed alongside in order to help us scale quickly and process with multiple lambdas when needed. We also rely on that architecture to automatically retry or manually reprocess messages that have failed after fixing the issue causing the failure.