DEV Community: Aydrian

Scaling Access Control: Leveling Up Roles with Relationships Using ReBAC

Aydrian — Wed, 23 Jul 2025 17:14:28 +0000

If you’ve ever built a collaborative app, you’ve probably reached for role-based access control (RBAC) right out of the gate—even if you didn’t call it that at the time. Maybe you just added a quick “is_admin” flag to your user model, or wrote a few if-statements to check if someone is a “member” before letting them do something. That’s RBAC in action, whether you realize it or not. It’s the default move: give users roles like “admin,” “member,” or “guest,” and lock down your endpoints accordingly. It’s simple, familiar, and gets the job done, at least at first.

Imagine you’re creating a shared travel app. In this case, it’s just the API, designed to help groups manage shared expenses on trips. Think splitting hotel bills, tracking who paid for gas, and making sure everyone settles up at the end. Like most projects, it starts with a straightforward RBAC setup. Users get roles, roles get permissions, and everything seems to work fine.

As the app grows, the requirements start to change. Suddenly, there’s a new challenge: what if you only want users to share expenses they actually created? RBAC doesn’t really have a good answer for that. Roles can’t capture the nuance of “this user owns that expense.” Now you need to be able to model a relationship between a user and an expense — and that’s where things start to get interesting.

The journey from RBAC to relationship-based access control (ReBAC) shows how real-world problems push access control beyond simple roles. Here’s how the shared-travel-app API used Oso to level up its authorization approach to meet those needs.

The Starting Point: Homemade RBAC

When you’re building out the first version of an API, you usually don’t overthink access control. Most developers just want to ship something that works.

That means reaching for the basics: you define a few roles, store them in your database, and sprinkle permission checks throughout your routes or controllers. There’s no fancy framework — just simple if-statements and a focus on getting the job done.

For the shared-travel-app API, the goal is to help groups manage shared expenses on trips. Users can create trips, add expenses, and invite others to join. To keep things organized, the app starts with three roles:

Role	What They Can Do
Organizer	Full control of the trip (edit/delete) Add, remove, or change participant roles Create and manage all expenses View everything (trip details, participants, expenses)
Participant	View trip details See who else is on the trip Create new expenses Edit/delete their own expenses View all expenses
Viewer	View trip details See who is on the trip View all expenses Cannot create or modify anything

To keep things simple, I stored each user’s role for a trip in a trip_roles table:

CREATE TABLE `trip_roles` (
    `id` text PRIMARY KEY NOT NULL,
    `trip_id` text NOT NULL,
    `user_id` text NOT NULL,
    `role_id` text NOT NULL,
    `created_at` integer DEFAULT CURRENT_TIMESTAMP NOT NULL,
    FOREIGN KEY (`trip_id`) REFERENCES `trips`(`id`) ON UPDATE no action ON DELETE cascade,
    FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE cascade,
    FOREIGN KEY (`role_id`) REFERENCES `roles`(`id`) ON UPDATE no action ON DELETE cascade
);

Then, I wrote a custom Hono.js middleware to check permissions by querying the database before handling each request. Here’s a simplified version:

async function rbacMiddleware(c, next) {
  const userId = c.req.header("x-user-id");
  const tripId = c.req.param("tripId");
  // Query the user's role for this trip
  const { rows } = await db.query(
    "SELECT role FROM trip_roles WHERE user_id = $1 AND trip_id = $2",
    [userId, tripId]
  );
  const role = rows[0]?.role;

  // Basic permission check
  if (role === "organizer") {
    return next();
  }
  if (
    role === "participant" &&
    c.req.method === "POST" &&
    c.req.path === `/trips/${tripId}/expenses`
  ) {
    return next();
  }
  if (role === "viewer" && c.req.method === "GET") {
    return next();
  }

  return c.text("Forbidden", 403);
}

I could then just drop this middleware into my routes and call it a day.

It worked fine for a while, but as the app grew, the permission logic started to get messy and hard to maintain. That’s when I started looking for something more robust.

Migrating to Oso: Streamlining RBAC

In preparation to support new features that required more advanced permission structures, I realized my homemade AuthZ solution, while working fine and handled neatly using Hono.js middleware, wasn’t going to cut it for the next stage. I also recognized that I would have to keep adding this permissions logic to my controllers, middleware, and other parts of the app, which would quickly become hard to read, debug, and maintain. I wanted to use an authorization system that could scale with my needs and make it easy to introduce relationship-based permissions down the line. So, before things got complicated, I decided to migrate my existing role checks to Oso, setting myself up for a smoother path to ReBAC and beyond.

Oso is authorization as a service, a policy engine that lets you centralize your authorization logic and keep your code clean. Instead of sprinkling permission checks everywhere, you define your rules in one place and let Oso handle the rest. With the official Oso Cloud Node.js SDK, integrating Oso into my app was straightforward.

The Dual Writes Pattern

One of the first things I had to tackle was keeping my existing roles in sync while I migrated. Enter the Dual Writes Pattern: this involves keeping Oso Cloud in sync with the database by following any changes with fact updates.

Here’s a simplified example of what that looked like:

// Assign the user the "Organizer" role when creating a trip
await this.db.insert(tripRoles).values({
  tripId: dbTrip.id,
  userId: userId,
  roleId: organizerRoleId,
});

await this.oso.insert([
  "has_role",
  { type: "User", id: userId },
  { type: "String", id: "organizer" },
  { type: "Trip", id: dbTrip.id },
]);

Centralizing Permissions with Oso

With Oso in place, I could finally move my permission logic out of the controllers and into policy files. Instead of a bunch of if statements, I defined my rules declaratively using Polar:

actor User {}

# Everyone will be added to "default" organization
resource Organization {
  roles = ["member"];
  permissions = ["trip.create", "trip.list"];

  "trip.create" if "member";
  "trip.list" if "member";
}

resource Trip {
  roles = ["organizer", "participant", "viewer"];
  permissions = ["manage", "view", "expense.create", "expense.list", "participants.list", "participants.manage"];
  relations = {
    organization: Organization,
  };

  "view" if "viewer";
  "participants.list" if "viewer";
  "expense.list" if "viewer";

  "viewer" if "participant";
  "expense.create" if "participant";

  "participant" if "organizer";
  "manage" if "organizer";
  "participants.manage" if "organizer";
}

resource Expense {
  roles = ["editor", "viewer"];
  permissions = ["manage", "view"];

  relations = {
    trip: Trip
  };

  "editor" if "participant" on "trip";
  "viewer" if "viewer" on "trip";


  "view" if "viewer";

  "viewer" if "editor";

  "manage" if "editor";
}

Now, checking permissions in my routes was as simple as:

const user = c.get("user");
const oso = getAuthz(c);
const resourceId = resourceIdGetter(c);

const authorizeParams: AuthorizeParams<T> = [
  { type: "User", id: user.id },
  action,
  { type: resource, id: resourceId },
] as const;

const authorized = await(oso.authorize as AuthorizeFunction)(
  ...authorizeParams
);

I no longer needed to rely on if statements and database queries to verify permissions. Switching to Oso api calls simplified my custom authZ middleware.

The Benefits

Switching to Oso gave me a bunch of wins:

Centralized policies: All my authorization logic lives in one place, making it easy to audit and update.
Cleaner code: No more scattered permission checks, just simple calls to Oso.
Easier onboarding: New contributors can see all the rules at a glance, without hunting through controllers.
Flexibility: When requirements change, I update the policy, not a dozen different files.

Migrating to Oso allowed me to keep moving fast without sacrificing security or maintainability. And as you’ll soon discover, it made it a lot easier to handle more complex scenarios down the road.

When Roles Aren’t Enough: The Expense-Sharing Challenge

Things were smooth with RBAC until I hit a real-world scenario that didn’t fit the mold: what if I only want participants to be able to view or edit expenses they actually created? In other words, a participant shouldn’t be able to view or edit expenses created by someone else on the same trip — unless the expense has specifically been shared with them. And only the creator of an expense should be able to share it with others.

This is where traditional RBAC starts to fall apart. Roles are great for broad strokes — organizers can do everything, participants can add expenses, viewers can look but not touch. But roles alone don’t capture the relationships between users and specific resources. In this case, the permission isn’t just about being a participant; it’s about being the creator of a particular expense, or having that expense shared with you by its creator.

To handle this, I needed something more granular: relationship-based access control (ReBAC). With ReBAC, I can define permissions based on how a user is connected to a resource, not just what role they have. This lets me say, “You can only manage expenses you created, or those that have been shared with you,” which is exactly what the app needed as it grew more complex.

Introducing ReBAC: Relationship-Based Access Control

So what exactly is ReBAC, and how is it different from the RBAC most of us start with?

RBAC (Role-Based Access Control) is all about grouping permissions based on roles. The organizer role might include view, create, edit, and delete permissions. If you’re assigned the organizer role, you gain access to those permissions; if you’re assigned a participant role, you get a set of permissions defined by that role. It’s simple and works well for broad categories of users, but it doesn’t account for the relationships between users and specific resources.

ReBAC (Relationship-Based Access Control) takes things a step further. Instead of just looking at what role a user has, ReBAC considers how a user is connected to a resource. For example, you might have permission to edit an expense not just because you’re a participant, but because you’re the creator of that expense — or because the creator has shared it with you.

This model is a perfect fit for the expense-sharing challenge. With ReBAC, I can say, “Participants can view or edit expenses they created, and can share those expenses with others.” The permissions aren’t just tied to roles, but to the relationships users have with each expense.

Extending Oso: From RBAC to ReBAC

One of the best things about using Oso is how easy it is to evolve your authorization model as your app’s needs change. I started with classic RBAC policies, but when it was time to support more granular, relationship-based permissions, I didn’t have to rewrite everything. I just extended my existing policies.

Oso’s policy engine is designed for this kind of flexibility. Instead of locking you into a rigid structure, it lets you define new facts and rules as your requirements grow. For the expense-sharing feature, I needed to express relationships like “user is the creator of this expense” or “expense has been shared with these users.”

Adding New Facts

To support these new relationships, I started keeping track of which users created which expenses, and which expenses were shared with which users. This meant syncing a couple of new facts to Oso Cloud whenever data changed:

// Set up Oso relations in create expense
await this.oso.batch(async (tx) => {
  // Set trip relation
  await tx.insert([
    "has_relation",
    { type: "Expense", id: newExpense.id },
    "trip",
    { type: "Trip", id: tripId },
  ]);

  // Set owner relation
  await tx.insert([
    "has_role",
    { type: "User", id: userId },
    { type: "String", id: "owner" },
    { type: "Expense", id: newExpense.id },
  ]);
});

Updating the Policy

With the new facts in place, updating my main.polar policy file was straightforward. I just added rules to check for these relationships:

resource Expense {
  roles = ["owner", "viewer"];
  permissions = ["manage", "view", "share"];
  relations = {
    trip: Trip,
    shared_with: User
  };

  # Owner permissions (expense creator)
  "owner" if "owner";
  "manage" if "owner";
  "view" if "owner";
  "share" if "owner";

  # Shared permissions
  "viewer" if "shared_with";
  "view" if "shared_with";

  # Trip organizers can manage any expense
  "manage" if "organizer" on "trip";
  "view" if "organizer" on "trip";
  "share" if "organizer" on "trip";

  # Inheritance
  "view" if "viewer";
}

No need to rip out my old RBAC logic — just layer on the new rules where needed.

The Developer Experience

The best part? Evolving my policies was natural and low-friction. I didn’t have to scatter new permission checks throughout the codebase or worry about breaking existing features. Oso let me keep all my authorization logic in one place, so I could focus on the new relationships and permissions without losing sleep over the old ones. Because I created a custom middleware to handle the Oso permission checking, I simply needed to add it to the route definition with the resource and permission.

router.post(
  "/",
  zValidator("param", tripParamSchema),
  withOsoAuth("Trip", "expense.create"),
  zValidator("json", createExpenseSchema),
  async (c) => {
    const db = c.get("db");
    const oso = c.get("oso");
    const user = c.get("user");
    const { tripId } = c.req.valid("param");
    const expenseData = c.req.valid("json");

    try {
      const expenseService: ExpenseService = new DefaultExpenseService(db, oso);

      const newExpense = await expenseService.createExpense(
        tripId,
        user!.id,
        expenseData
      );

      return c.json(newExpense, 201);
    } catch (error) {
      console.error("Error adding new expense:", error);
      throw new HTTPException(500, { message: "Internal Server Error" });
    }
  }
);

Moving from RBAC to ReBAC with Oso was more about extending what I already had than starting from scratch. That’s a huge win for developer sanity and for keeping your app’s access control both powerful and maintainable.

Check out the entire project and see what it looked like at each stage in the shared-travel-app GitHub repository.

Conclusion

Building access control for an app always starts simple, but real-world requirements have a way of pushing things further. I began with homemade RBAC, then leveled up to Oso for cleaner, more scalable policies. When the need for more nuanced permissions came along, moving to ReBAC was a natural next step — and Oso made that transition smooth.

The best part is that I didn’t have to throw out what I’d already built. With Oso, evolving from roles to relationships was just a matter of adding new facts and rules, not rewriting everything from scratch. If you’re working on an app that’s growing in complexity, having a flexible policy engine like Oso can save you a ton of time and headaches.

Access control shouldn’t slow you down. With the right tools, you can keep your code clean, your policies clear, and your users happy as your app grows.

Current Trends in Authorization: Simplifying Access Control

Aydrian — Tue, 10 Jun 2025 20:13:47 +0000

Introduction

Authorization is a critical component of modern application security, but implementing it effectively can be challenging. Developers are increasingly turning to proven approaches like Zero-Trust Authorization, Policy-as-Code, and Context-Aware Authorization to simplify access control and enhance security. These approaches not only address the complexities of scaling applications but also integrate seamlessly into modern development workflows.

This post explores the current authorization trends shaping how developers build secure, scalable systems. You'll also discover tools like Open Policy Agent (OPA) and Oso that simplify policy management and help you build secure, scalable systems. Whether you're designing a new application or modernizing an existing one, these trends and tools will help you simplify authorization and build more secure systems.

Why Traditional Authorization Falls Short

Authorization has always been a critical part of application security, ensuring users only access the resources they’re permitted to. Historically, models like Role-Based Access Control (RBAC) and Discretionary Access Control (DAC) have been the go-to solutions. While these methods worked well in simpler, static environments, they struggle to keep up with the demands of today’s dynamic, cloud-native systems.

Here’s why traditional authorization methods fall short:

Lack of Scalability

Managing static roles and permissions becomes a nightmare as systems grow more complex. For example, in a microservices architecture with hundreds of services, maintaining hardcoded roles across each service can lead to inconsistencies and errors. Developers often find themselves duplicating effort or manually updating permissions, which is both time-consuming and error-prone.

No Support for Fine-Grained Access Control

Traditional models are too rigid to handle modern requirements like fine-grained access control. For instance, RBAC might allow a user to access an entire database, but what if you need to restrict access to specific rows, columns, or even individual data points? Implementing this level of granularity with traditional methods is cumbersome and often requires custom, ad-hoc solutions.

Fragmented Authorization Across Systems

Many organizations rely on a patchwork of authorization mechanisms embedded within individual applications. This leads to duplicated logic, inconsistent policies, and security gaps. For developers, this fragmentation makes it difficult to enforce centralized policies or gain visibility into who has access to what.

Inability to Handle Real-Time Context

Static policies are no longer sufficient in a world where access requirements can change on the fly. Traditional systems lack the agility to evaluate real-time context, such as a user’s location, device, or behavior. For example, if a user logs in from an untrusted device, traditional models can’t dynamically adjust permissions or enforce additional authentication steps.

These limitations highlight the need for modern, scalable, and adaptive authorization solutions. Developers require tools and frameworks that simplify implementation, support dynamic policies, and integrate seamlessly with today’s distributed systems.

Current Trends in Authorization

To address these challenges, developers are adopting modern approaches that simplify implementation, improve scalability, and enhance security. Below, we’ll dive into key trends that are shaping the current landscape of authorization.

Policy-as-Code: Simplifying Authorization Management

Managing access control policies manually can quickly become error-prone and unscalable, especially in complex systems. Policy-as-code addresses this challenge by allowing developers to define, version, and manage policies programmatically, just like application code. This approach integrates seamlessly into modern development workflows, enabling better collaboration, testing, and automation.

With policy-as-code, you can:

Version policies: Store policies in source control systems like Git, enabling version history, rollbacks, and collaboration.
Automate testing: Write unit tests for policies to ensure they behave as expected under different scenarios.
Integrate into CI/CD pipelines: Automatically validate and deploy policy changes alongside application updates.

For example, Open Policy Agent (OPA) provides a declarative language called Rego for writing policies. Here's a simple example of an OPA policy that restricts access to an API based on user roles:

package example.authz

default allow = false

allow {  
  input.user.role == "admin"  
}

This policy ensures that only users with the admin role can access the resource. Policies like this are well-suited to enforcement at the infrastructure level (e.g., Kubernetes, API gateways).

Another tool, Oso, lets you decouple authorization logic from your application code. With Oso, you can write policies in a declarative format and integrate them into your codebase via API. For instance:

actor User {}

resource Project {  
  permissions = ["read", "manage"]  
  roles = ["member", "admin"];

  # admins can do everything members can do  
  "member" if "admin";

  # member permissions  
  "read" if "member";

  # admin permissions  
  "manage" if "admin";  
}

Getting Started

Choose a tool: Explore tools like OPA or Oso based on your application’s needs.
Define policies: Start with simple policies and gradually expand to handle more complex scenarios.
Integrate and test: Add policies to your application and write tests to validate their behavior.
Automate deployment: Use CI/CD pipelines to manage and deploy policy updates efficiently.

Policy-as-code empowers developers to treat authorization as a first-class citizen in their development process, ensuring security and scalability without adding unnecessary complexity.

Context-Aware Authorization: Adapting to Dynamic Environments

Traditional authorization models rely on static rules, such as predefined roles or permissions. However, modern applications operate in dynamic environments where access decisions need to account for real-time factors like user location, device type, time of access, and more. Context-aware authorization enables more granular and adaptive access control by incorporating these dynamic attributes into decision-making.

Why Context Matters

Static rules alone may not be sufficient to secure modern systems. For example:

A user with admin privileges might be accessing sensitive data from an untrusted network.
An employee logging in outside of business hours could indicate a potential security threat.

By incorporating context, you can enforce policies that adapt to these scenarios and mitigate risks.

How It Works

Context-aware authorization evaluates additional attributes (often called claims) at runtime. These attributes can include:

User attributes: Role, department, or group membership.
Environmental factors: IP address, geolocation, or device type.
Temporal conditions: Time of day or session duration.

For example, consider a policy that grants access to a resource only if the user is an admin and is accessing the system from a trusted network:

actor User {}

resource Device {  
  Relations = { user:User };  
}

resource Project {  
  roles = ["member", "admin"];  
  permissions = ["read", "update", "delete"]

  # admins can do everything members can do  
  "member" if "admin";

  # member permissions  
  "read" if "member";

  # admin permission  
  "update" if "admin";  
  "delete" if "admin";  
}

# Only allow an operation if the user has the requested permission  
# AND they are requesting from a trusted device  
allow(user:User, permission:String, project:Project) if  
  has_permission(user, permission, project) and  
  device matches Device and  
  is_using_trusted_device(user, device) and  
  is_on_trusted_network(user);

# Rule to determine whether a user is using a trusted device  
is_using_trusted_device(user:User, device:Device) if  
  has_relation(device, "user", user) and  
  is_trusted(device) and  
  is_active_device(user, device);

This policy ensures that even admins are restricted when accessing from untrusted devices and networks, adding an extra layer of security.

Real-World Use Cases

Restricting access based on location: Allow access to sensitive resources only when users are within the corporate network or a specific geographic region.
Device-based policies: Require additional authentication steps if a user logs in from an unknown or untrusted device.
Time-based restrictions: Limit access to certain resources outside of business hours or during maintenance windows.

Getting Started

Identify dynamic factors: Determine which contextual attributes are relevant to your application (e.g., location, device, time).
Integrate context providers: Use tools like identity providers (e.g., Auth0, Okta) or custom middleware to collect and pass contextual data to your authorization system.
Write adaptive policies: Define policies that incorporate these attributes using tools like Open Policy Agent (OPA) or Oso.
Test and monitor: Simulate various scenarios to validate your policies and monitor their effectiveness in production.

Context-aware authorization allows you to enforce fine-grained, adaptive access control, improving security without compromising user experience.

Zero-Trust Authorization: Trust Nothing, Verify Everything

In traditional security models, users and devices inside the network perimeter are often implicitly trusted. However, as applications become more distributed and users access resources from various locations and devices, this approach is no longer sufficient. Zero-trust authorization enforces strict identity verification for every access request, regardless of whether the request originates from inside or outside the network.

Core Principles of Zero-Trust Authorization

Verify explicitly: Always authenticate and authorize users and devices based on all available data points, such as identity, location, and device posture.
Least privilege access: Grant users and devices the minimum level of access required to perform their tasks.
Assume breach: Design systems with the assumption that an attacker may already have access, ensuring that security controls are in place at every layer.

How Zero-Trust Authorization Works

Zero-trust authorization evaluates every access request dynamically, using contextual information to make decisions. For example:

A user attempting to access a sensitive resource must authenticate using multi-factor authentication (MFA).
Access is granted only if the user’s device meets security requirements (e.g., up-to-date antivirus software, encrypted storage).

The context-aware policy example in the previous section implements a zero-trust model.

Real-World Use Cases

Securing remote work environments: Enforce strict access controls for employees working from home or public networks.
Protecting sensitive resources: Require additional authentication for accessing critical systems or data.
Dynamic access decisions: Continuously evaluate access requests based on changing conditions, such as device health or user behavior.

Getting Started with Zero-Trust Authorization

Adopt a zero-trust mindset: Treat all access requests as untrusted until explicitly verified.
Leverage identity providers: Use tools like Okta, Azure AD, or Auth0 to manage user identities and enforce MFA.
Implement policy-as-code: Use tools like Open Policy Agent (OPA) or Oso to define and enforce fine-grained access policies.
Monitor and audit: Continuously monitor access requests and audit logs to detect anomalies and improve policies.

Zero-trust authorization provides a robust framework for securing modern applications by continuously verifying every access request. By combining identity, context, and policy enforcement, you can reduce the attack surface and protect your systems against evolving threats.

Fine-Grained Access Control: Precision in Authorization

As applications grow more complex, traditional role-based access control (RBAC) often falls short in meeting modern security and scalability requirements. Fine-grained access control (FGAC) provides a more precise way to manage permissions by evaluating multiple attributes, such as user roles, resource types, actions, and contextual factors, to make access decisions.

Why Fine-Grained Access Control?

RBAC typically assigns permissions at a coarse level, such as granting access to all resources within a department. However, this approach can lead to over-permissioning, where users have access to resources they don’t need. FGAC solves this problem by enabling:

Granular permissions: Define access rules at the level of individual resources or actions.
Dynamic decision-making: Incorporate real-time context, such as user location or device status, into access decisions.
Improved security: Minimize the attack surface by granting users only the permissions they need.

How Fine-Grained Access Control Works

FGAC evaluates access requests based on a combination of attributes, such as:

User attributes: Role, department, or clearance level.
Resource attributes: Type, sensitivity, or ownership.
Action attributes: Read, write, delete, or execute operations.
Contextual attributes: Time of access, location, or device compliance.

For example, consider a policy that allows a user to edit a document only if they are the document owner and are accessing the system during business hours:

For example, consider a policy that allows a user to access a server only when granted elevated privileges for 1 hour.

actor User {}

resource Server {  
  permissions = ["login"]  
  roles = ["member", "triage"];

  # triage can do everything members can do  
  "member" if "triage";

  # member permissions  
  "login" if "triage";  
}

# triage_expiration_time is 1 hour after the triage role is granted
has_role(actor: Actor, "triage", server: Server) if  
  has_role(actor, "member", server) and  
  @current_unix_time < triage_expiration_time;

This policy ensures that:

The user is a member of the resource.
The action is explicitly allowed (e.g., "login").
The request is made within the triage expiration timeframe.

Real-World Use Cases

Multi-tenant applications: Restrict access to data so that users can only view or modify resources belonging to their organization.
Data privacy compliance: Enforce policies that limit access to sensitive data based on user roles and geographic location (e.g., GDPR compliance).
API security: Grant permissions for specific API endpoints based on user roles and the type of request (e.g., read vs. write).

Getting Started with Fine-Grained Access Control

Define your attributes: Identify the user, resource, action, and contextual attributes relevant to your application.
Choose a policy engine: Use tools like Open Policy Agent (OPA) or Oso to define and enforce fine-grained policies.
Write and test policies: Start with simple policies and expand to handle more complex scenarios. Use automated tests to validate policy behavior.
Integrate with your application: Embed the policy engine into your application or infrastructure to enforce access control dynamically.

Fine-grained access control provides the flexibility and precision needed to secure modern applications. By combining attributes and context, you can enforce policies that adapt to your application's unique requirements while minimizing over-permissioning.

Modern Authorization Services: Simplifying Access Control

As applications grow in complexity, managing access control becomes a significant challenge for developers. Authorization is no longer just about assigning roles or permissions—it’s about building secure, scalable systems that adapt to dynamic environments. Modern authorization services address these challenges by simplifying implementation, enabling fine-grained control, and providing tools to centralize and streamline access management.

Why Authorization is Challenging for Developers

Authorization is often one of the most frustrating aspects of application development. Common pain points include:

Scattered Logic: Hardcoding access rules across multiple services leads to unmanageable and inconsistent policies that are difficult to debug or update.
Dynamic Contexts: Modern applications require decisions based on real-time factors like user location, device type, or time of access.
Fine-Grained Permissions: Developers need to go beyond simple "admin vs. user" roles to define granular rules, such as controlling access to specific resources or actions.
Scalability: Handling millions of users and resources without introducing performance bottlenecks is a critical requirement for modern systems.

These challenges make it clear that traditional, ad-hoc approaches to authorization are no longer sufficient.

What Modern Authorization Services Offer

Modern authorization services are designed to address these pain points by providing:

Policy-as-Code: Define access rules in a centralized, declarative format that integrates seamlessly with your application. This approach ensures consistency and simplifies updates.
Flexibility: Support for multiple authorization models, including Role-Based Access Control (RBAC), Attribute-Based Access Control (ABAC), and custom logic.
Integration: SDKs and APIs that work with a variety of architectures, from monoliths to microservices, enabling easy adoption without overhauling your stack.
Observability: Tools to debug, test, and monitor policies, ensuring they work as expected and providing visibility into access decisions.

How Developers Can Benefit

Using an authorization-as-a-service platform allows developers to focus on building application features instead of reinventing access control. For example:

Centralized Policy Management: Platforms like Oso let you define policies in a human-readable language like Polar and enforce them across your application.
Simplified Complex Scenarios: Handle advanced use cases such as multi-tenant architectures, context-aware rules, or fine-grained permissions with minimal effort.
Improved Debugging and Testing: Test and debug policies in isolation to ensure they behave as expected before deploying them to production.

Modern authorization services simplify access control, allowing developers to build secure, scalable systems without getting bogged down in boilerplate code or scattered logic. By adopting these tools, you can focus on delivering features while ensuring robust, maintainable authorization.

Conclusion

Authorization is no longer a static, one-size-fits-all process. Modern systems demand flexible, scalable, and secure approaches to managing access. As developers, you play a critical role in shaping secure and adaptive systems that protect sensitive resources while delivering seamless user experiences.

By embracing advancements like Zero-Trust Authorization, Policy-as-Code, and Context-Aware Authorization, you can address the challenges of today’s dynamic environments. Tools such as Open Policy Agent (OPA) and Oso simplify policy management, making it easier to implement these practices effectively.

Whether you're building a new application or modernizing an existing one, adopting these trends and tools will help you create secure, adaptable systems that meet the demands of modern development workflows.

Best of Courier at Hackabull 2021: Speed Friender

Aydrian — Thu, 03 Jun 2021 22:48:15 +0000

During the weekend of March 13th, 2021, we sponsored Hackabull, a 24 hour virtual hackathon hosted by the Society of Hispanic Professional Engineers at the University of South Florida. This was the first hackathon Courier has sponsored. We know that events are a great way to talk directly to developers and I am a huge advocate for the value a hackathon can bring to a company with a new developer focused product. In all my years as a developer advocate and a software engineer before that, I learned how valuable hackathons can be for gathering real time feedback and building community around your product. Unfortunately, due to the pandemic, attending developer events has not been a possibility and virtual events haven't felt like the best fit. After consulting with Jonathan Gottfried of Major League Hacking (MLH), he convinced me to take another look at virtual hackathons. Jonathan gave me a short list of hackathons to consider and Hackabull was near the top. This seemed like a great choice to me because I had attended their 2018 hackathon in person and knew what to expect.

From engaging with the developer community to encouraging Courier sign ups, Hackathons have countless business benefits and opportunities. My favorite part, however, is watching the participants come up with interesting and unique ways to use Courier that we may never have thought of before. What’s great about Courier is that it is quick to implement and is immediately actionable, which the team that built Speed Friender was able to take full advantage of. Speed Friender is a Discord app that provides online speed dating for friends, and most importantly, it was built in just 24 hours.

The first step to getting teams like this one ready for the Hackathon was to help them understand more about how to use Courier and the many provider integrations together, which I was able to do through a workshop at the beginning of the event. I built the workshop around my Twitch Notification blog series and I chose to use Discord since it was part of the virtual hackathon platform.

I had one hour to guide the participants through setting up an application that would accept a Twitch EventSub webhooks when a streamer started streaming and send a notification to a Discord user using Courier.

In the interest of time and to help focus the workshop on the use of Courier, I provided the participants with a Glitch application they could remix and use as a starting point. The Glitch application used Node.js and Express.js to accept POSTs from Twitch EventSub, query the Twitch API for more details, and call the Courier List Send. Once the participants created a Twitch application and updated some environment variables, they were ready to set up Courier and start building their notification.

Next, the participants created a new Courier developer account and went through the onboarding process. Once everyone was finished, we continued by setting up the Discord Integration. Courier can send messages to channels and users on behalf of a Discord Bot. Prior to the workshop, I created a Courier Bot and installed it on the Hackabull Discord Server. I shared the bot token with the participants so they would be able to send to Hackabull Discord users without having to set up their own Discord Bot.

With the Discord Integration configures, we moved on to the fun part: building the notification that would be sent when a streamer went live on Twitch. I showed the participants an example notification, how to use the stream data passed to Courier, and how to preview the notification. Then I gave them some time to be creative and ask questions.

Once the notification was designed and published, we created a recipient with a Discord profile and subscribed it to the twitch.stream.online list. We were able to accomplish this using a couple cURL commands

Create a recipient with a Discord profile:

curl --request POST \
  --url https://api.courier.com/profiles/YOUR_ID \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer COURIER_AUTH_TOKEN' \
  --header 'Content-Type: application/json' \
  --data '{"profile":{"discord":{"user_id":"DISCORD_USER_ID"}}}'

Create a recipient with a Discord profile

Subscribe the recipient to the list:

curl --request PUT \
  --url https://api.courier.com/lists/twitch.stream.online/subscriptions/YOUR_ID \
  --header 'Authorization: Bearer COURIER_AUTH_TOKEN'

Subscribe the recipient to the list

Finally, with our Glitch app set up and Courier account configured, we were ready to set up our Twitch subscription to start accepting the stream online events from EventSub. We did this using the Twitch CLI.

Get broadcaster id for Twitch user:

twitch api get users -q login=trycourier

Get broadcaster id for Twitch user

Subscribe to the stream.online event:

twitch api post eventsub/subscriptions -b '{
    "type": "stream.online",
    "version": "1",
    "condition": {
        "broadcaster_user_id": "YOUR_BROADCASTER_ID"
    },
    "transport": {
        "method": "webhook",
        "callback": "https://EXTERNAL_URL/webhook/callback",
        "secret": "YOUR_SECRET"
    }
}'

Subscribe to the stream.online event

By the end of the workshop, all the attendees walked away with a working application that allowed them to start exploring the different features of Courier.

This workshop provided the Speed Friender team with the foundation they needed to build an application worthy of the Best Use of Courier prize.

The Speed Friender team was made up of members Mara Hart and Danielle Zevitz. Mara and Danielle are both computer science majors at the University of Virginia and Major League Hacking coaches. Mara helped found the UVA chapter of Girls Who Code and enjoys showcasing her personal projects and helping others get started coding on Twitch. Together they were inspired to build Speed Friender after noticing that new students were having trouble connecting with others due to the pandemic. To make up for the lack of spontaneous connection, they created a Discord bot that would facilitate creating connections between members and creating rooms for private conversations.

After setting up their Courier accounts to use Discord during the workshop, Mara and Danielle started looking into the capabilities of Discord bots. After a little research, they were able to create their own Discord bot and install it into their own Discord server. They handed the bot logic by adding Discord.js to the Remixed Glitch application from the workshop. After updating their Discord Integration to use their bot, they added Twilio for SMS and PostMark for email. Now using a single API call, the Discord bot is able to notify members when they have a new match using their preferred channel.

Click To Play Hackbull Youtube Stream

They plan to continue developing this project with their Girls Who Code chapter and put it into practice in their Discord server.

I was personally very excited about this project. The Discord Integration was my Fall 2020 Dispatch Project so it was great to see it being used at a hackathon. This project also inspired some new features I'd like to add to Pigeon Bot in our Courier Community Discord Server. Having a bot be able to spin up private rooms and then tear them down would be great handling community support requests that involve sharing sensitive information.

How We Kept Datadog From Blowing Up Our AWS Bill

Aydrian — Wed, 12 May 2021 04:43:12 +0000

What are our real costs?
Identifying the problem
The impact of removing the Datadog layer

Some time back, our engineering leaders suddenly noticed dramatically increased billings from AWS that far outpaced typical benchmarks. The problem turned out to be data insights tools sending an unexpectedly large volume of data. The solution required a deep dive into how AWS billing works and an intricate series of steps to discover the location of the firehose of data. Courier’s Developer Advocate, Aydrian Howard, sat down with CTO Seth Carney to review and discuss the experience and how the team ultimately managed to control and reduce AWS costs.

Aydrian:

All right. Hello, Seth.

Seth:

Hey, Aydrian. How's it going?

How AWS credits from the YC program made life easier

Aydrian:

Good. So Courier is coming up on two years old. We're a YC company, which means we have some benefits that come along with that. Can we start by talking about the AWS benefits?

Seth:

Yeah, absolutely. So one of the really fantastic benefits of going through the YC program is that there is a fairly sizable chunk of credits that you can apply to your AWS account. With such a great benefit, we've been able to focus on investing the capital we would have spent there on other areas of Courier.

Aydrian:

Nice. So, this is really a benefit that not a lot of people get. How does this affect the way that we built Courier? If we did not have these credits, what would we have done differently from your perspective?

Seth:

Interesting question. Courier has got a floor as far as what we minimally have to pay to keep that infrastructure running. This includes provisioned concurrency and minimal costs for our Kinesis streams. We didn’t have to spend a lot of time thinking through how we would optimize our per-message costs. As an early-stage start-up, we have to make strategic decisions around where to invest. Instead of spending thousands of dollars monthly on hosting costs, we were able to focus on growing the team instead. So, I think the biggest thing that it has enabled us to do is really focus on building a product that our customers can find value in.

What are our real costs?

Aydrian:

So like I said, we're coming up on two years. The credits aren't going to last forever. I know that we have started looking into the costs, but really what kicked off this analysis of the cost - what we were paying for AWS?

Seth:

Yeah. It was an interesting question that our CEO [Troy] posed to me: what was our per message cost? This is a slightly more difficult question to answer than it seems on the surface. To be able to answer this question you have to understand what our floor cost is and what kind of volume that floor supports. And we didn't have that information. We couldn't look at our various sources and say, this is what it costs us to send each and every message that comes out of our platform..

When you're on credits, this is a little harder to figure out than you would think. You start to go back and look at historical costs and all you see is zero because you didn't pay anything.

Aydrian:

Exactly. So in looking into this, I guess, how did you go about figuring out what our cost was? And what surprised you about that?

Seth:

We cast a broad net at first, just trying to look at where we might be able to get some of the costs and information. We obviously started looking to AWS billing utilities for cost data. You could see them in real-time by month, but some of the more historical stuff was the challenge. We had a product called Datadog in place that did some cost estimations for us, but they weren’t particularly accurate as far as how some of our configuration is set up.

For example, we have a pretty high provision concurrency rate set up on our authorizers since that's a stove pipe in our application. All of our public-facing Lambda functions use a custom authorizer. We wanted to make sure we've got enough capacity to handle normal traffic in addition to having auto-scaling in place to handle spikes in usage. Datadog cost estimation doesn't really take things like that into account. We ended up connecting with a company called CloudZero that thought they could help us gain some of this insight. We integrated their product and started using it to analyze some of the costs per service, different breakdowns like per developer accounts, per service, per tag, and some other various things to start understanding where some of our cost breakdowns were. We definitely found some interesting things, and definitely unexpected things as well.

One of the things that stood out immediately was how high our CloudWatch and X-ray spend was. Our CloudWatch spend should have been around one-fifth to one-sixth of our total monthly Lambda invocation costs. Ours was approximately 1.5x our Lambda invocation costs. X-ray was also wildly out of bounds. It was even higher than our Cloudwatch spend. Somewhere in the neighborhood of 3x our Lambda costs.

Identifying the problem

Aydrian:

Let's talk about the steps that we took to identify and resolve the problem. So in using CloudZero, you said it gave you some resolutions. Is that how you went about it? Like we got the analysis and some steps to resolve issues?

Seth:

The product didn’t necessarily give us the solution. It gave us threads to pull on. We had to do the underlying analysis as to what was causing the increase in spend. The CloudZero team was actually super helpful as well. Ultimately, we had to dig in and figure out what was causing the costs for these services to be outside of expected bounds: determine which of our Lambda functions are generating large volumes of operational data and why. Did we have log processes in place that should be removed or cleaned up? Was our logging framework configured at the right level?

Aydrian:

And how did you figure out which functions were generating the most logs or how did you zero in on these different Lambda functions to clean them up? I know we are a serverless company and we probably have lots of Lambda functions that I don't even know about. So are we talking about hundreds of Lambda functions?

Seth:

That's a good question. I’d say we have around 100 functions. We started by looking at cost breakdowns by cost category to see where the outliers were. We started to tackle Cloudwatch logs first so we started analyzing those by function and log group. We found that there were a couple of things happening.

First, we had some cleanup to do. There were a bunch of debug messages that were being output in production. In development, in staging, you don’t think much of it. But, when you have a function being invoked millions of times. It can add up. In our case, we have functions that are executed more than a hundred million times per month.

This had some nominal impact on the log costs, but something was still generating large volumes of log data. It turns out that the Datadog layer that we had put in a few months ago to help get further insights into some of our Lambda functions was injecting the extra data into CloudWatch.

So we started this slow process of turning configuration bits off. Unfortunately, no matter how many bits we turned off, there were still minimal amounts of data being pumped in. But, with high request volumes, the extra bits were really adding up. As a test, we removed the layer entirely and saw an instantaneous reduction in both our X-ray costs and our CloudWatch costs.

You haven't heard me talk much about X-ray, but I will tell you that there was a constant thread under the current CloudWatch that I was constantly looking at in X-ray where we weren’t making much progress. It turns out the Datadog layer was pumping huge amounts of data into X-Ray even though it was turned off via configuration.

The impact of removing the Datadog layer

Aydrian:

So once you removed the Datadog layer and cleaned up, did usage drop back down to a level that you would expect, or was there more to be done?

Seth:

Things were back down into nominal zones at that point. Absolutely.

Aydrian:

Are there insights that we wish we had that we no longer do because we removed this? I know that it's not worth the cost, but are there these insights that it would have been nice to continue to monitor?

Seth:

I think there are insights that would be nice to have, but not at the expense that they were generating.

We would love more accuracy around timeouts of functions and things like that, but we can get in other ways. AWS has also recently released an alternative to the Datadog layer called AWS Lambda insights. We've toyed with replacing it with that, though we haven't yet gone that route.

Advice for other companies to reduce AWS costs

Aydrian:

So for the companies out there who are starting with the serverless model, such as ours, what types of insights for Lambda functions and for serverless architecture would you recommend that people keep an eye on?

Seth:

There are the pretty obvious things you want to monitor: errors, invocation time, memory utilization, and timeouts. But you also want to make sure you’re monitoring things like cold starts and how much of your provisioned capacity that you are utilizing. In conjunction with autoscaling, this will help you keep latency low for your consumers and scale-out when you have unexpected influxes in traffic.

If your Lambda functions are connected to SQS Queue or Kinesis Streams or Dynamo Streams, you’ll also want to monitor iterator ages, just to make sure that you're keeping up with the volume that's in the queue. Build up in iterator age might mean you need to adjust your parallelization factor or batch sizes to accommodate increased traffic.

Aydrian:

So to summarize, we did have a problem with costs. We seem to have those under control. We are looking to add some of the insights back without adding a lot of the costs. Do you have any final tips or recommendations for engineers and startups who are looking to start using serverless and setting this up from scratch?

Seth:

Both Serverless and AWS have helped us focus on the product we’re trying to build at Courier. But, you have to make sure you understand your ecosystem and use cases. Serverless makes it very easy to opt-in to new and (sometimes) useful features. But, it’s not always particularly easy to understand the cost impact of opting into those services. Especially, as you start to thread multiple services together at the scale of even tens of thousands of requests.

I suppose if you're on credits, you can be a little careless (for now). Perhaps more incentive to apply to the YC program!

We would love to hear more about your own experiences and issues with sky-high AWS costs. How did you solve the problem? Let us know by tweeting at us @trycourier or by joining our Discord server.

Authors: Seth Carney and Aydrian Howard > Courier Employees

Twitch notifications (part three): How to create and notify a list of subscribers using Courier

Aydrian — Tue, 09 Feb 2021 13:30:03 +0000

In this series, I explain how to use Twitch EventSub and Courier to automatically send notifications to multiple destinations – Slack, Discord, and more – when your Twitch stream goes live.

In part one, we built a Node.js app using Express.js to accept events from Twitch EventSub. In part two, we listened for our event and triggered a notification using Courier. Now, in part three, we're going to use Courier’s List API to send multiple notifications when our event is triggered.

Follow along with the series:

Part one: How to handle real-time events from Twitch
Part two: How to send notifications when your stream goes Twitch live
Part three (this post): How to create and notify a list of subscribers using Courier

Need help getting started with sending notifications about your Twitch stream? Join our community on Discord – we’re happy to chat!

How to create and notify a list of subscribers using Courier

In this tutorial, I'll show you how to extend the Node.js and Express app that we updated in part two to send notifications to more than one destination using Courier’s Lists API. We'll update the sendOnline function to use a List send. I'll also demo sending to a Discord channel.

Prerequisites

To complete this tutorial, you'll need a few things:

Node.js and Express.js app from part two
Courier account – it’s free to sign up and includes 10,000 notifications per month
Discord Bot
Twitch CLI (v0.5.0+)

If you’re using the Node.js and Express.js app we created in part one, it should either be deployed somewhere publicly accessible that supports HTTPS and port 443, or be running locally using ngrok.

We'll need an existing Discord Bot that Courier can use to send your notifications. If you don't have one, check out our Discord Integration Guide to get started.

You'll also need your Courier Auth Token for the following steps. You can find your Courier Auth Token in Settings > API Keys in your Courier account. Use the Published Production Key.

Step one: Update your code to send to a list

To send to multiple recipients, we'll need to refactor the sendOnline function to use a list send instead of the regular send. We'll also need to create a list of recipients. To continue sending the SMS notification we created in part two, we'll create a stored profile for the recipient and subscribe them to our list.

Create a List in Courier

To create our list, we'll use the Courier Lists API. Our list will need a list id and a name. For this tutorial, we'll create a list with an id of "twitch.stream.online" and a name of "Twitch Stream Online." You can learn more about using list id patterns in our Help Center.

Let's create our list by executing the following cURL command in your terminal, replacing COURIER_AUTH_TOKEN with your auth token:

curl --request PUT \
  --url https://api.courier.com/lists/twitch.stream.online \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer COURIER_AUTH_TOKEN' \
  --header 'Content-Type: application/json' \
  --data '{"name":"Twitch Stream Online"}'

Your new list should now be visible in the data tab in your Courier Account.

Add a new subscriber to the List

Now that we have a list, let's subscribe the recipient we used in part two to it. To do this, we'll first need to use the Profiles API to store the recipient's profile information in Courier. Then, we'll make a call to the List API to subscribe them to the list.

We'll use the recipient id and profile information from the existing send command. Execute the following cURL command in your terminal using your values:

curl --request POST \
  --url https://api.courier.com/profiles/AYDRIAN10036 \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer COURIER_AUTH_TOKEN' \
  --header 'Content-Type: application/json' \
  --data '{"profile":{"phone_number":"+12025550140"}}'

Now that we have the profile stored, we can use the recipient id and subscribe it to our list. Execute the following cURl command in your terminal replacing AYDRIAN10036 with your recipient id:

curl --request PUT \
  --url https://api.courier.com/lists/twitch.stream.online/subscriptions/AYDRIAN10036 \
  --header 'Authorization: Bearer COURIER_AUTH_TOKEN'

Repeat this process to add more subscribers to the list. When you’re ready, let's update the code to send to our new list.

Swap out the Courier Send

Previously, we told Courier to send to a single recipient. In order to send to the list we just created, we’ll need to use a List send instead.

In your index.js file, replace the following in the sendOnline function:

const { messageId } = await courier.send({
  eventId: "TWITCH_ONLINE",
  recipient: "AYDRIAN10036",
  profile: {
    phone_number: "+12025550140"
  },
  data: { stream, game }
});

With the following:

const { messageId } = await courier.send({
  event: "TWITCH_ONLINE",
  list: "twitch.stream.online",
  data: { stream, game }
});

Now if you were to run this code, it would still deliver the notification via SMS.

Step two: Add Discord to your notification in Courier

Now that we can send notifications to multiple recipients with Lists, let's expand the available destinations. Recently, Discord committed to fully supporting online communities, making it a top choice for notifying people about our Twitch stream. Let's add the ability to have Courier post to a channel using a Discord Bot.

Configure the Discord integration in Courier

Let's start by configuring the Discord integration. This will require you to enter the bot token for the bot that Courier will send as.

Design the Discord notification

Now we can update our existing Twitch Online Alert notification. We'll add Discord by clicking “Add Channel” and selecting Discord from the list of configured integrations.

We can now select Discord under Channels to the left and start designing our notification. Because we have already created our SMS notification, we can reuse those content blocks for Discord. Simply drag the blocks in the Library section to our Discord notification.

We now have a message that matches our SMS. Feel free to add more content blocks to your Discord notifications. When you’re finished, click “Publish Changes” in the upper righthand corner.

If you'd like, you can preview the generated Discord markdown using the Preview tab. You can use the test event we created in part two.

Step three: Subscribe a Discord channel to the List

Your notification is now ready to start sending to Discord. The last step is to identify the Discord channel that you want to post your notification in and add it as a recipient to our list. Similar to how we added a recipient for our SMS notification, we'll first create a profile in Courier and then subscribe it to the list.

We'll need the channel id of the channel we want to send to. An easy way to retrieve that is to turn on Developer Mode in Discord. You can go to User Settings > Appearance and scroll to Advanced at the bottom and toggle Developer Mode to on. This will allow you to right click on a channel and copy the id.

I'm going to use the #show-and-tell channel in Courier’s Discord server, which you’re welcome to join. For the recipient id, I'm going to use DISCORD_COURIER_SHOW_AND_TELL. It's a little long but descriptive.

Execute the following cURL command to create a profile for the channel in Courier:

curl --request POST \
  --url https://api.courier.com/profiles/DISCORD_COURIER_SHOW_AND_TELL \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer COURIER_AUTH_TOKEN' \
  --header 'Content-Type: application/json' \
  --data '{"profile":{"discord":{"channel_id":"801886566419136592"}}}'

Now we can execute the following cURL command to subscribe it to our list:

curl --request PUT \
  --url https://api.courier.com/lists/twitch.stream.online/subscriptions/DISCORD_COURIER_SHOW_AND_TELL \
  --header 'Authorization: Bearer COURIER_AUTH_TOKEN'

We can test our application using the Twitch CLI. Run the following command with the needed substitutions:

twitch event trigger streamup --to-user YOUR_BROADCASTER_ID -F https://EXTERNAL_URL/webhook/callback -s YOUR_SECRET

This command will trigger an example stream.online event using your broadcaster id. You should see the event in the Courier Logs. You should receive an SMS message and that your Discord Bot has posted the following:

Putting it all together: Full application code

With the update to the sendOnline function, your finished application should look like the following.

require("dotenv").config();
const express = require("express");
const crypto = require("crypto");
const { CourierClient } = require("@trycourier/courier");
const app = express();
const port = process.env.PORT || 3000;
const twitchSigningSecret = process.env.TWITCH_SIGNING_SECRET;
const courier = CourierClient();
const { ApiClient } = require("twitch");
const { ClientCredentialsAuthProvider } = require("twitch-auth");
const authProvider = new ClientCredentialsAuthProvider(
  process.env.TWITCH_CLIENT_ID,
  process.env.TWITCH_CLIENT_SECRET
);
const twitch = new ApiClient({ authProvider });

app.get("/", (req, res) => {
  res.send("Hello World!");
});

const verifyTwitchSignature = (req, res, buf, encoding) => {
  const messageId = req.header("Twitch-Eventsub-Message-Id");
  const timestamp = req.header("Twitch-Eventsub-Message-Timestamp");
  const messageSignature = req.header("Twitch-Eventsub-Message-Signature");
  const time = Math.floor(new Date().getTime() / 1000);
  console.log(`Message ${messageId} Signature: `, messageSignature);

  if (Math.abs(time - timestamp) > 600) {
    // needs to be < 10 minutes
    console.log(
      `Verification Failed: timestamp > 10 minutes. Message Id: ${messageId}.`
    );
    throw new Error("Ignore this request.");
  }

  if (!twitchSigningSecret) {
    console.log(`Twitch signing secret is empty.`);
    throw new Error("Twitch signing secret is empty.");
  }

  const computedSignature =
    "sha256=" +
    crypto
      .createHmac("sha256", twitchSigningSecret)
      .update(messageId + timestamp + buf)
      .digest("hex");
  console.log(`Message ${messageId} Computed Signature: `, computedSignature);

  if (messageSignature !== computedSignature) {
    throw new Error("Invalid signature.");
  } else {
    console.log("Verification successful");
  }
};

const sendOnline = async (event) => {
  const stream = await twitch.helix.streams.getStreamByUserId(
    event.broadcaster_user_id
  );
  const game = await stream.getGame();

  const { messageId } = await courier.send({
    event: "TWITCH_ONLINE",
    list: "twitch.stream.online",
    data: { stream, game }
  });
  console.log(
    `Online notification for ${event.broadcaster_user_name} sent. Message ID: ${messageId}.`
  );
};

app.use(express.json({ verify: verifyTwitchSignature }));

app.post("/webhooks/callback", async (req, res) => {
  const messageType = req.header("Twitch-Eventsub-Message-Type");
  if (messageType === "webhook_callback_verification") {
    console.log("Verifying Webhook");
    return res.status(200).send(req.body.challenge);
  }

  const { type } = req.body.subscription;
  const { event } = req.body;

  console.log(
    `Receiving ${type} request for ${event.broadcaster_user_name}: `,
    event
  );

  if (type === "stream.online") {
    try {
      sendOnline(event);
    } catch (ex) {
      console.log(
        `An error occurred sending the Online notification for ${event.broadcaster_user_name}: `,
        ex
      );
    }
  }

  res.status(200).end();
});

const listener = app.listen(port, () => {
  console.log("Your app is listening on port " + listener.address().port);
});

Our application will process stream.online events and pass them to Courier along with additional stream data. Courier will then create SMS or Discord notifications based on the profiles in your list of subscribers.

So, what's next?

You now have an application that will send notifications to a list of subscribers, via SMS and Discord, when you start your Twitch stream. I encourage you to explore adding more subscribers to your list and adding even more destinations like Slack and Facebook Messenger. Join our Discord community and let me know where you go from here!

-Aydrian

Twitch notifications (part two): How to send notifications when your Twitch stream goes live

Aydrian — Tue, 02 Feb 2021 13:33:47 +0000

In this series, I explain how to use Twitch EventSub and Courier to automatically send notifications to multiple destinations – Slack, Discord, and more – when your Twitch stream goes live.

In part one, we built a Node.js app using Express.js to accept events from Twitch EventSub. Now, in part two, we’re going to listen for our event and trigger a notification using Courier.

Follow along with the series:

Part one: How to handle real-time events from Twitch
Part two (this post): How to send notifications when your stream goes Twitch live
Part three (coming soon): How to create and notify a list of subscribers using Courier

Need help getting started with sending notifications about your Twitch stream? Join our community on Discord – we’re happy to chat!

How to send notifications when your Twitch stream goes live

In this tutorial, I’ll show you how to take the Node.js and Express app that we built in part one and use it to listen for our events. From there, we’ll create and trigger a notification in Courier. I’ll demo sending an SMS notification with Twilio, but you can use Courier to send notifications to any channel, including popular chat apps like Discord and Facebook Messenger.

Prerequisites

To complete this tutorial, you'll need a few things:

Node.js and Express.js app from part one
Twitch Developer account
Twitch CLI (v0.5.0+)
Courier account – it’s free to sign up and includes 10,000 notifications per month
Twilio account configured for SMS

We'll use a new or existing Twitch application from your Developer Console along with the Twitch CLI to subscribe to the stream.online event and point it to the /webhooks/callback route on our server. Then, we'll update our server code to capture that event and send it to Courier to create and send the notification.

Keep in mind that there could be around a 45 second delay before Twitch triggers the event from the time your stream goes online.

Step one: Subscribe to the Twitch EventSub online event

To begin receiving requests from Twitch EventSub, we first need to create a subscription. We'll use the Twitch CLI to create a Stream Online subscription type and give it the callback URL from our Node.js application. It's important that the application is running and publicly available because EventSub will attempt to validate the callback URL when creating the subscription.

Configure the Twitch CLI

First, we need to configure the Twitch CLI using a new or existing Twitch application. If you have already configured your Twitch CLI, you can skip this step.

Navigate to the Twitch Developer Console and create a new by clicking “Register Your Application” or open an existing application. Add http://localhost:3000 as an OAuth Redirect URL.

Take note of the Client ID, as we'll be using this shortly. You'll also need to generate a new Client Secret by clicking on the “New Secret” button. Be sure to save this somewhere safe because it won't be shown again. If you have an existing Client Secret for this application, you can use it. Generating a new secret will invalidate any existing secret.

Now, let's use these values to configure the Twitch CLI. In a terminal, run the following command:

twitch configure

You will be prompted to enter your Client ID and Secret. To fetch an access token, run the following command:

twitch token

You're now ready to start making Twitch API calls using the Twitch CLI.

Subscribe to the Stream Online Event

To create our Stream Online subscription, we'll use the Twitch CLI to POST to the EventSub Subscriptions endpoint. You'll need to provide the full URL for your webhook callback, the secret value set in the TWITCH_SIGNING_SECRET environment variable of your Node.js application, and your Twitch broadcaster user ID.

To find your broadcaster user ID, run the following command replacing trycourier with your Twitch login ID:

twitch api get users -q login=trycourier

This will output your Twitch user object in JSON format. Your broadcaster user ID will be the ID.

Now let's create the subscription. Run the following command with the needed substitutions:

twitch api post eventsub/subscriptions -b '{
    "type": "stream.online",
    "version": "1",
    "condition": {
        "broadcaster_user_id": "YOUR_BROADCASTER_ID"
    },
    "transport": {
        "method": "webhook",
        "callback": "https://EXTERNAL_URL/webhook/callback",
        "secret": "YOUR_SECRET"
    }
}'

You should see "Verification successful" in the console of your running application. Every time you go online, your application will now receive a POST with a payload similar to the following:

{
    "subscription": {
        "id": "5d179ae7-d744-45a4-a259-5129634dd788",
        "type": "stream.online",
        "version": "1",
        "condition": {
            "broadcaster_user_id": "493127514"
        },
         "transport": {
            "method": "webhook",
            "callback": "https://15a1265bdd3c.ngrok.io/webhook/callback"
        },
        "created_at": "2021-01-26T17:15:17Z"
    },
    "event": {
        "id": "9001",
        "broadcaster_user_id": "493127514",
        "broadcaster_user_login": "trycourier",
        "broadcaster_user_name": "TryCourier",
        "type": "stream.online"
    }
}

Now we can update our application to accept and process this event.

Step two: Capture the event and send it to Courier

Now that we’ve created our Stream Online subscription, the next step is to send it to Courier, which we’ll use to create and deliver notifications about our Twitch stream. To do this, we need to add a call to Courier's Send API when a stream.online event comes in. We'll use the Courier Node.js SDK to do this. We'll also use the Twitch.js library to query the Twitch API to grab more details about the stream that we can send to Courier.

First, let's add these npm packages and configure the necessary environment variables.

Gather your environment variables

We've reached a point where we are using enough environment variables that we should use a better method of loading them. Let's create a .env file and use the dotenv package to load them when the application starts.

Create a .env file with the following:

TWITCH_SIGNING_SECRET=purplemonkeydishwasher
TWITCH_CLIENT_ID=your-twitch-client-id
TWITCH_CLIENT_SECRET=your-twitch-client-id
COURIER_AUTH_TOKEN=your-courier-auth-token

Use the Twitch values from step one of our tutorial. You can find your Courier Auth Token in Settings > API Keys in your Courier account. Use the Published Production Key.

Now let's install the dotenv package, along with the other packages mentioned above:

npm install dotenv @trycourier/courier twitch

And add the following line to the top of your index.js:

require("dotenv").config();

Now when you run your application, these values will be loaded and ready for your application to use.

Process the `stream.online` event

Let's continue updating our application by running a function when the type of the event is stream.online.

Just below the console.log in the /webhooks/callback handler, add the following:

if (type === "stream.online") {
  try {
    sendOnline(event);
  } catch (ex) {
    console.log(
      `An error occurred sending the Online notification for ${event.broadcaster_user_name}: `,
      ex
    );
  }
}

Next, let's create sendOnline as an async function. This function will handle grabbing any additional information about the Twitch stream and sending it to Courier.

Add the following to the top of index.js with the rest of the require statements:

const { CourierClient } = require("@trycourier/courier");
const courier = CourierClient();
const { ApiClient } = require("twitch");
const { ClientCredentialsAuthProvider } = require("twitch-auth");
const authProvider = new ClientCredentialsAuthProvider(
  process.env.TWITCH_CLIENT_ID,
  process.env.TWITCH_CLIENT_SECRET
);
const twitch = new ApiClient({ authProvider });

This will create the Courier and Twitch clients we'll use in the sendOnline function. Add the following function to your application:

const sendOnline = async event => {
  const stream = await twitch.helix.streams.getStreamByUserId(
    event.broadcaster_user_id
  );
  const game = await stream.getGame();

  const { messageId } = await courier.send({
    eventId: "TWITCH_ONLINE",
    recipientId: "AYDRIAN10036",
    profile: {
      phone_number: "+12025550140"
    },
    data: {stream, game}
  });
  console.log(
    `Online notification for ${event.broadcaster_user_name} sent. Message ID: ${messageId}.`
  );
};

This function will use the Twitch client to grab information about the stream such as the title and game info and then pass it to the call to Courier's Send API so it can be used in the creation of your notification. You'll also want to update the recipientId to a unique string – I used my name and zip in all caps without spaces: AYDRIAN10036 – and the phone_number with your phone number. You’ll need both of these in order to receive the notification we create in Courier.
The next time you go online, the stream.online event will flow into Courier. Next, we'll use the stream information to create a notification using Courier's template builder.

Step three: Create your notification in Courier

For this tutorial, we'll be creating a text notification for our Twitch stream. We'll explore other notification channels in the third part of this series.

Configure Twilio as your SMS provider

Let's start by configuring the Twilio integration in Courier. This will require you to enter details about your Twilio account. Check out our Getting Started with Twilio guide for more details.

Design your SMS notification

Now it's time to design the notification in Courier. Navigate to the Notification Designer and select “Create Notification.” Click “Untitled Notification” on the top left to give your notification a descriptive name – in this case, I’ve named mine "Twitch Online Alert.”

Now let's add SMS as a channel for our notification by selecting SMS and choosing Twilio from the dropdown. We can now select SMS under Channels to the left and start designing our notification.

We'll design a simple SMS notification. First, we'll use a text block – click the “T” on the toolbar – and add the following text: "{stream._data.user_name} is playing {game._data.name} on Twitch." Next, we'll add another text block with the following text: "{stream._data.title}.” And we'll add one more text block with the following text: "https://twitch.tv/{stream._data.user_name}". We’re personalizing the SMS using the stream information, which we passed to the notification in the data object as part of calling the Courier API.

This is enough for now, but feel free to add more content blocks and continue designing the SMS message. When you’re finished, click “Publish Changes” in the upper righthand corner.

If you’d like, you can preview the email using the Preview tab and ensure your variables are templated properly. You'll be prompted to Create a Test Event and then you'll want to update the JSON object with the following example, replacing the phone_number with your own:

{
  "data": {
    "stream": {
      "_data": {
        "id": "40078987165",
        "user_id": "493127514",
        "user_name": "trycourier",
        "game_id": "417752",
        "game_name": "Talk Shows & Podcasts",
        "type": "live",
        "title": "Courier Live: Twitch EventSub and Courier",
        "viewer_count": 0,
        "started_at": "2021-01-05T19:54:35Z",
        "language": "en",
        "thumbnail_url": "https://static-cdn.jtvnw.net/previews-ttv/live_user_trycourier-{width}x{height}.jpg",
        "tag_ids": null
      }
    },
    "game": {
      "_data": {
        "id": "417752",
        "name": "Talk Shows & Podcasts",
        "box_art_url": "https://static-cdn.jtvnw.net/ttv-boxart/Talk%20Shows%20&%20Podcasts-{width}x{height}.jpg"
      }
    }
  },
  "profile": {
    "phone_number": "+12025550140"
  }
}

Once you save your test event, you should see the name variable populate in the Preview tab with whatever value you’ve set.

Map your notification to the event specified in your Send call

The last thing we want to do is to map the event we specified earlier in the Courier Send call to this notification. Next to the notification name, click the gear icon to launch the Notification Settings. Select Events from the left menu and enter “TWITCH_ONLINE” in the Events box.

Close the dialog and your notification is ready to send. If you don't want to wait for the next time you go online, you can test your notification using the Send Tab.

We can now test our application using the Twitch CLI. Run the following command with the needed substitutions:

twitch event trigger streamup --to-user YOUR_BROADCASTER_ID -F https://EXTERNAL_URL/webhook/callback -s YOUR_SECRET

This command will trigger an example stream.online event using your broadcaster id. You should see the event in the Courier Logs and receive an SMS message.

Putting it all together: Full application code

With all the new updates, your finished application should look like the following.

require("dotenv").config();
const express = require("express");
const crypto = require("crypto");
const { CourierClient } = require("@trycourier/courier");
const app = express();
const port = process.env.PORT || 3000;
const twitchSigningSecret = process.env.TWITCH_SIGNING_SECRET;
const courier = CourierClient();
const { ApiClient } = require("twitch");
const { ClientCredentialsAuthProvider } = require("twitch-auth");
const authProvider = new ClientCredentialsAuthProvider(
  process.env.TWITCH_CLIENT_ID,
  process.env.TWITCH_CLIENT_SECRET
);
const twitch = new ApiClient({ authProvider });

app.get("/", (req, res) => {
  res.send("Hello World!");
});

const verifyTwitchSignature = (req, res, buf, encoding) => {
  const messageId = req.header("Twitch-Eventsub-Message-Id");
  const timestamp = req.header("Twitch-Eventsub-Message-Timestamp");
  const messageSignature = req.header("Twitch-Eventsub-Message-Signature");
  const time = Math.floor(new Date().getTime() / 1000);
  console.log(`Message ${messageId} Signature: `, messageSignature);

  if (Math.abs(time - timestamp) > 600) {
    // needs to be < 10 minutes
    console.log(
      `Verification Failed: timestamp > 10 minutes. Message Id: ${messageId}.`
    );
    throw new Error("Ignore this request.");
  }

  if (!twitchSigningSecret) {
    console.log(`Twitch signing secret is empty.`);
    throw new Error("Twitch signing secret is empty.");
  }

  const computedSignature =
    "sha256=" +
    crypto
      .createHmac("sha256", twitchSigningSecret)
      .update(messageId + timestamp + buf)
      .digest("hex");
  console.log(`Message ${messageId} Computed Signature: `, computedSignature);

  if (messageSignature !== computedSignature) {
    throw new Error("Invalid signature.");
  } else {
    console.log("Verification successful");
  }
};

const sendOnline = async (event) => {
  const stream = await twitch.helix.streams.getStreamByUserId(
    event.broadcaster_user_id
  );
  const game = await stream.getGame();

  const { messageId } = await courier.send({
    eventId: "TWITCH_ONLINE",
    recipient: "AYDRIAN10036",
    profile: {
      phone_number: "+12025550140"
    },
    data: { stream, game }
  });
  console.log(
    `Online notification for ${event.broadcaster_user_name} sent. Message ID: ${messageId}.`
  );
};

app.use(express.json({ verify: verifyTwitchSignature }));

app.post("/webhooks/callback", async (req, res) => {
  const messageType = req.header("Twitch-Eventsub-Message-Type");
  if (messageType === "webhook_callback_verification") {
    console.log("Verifying Webhook");
    return res.status(200).send(req.body.challenge);
  }

  const { type } = req.body.subscription;
  const { event } = req.body;

  console.log(
    `Receiving ${type} request for ${event.broadcaster_user_name}: `,
    event
  );

  if (type === "stream.online") {
    try {
      sendOnline(event);
    } catch (ex) {
      console.log(
        `An error occurred sending the Online notification for ${event.broadcaster_user_name}: `,
        ex
      );
    }
  }

  res.status(200).end();
});

const listener = app.listen(port, () => {
  console.log("Your app is listening on port " + listener.address().port);
});

Our application will now process stream.online events and pass them to Courier along with additional stream data. Courier will then create an SMS notification and send it.

So, what's next?

In the next post, we'll explore other types of notifications and how to use Courier to send to them all at the same time. In the meantime, take a look at the different Courier Integrations and see if you can update your application to support a new one. Join our Discord community and let me know which one you're most interested in!

-Aydrian

Twitch notifications (part one): How to handle real-time events from Twitch

Aydrian — Tue, 26 Jan 2021 13:12:59 +0000

Over the last few years, Twitch has become the streaming platform for gaming, esports, live coding, DJ-ing, and more. If you’re a streamer, whether for work or for fun, you know that one of the biggest challenges is building your audience and attracting viewers to your Twitch channel when you go live.

Unfortunately, the options for sending notifications in Twitch are pretty limited. When you go live, Twitch will automatically send your followers an email, push, or in-app notification. But this isn’t very helpful for acquiring new viewers or engaging your community outside of Twitch.

In this series, I'll show you how to use Twitch EventSub and Courier to automatically send notifications to many destinations – Discord, Slack, Facebook Messenger, and more – when your stream begins.

Part one (this post): We’ll create a small Node.js and Express app to accept webhooks from Twitch EventSub.
Part two (coming soon): We’ll subscribe to the stream.online event and process the request using Courier. Then, we’ll use Courier to create and design our notifications.
Part three (coming soon): Finally, we'll create a list of subscribers and use Courier to notify the entire list across multiple destinations with one API call.

Have questions about sending notifications using Twitch EventSub and Courier? Join our new community on Discord – we're happy to help!

How to handle real-time events from Twitch

During last year's Twitch Developer Day, Twitch introduced EventSub as a single product to handle real-time events. EventSub is a transport-neutral solution that will eventually replace their existing PubSub and Webhook APIs. Today, EventSub only supports webhooks.

Let's start by creating a Node.js application and use Express to expose a POST route that Twitch EventSub can communicate with.

Prerequisites

To complete this tutorial, you'll need a couple things:

A Node.js v14.x dev environment
The Twitch CLI (for testing)

We'll be creating a Node.js application that needs to be accessible externally. If you are working locally, you can use ngrok to expose your local endpoint. Alternatively, you can use a tool like Glitch to build and host your application.

Creating a basic Express Application

We'll start by creating an Express application with minimal features. First, create a new folder and initialize it with a package.json file.

mkdir eventsub-handler && eventsub-handler
npm init --yes

Now we are able to install the Express package.

npm install express

Let's use express to create a simple HTTP server. Create an index.js file and add the following:

const express = require("express");
const app = express();
const port = process.env.PORT || 3000;

app.get("/", (req, res) => {
  res.send("Hello World!");
});

const listener = app.listen(port, () => {
  console.log("Your app is listening on port " + listener.address().port);
});

Let's run our application by executing node index.js in the terminal. If you open http://localhost:3000 in your browser, you should see "Hello World!"

Congrats! You now have a working (albeit minimal) Express server. Next, we'll add the ability to receive a POST request from Twitch.

Handling the Twitch POST request

In order to accept real-time events from Twitch, we'll need to create a callback URL. We can do this by creating a new POST route. In the index.js file above where the listener is created, add the following lines of code:

app.use(express.json());

app.post("/webhooks/callback", async (req, res) => {
  const { type } = req.body.subscription;
  const { event } = req.body;

  console.log(
    `Receiving ${type} request for ${event.broadcaster_user_name}: `,
    event
  );

  res.status(200).end();
});

First, we are telling our Express application to use the express.json() middleware to parse any incoming JSON payloads. Then, we added a callback route that will log the request and return a 200 status. Twitch expects this 2XX response to confirm you've received the request. If it doesn't receive a response in a couple seconds, it will retry the request.

Let's test this route using the Twitch CLI. Restart your application and use the following command to trigger a test subscribe event.

twitch event trigger subscribe -F http://localhost:3000/webhooks/callback

In the terminal running your application, you should see the event JSON for a channel.subscribe event. Next, we'll want to handle callback verification.

Handling callback verification

When you subscribe to an event, EventSub will send an initial request to the callback URL you specified. It expects a challenge response to verify you own the callback URL. We can handle this by checking the value of the Twitch-Eventsub-Message-Type header and respond with the challenge value provided in the request payload.

Update the callback code to the following:

app.post("/webhooks/callback", async (req, res) => {
  const messageType = req.header("Twitch-Eventsub-Message-Type");
  if (messageType === "webhook_callback_verification") {
    console.log("Verifying Webhook");
    return res.status(200).send(req.body.challenge);
  }

  const { type } = req.body.subscription;
  const { event } = req.body;

  console.log(
    `Receiving ${type} request for ${event.broadcaster_user_name}: `,
    event
  );

  res.status(200).end();
});

Let's test this using the Twitch CLI. Restart your application and run the following CLI command:

twitch event verify-subscription subscribe -F http://localhost:3000/webhooks/callback

This command will send a fake "subscription" EventSub subscription and validate if the endpoint responded with a valid status code and response.

Verifying the request

When accepting webhooks, it's good practice to verify that it came from the expected sender. We can do this by using the signature provided in the Twitch-Eventsub-Message-Signature header. We can create our own signature using the message ID, timestamp, and secret provided when subscribing to the event and compare it to the signature provided.

Let's update our use of the express.json() middleware to include a verify function. Add the following lines to the top of your index.js file:

const crypto = require("crypto");
const twitchSigningSecret = process.env.TWITCH_SIGNING_SECRET;

And replace the app.use(express.json()); line with the following lines of code:

const verifyTwitchSignature = (req, res, buf, encoding) => {
  const messageId = req.header("Twitch-Eventsub-Message-Id");
  const timestamp = req.header("Twitch-Eventsub-Message-Timestamp");
  const messageSignature = req.header("Twitch-Eventsub-Message-Signature");
  const time = Math.floor(new Date().getTime() / 1000);
  console.log(`Message ${messageId} Signature: `, messageSignature);

  if (Math.abs(time - timestamp) > 600) {
    // needs to be < 10 minutes
    console.log(`Verification Failed: timestamp > 10 minutes. Message Id: ${messageId}.`);
    throw new Error("Ignore this request.");
  }

  if (!twitchSigningSecret) {
    console.log(`Twitch signing secret is empty.`);
    throw new Error("Twitch signing secret is empty.");
  }

  const computedSignature =
    "sha256=" +
    crypto
      .createHmac("sha256", twitchSigningSecret)
      .update(messageId + timestamp + buf)
      .digest("hex");
  console.log(`Message ${messageId} Computed Signature: `, computedSignature);

  if (messageSignature !== computedSignature) {
    throw new Error("Invalid signature.");
  } else {
    console.log("Verification successful");
  }
};

app.use(express.json({ verify: verifyTwitchSignature }));

We just added a function to handle verifying the signature using the information from the request headers and used the crypto library to generate our own signature to which to compare it. This process used the signing secret that I'm storing in an environment variable because, well, your signing secret should stay secret.

Let's test that the signature validation is working using the Twitch CLI. You'll want to restart your app with the following command that includes the environment variable.

TWITCH_SIGNING_SECRET=purplemonkeydishwasher node index.js

Then in another terminal, run the following CLI command:

twitch event trigger subscribe -F http://localhost:3000/webhooks/callback -s purplemonkeydishwasher

You should now see the provided and computed signatures and that verification was successful.

Putting it all together: Full application code

Your finished application code should look like the following.

const express = require("express");
const crypto = require("crypto");
const app = express();
const port = process.env.PORT || 3000;
const twitchSigningSecret = process.env.TWITCH_SIGNING_SECRET;

app.get("/", (req, res) => {
  res.send("Hello World!");
});

const verifyTwitchSignature = (req, res, buf, encoding) => {
  const messageId = req.header("Twitch-Eventsub-Message-Id");
  const timestamp = req.header("Twitch-Eventsub-Message-Timestamp");
  const messageSignature = req.header("Twitch-Eventsub-Message-Signature");
  const time = Math.floor(new Date().getTime() / 1000);
  console.log(`Message ${messageId} Signature: `, messageSignature);

  if (Math.abs(time - timestamp) > 600) {
    // needs to be < 10 minutes
    console.log(
      `Verification Failed: timestamp > 10 minutes. Message Id: ${messageId}.`
    );
    throw new Error("Ignore this request.");
  }

  if (!twitchSigningSecret) {
    console.log(`Twitch signing secret is empty.`);
    throw new Error("Twitch signing secret is empty.");
  }

  const computedSignature =
    "sha256=" +
    crypto
      .createHmac("sha256", twitchSigningSecret)
      .update(messageId + timestamp + buf)
      .digest("hex");
  console.log(`Message ${messageId} Computed Signature: `, computedSignature);

  if (messageSignature !== computedSignature) {
    throw new Error("Invalid signature.");
  } else {
    console.log("Verification successful");
  }
};

app.use(express.json({ verify: verifyTwitchSignature }));

app.post("/webhooks/callback", async (req, res) => {
  const messageType = req.header("Twitch-Eventsub-Message-Type");
  if (messageType === "webhook_callback_verification") {
    console.log("Verifying Webhook");
    return res.status(200).send(req.body.challenge);
  }

  const { type } = req.body.subscription;
  const { event } = req.body;

  console.log(
    `Receiving ${type} request for ${event.broadcaster_user_name}: `,
    event
  );

  res.status(200).end();
});

const listener = app.listen(port, () => {
  console.log("Your app is listening on port " + listener.address().port);
});

We now have a Node.js and Express application that can receive real-time events from Twitch using EventSub. We've tested it locally using the Twitch CLI but, remember, before you can start using it with Twitch, you'll need to make sure the route uses HTTPS and port 443 and is publicly available. If you want to continue running it locally, look into using ngrok.

So, what’s next?

In the next post, we'll walk through creating a subscription for the stream.online event and use Courier to design and send our notifications. In the meantime, feel free to create subscriptions to any of the many supported events and try out your application.

-Aydrian

Tutorial: How to send emails with attachments using Amazon S3

Aydrian — Thu, 03 Dec 2020 17:26:06 +0000

Nearly all software products rely on email to communicate with their users. In many cases, it’s the primary channel for sending transactional notifications, or notifications that are automatically triggered by a user’s behavior in the application. These transactional emails frequently include attachments, such as an invoice, order confirmation, or other statement.

As a developer, it's up to you to generate or retrieve the file and then attach it to the appropriate email using one of the many email provider APIs. Depending on your email provider, this can be a difficult task – Amazon SES, which we’ll use as an example in this tutorial, doesn’t make it easy if you’re relying on a direct integration – and, for many email providers, the documentation can often be hard to follow.

Let's take a look at how we can accomplish this using a couple popular offerings from Amazon Web Services (AWS). We'll retrieve a file from an Amazon S3 bucket and then attach it to an email sent using Amazon Simple Email Service (SES), which we’ll integrate with Courier for template management and delivery.

Prerequisites

To complete this tutorial, you’ll need a few things:

An AWS account with an S3 bucket created.
An SES domain that’s been verified.
A Courier account – it’s free to sign up and includes 10,000 notifications per month.

We'll use Courier to create the email and send it through AWS SES with an attachment stored in AWS S3. Using Courier allows us to manage our email template outside the source code and take advantage of additional functionality like retrying failed sends and tracking delivery and user engagement.

You'll need a Node.js v12+ environment to run the code.

1. Build your email notification in Courier

Configure Amazon SES as your email provider

Once you've created your Courier account, we'll start by configuring Amazon SES as our email provider. This will allow us to use Courier’s API to call Amazon SES and deliver the email we’re about to compose, plus our attachment.

First, navigate to Integrations and select AWS SES from the Integrations Catalog. We'll need the access key ID and secret access key from an IAM user with SES access. You can learn more about how to get them using the AWS Developer Guide.

Next we'll need a “From Address” for our email. This can be any email address that uses the domain you have configured. Lastly, select the region your SES account is configured for. You can now click Install and we’re ready to create our email notification.

Design your email notification

Navigate to the Notification Designer and select Create Notification. Click “Untitled Notification” on the top left to give your notification a descriptive name – in this case, I’ve named mine "New Invoice.”

Now let's add email as a channel for our notification by selecting Email and choosing AWS SES from the dropdown. We can now add Email under Channels to the left and start designing our notification.

We’ll design a simple email notification. First, let's update the subject line to "New Invoice" by clicking on New Subject and updating the text. Next, we'll use a text block – click the “T” on the toolbar – to add a short greeting. Feel free to copy-paste the following text: "Hello {name}, your invoice is attached below." We’re personalizing the email with a “name” variable, which we'll pass to the notification below in the data object as part of calling the Courier API.

This is enough for now, but feel free to add more content blocks and continue designing the email. When you’re finished, click Publish Changes in the upper righthand corner.

If you’d like, you can preview the email using the Preview tab and ensure your variables are templated properly. You'll be prompted to Create a Test Event and then you'll want to add the name property to the data JSON object. Once you save your test event, you should see the name variable populate in the Preview tab with whatever value you’ve set.

Retrieve your Notification ID

The last thing we need to do before moving onto code is retrieve the Notification ID. We'll need this to send the right notification when we call the Courier API later.. Next to the notification name, click the gear icon to launch the Notification Settings. Copy the Notification ID value and save it to use in the code below.

2. Code the send

Now that we have a notification setup in Courier, we'll use the Courier Node.js SDK to send it. We'll start by creating a new npm project.

> mkdir courier-send && cd courier-send
> npm init --yes

Now we can add a couple packages that will assist us in calling the Courier API. We'll install the Courier Node.js package and since we'll be using environment variables, we'll go ahead and install the dotenv package.

> npm install @trycourier/courier dotenv

To handle authentication with the Courier API, we'll store our Courier Auth Token in the environment variable COURIER_AUTH_TOKEN using a .env file. Be sure not to check this file into source control. You can find your Courier Auth Token in Settings > API Keys in your Courier account. Let's create the .env file and populate it with your auth token.

> echo "COURIER_AUTH_TOKEN=YOUR_AUTH_TOKEN" > .env

Now we can create an index file and open it in our favorite editor. I'll be using VS Code.

> touch index.js && code .

Paste in the following code:

require("dotenv").config();
const { CourierClient } = require("@trycourier/courier");

const courier = CourierClient();

const main = async () => {

};

main();

This code will load the environment variables from our .env file and create a Courier client using our auth token. It also sets up an async main function so we can use async/wait. Now let's add the Courier send call. In the main function, add the following code:

const { messageId } = await courier.send({
  eventId: "YOUR_NOTIFICATION_ID",
  recipientId: "YOUR_RECIPIENT_ID",
  profile: {
    email: "YOUR_EMAIL"
  }
  data: {
    name: "Aydrian"
  }
});
  console.log("Sent notification: ", messageId);

This code will send the notification specified by the eventId to the specified recipient. Make sure you replace the eventId value with the Notification ID you copied earlier. You'll also want to update the recipientId to a unique string (For my example, I use my name and zip in all caps without spaces: AYDRIAN10036). You'll also want to update email with your email address. Now if you were to run this, you would receive the email without an attachment. Let's tackle that next.

Add your email attachment

To add the attachment, we'll need to first retrieve it from our S3 Bucket and convert it to a base64 string. Then we'll be able to add it to the send call above using a provider override. Each provider has its own override configuration and you can see them all in the Courier Integration Guides. We'll be using the attachment override for the AWS SES integration.

Let's start by adding the AWS SES SDK:

> npm install @aws-sdk/client-s3

Next we'll configure the environment variables needed for authentication. For this you'll need to get your AWS credentials. They consist of an access key ID and a secret access key. You can learn more about how to get them on the AWS Developer Guide. Make sure the IAM user you’re using has at least S3 Read Access.

Open your .env file and add the following lines and replace the values with your credentials.

AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY=YOUR_SECRET_ACCESS_KEY

Now go back to the index.js and add the following lines above the main function:

const S3 = require("@aws-sdk/client-s3");
const s3Client = new S3.S3Client({
  region: "us-east-1"
});

This code will create an S3 Client using your credentials stored in the .env file. If you aren't using us-east-1, you should change it to your region. Now we can create the command to get an object from your S3 bucket and have the client execute it.

Add the following code to the beginning of the main function:

const command = new S3.GetObjectCommand({
  Bucket: "courier-test-ajh",
  Key: "test-pdf.pdf"
});
const data = await s3Client.send(command);

Update the values of Bucket and Key to match your bucket id and the key of the file you'd like to attach. The data contains all we need to attach the file, but we'll have to convert the Body from a readable stream to a buffer so we can get it as a base64 string. We'll use a helper function to convert it.

Add the following function above the main function:

function streamToBuffer(stream) {
  return new Promise((resolve, reject) => {
    let buffers = [];
    stream.on("error", reject);
    stream.on("data", (data) => buffers.push(data));
    stream.on("end", () => resolve(Buffer.concat(buffers)));
  });
}

Now we can use it right after data in the main function:

const command = new S3.GetObjectCommand({
  Bucket: "courier-test-ajh",
  Key: "invoice.pdf"
});
const data = await s3Client.send(command);
const buff = await streamToBuffer(data.Body);

And we'll use all this to create an attachment object right below it.

const attachment = {
  filename: "invoice.pdf",
  contentType: data.ContentType,
  data: buff.toString("base64")
};

Now let's update our Courier send call to use the override:

const { messageId } = await courier.send({
  eventId: "JBP08RT52PM35CNAJNM2GFCB9HHW",
  recipientId: "AYDRIAN10036",
  data: {
    name: "Aydrian"
  },
  override: {
    "aws-ses": {
      attachments: [attachment]
    }
  }
});

Putting it all together

Now if you run the code again, it should pull the specified file from S3, attach it to your email, and send it to you.

Your completed code should look like the following:

require("dotenv").config();
const S3 = require("@aws-sdk/client-s3");
const { CourierClient } = require("@trycourier/courier");

const s3Client = new S3.S3Client({
  region: "us-east-1"
});
const courier = CourierClient();

// Helper function that takes a stream and returns a buffer
function streamToBuffer(stream) {
  return new Promise((resolve, reject) => {
    let buffers = [];
    stream.on("error", reject);
    stream.on("data", (data) => buffers.push(data));
    stream.on("end", () => resolve(Buffer.concat(buffers)));
  });
}

const main = async () => {
  // Retrieve the file from an S3 Bucket
  const command = new S3.GetObjectCommand({
    Bucket: "courier-test-ajh",
    Key: "invoice.pdf"
  });
  const data = await s3Client.send(command);
  // Convert the readable stream to a buffer
  const buff = await streamToBuffer(data.Body);

  // Create an attachment object to provide the override
  const attachment = {
    filename: "invoice.pdf",
    contentType: data.ContentType,
    data: buff.toString("base64")
  };

  const { messageId } = await courier.send({
    eventId: "JBP08RT52PM35CNAJNM2GFCB9HHW",
    recipientId: "AYDRIAN10036",
    data: {
      name: "Aydrian"
    },
    override: {
      "aws-ses": {
        attachments: [attachment]
      }
    }
  });
  console.log("Sent notification: ", messageId);
};

main();

I hope this was helpful. If you're not using AWS SES, you can easily configure Courier to send attachments using another email provider. For other email providers, you can see what changes need to be made to the override to handle attachments by visiting the Courier Email Integrations docs. Give it a try and let me know what you think.

Having trouble getting started, or curious how this would work with a different email provider? Chat with us in the comments below, or email us at support@courier.com – we're happy to help.

-Aydrian

A Guide to Using SparkPost with Node.js

Aydrian — Fri, 01 Sep 2017 13:05:16 +0000

As a Developer Advocate for SparkPost, I write a lot of sample applications. My background is mostly front-end development, therefore my strongest language is JavaScript. Thanks to Node.js, I’m a decent backend developer as well. Does this mean I’m a full stack developer now? Anyway, it was important for me that we had an awesome SparkPost client library for Node.js. So, I dove right in and became a contributor (even before I was hired).

Allow me to help you get started sending emails with SparkPost on your Node.js project.

Installing & Setup

I’m going to assume that you have Node.js installed. Because we follow the Node.js Long Term Support (LTS) schedule, you’ll need to be running version 4 or higher. You can see which version you’re running using the node --version command in your terminal window.

Let’s create a new npm project. If you already have one, you can skip this part.

> mkdir sparkpost-test
> cd sparkpost-test
> npm init --yes

This will create a new project and accept all the defaults. You can also instead run npm init and answer all the prompts.

Now we can install node-sparkpost:

> npm install sparkpost --save

Once installed you can import and create an instance of the SparkPost class:

const SparkPost = require(‘sparkpost’)
const client = new SparkPost('YOUR API KEY')

It’s good practice to avoid putting your API key in code. We highly recommend storing it outside your code, so we set up the client library to detect the SPARKPOST_API_KEY environment variable.

Sending Email

Now that you have a SparkPost instance, you’re ready to send. There are quite a few options available for sending, but let’s start with a simple example. Here’s how you send an email to a single recipient, specifying inline content:

client.transmissions.send({
  content: {
    from: ‘test@your-sending-domain.com’,
    subject: ‘Hello from node-sparkpost’,
    html: ‘<p>Hello world</p>’
  },
  recipients: [
    {address: 'someone@somedomain.com'}
  ]
})
.then(data => {
  console.log('Woohoo! You just sent your first mailing!')
  console.log(data)
})
.catch(err => {
  console.log('Whoops! Something went wrong')
  console.log(err)
})

Note: This example uses Promises, but don’t worry. We also support callback functions.

There are more options available with transmissions, including specifying stored templates or recipient lists, cc and bcc, adding attachments, specifying a campaign, using substitution data, and much more. Check out the examples, docs for the Transmissions resource, and the Transmissions API documentationfor more info.

Bonus: Sending Email with Nodemailer

Nodemaileris a popular library for Node.js that make sending email “easy as cake. For those of you choosing to use this library, we created a SparkPost transport for Nodemailer. You’ll need to install the nodemailer and nodemailer-sparkpost-transport packages in your project.

npm install nodemailer nodemailer-sparkpost-transport --save

Now you can create a nodemailer transport instance:

const nodemailer = require('nodemailer')
const sparkPostTransport = require('nodemailer-sparkpost-transport')
const transporter = nodemailer.createTransport(sparkPostTransport({
  'sparkPostApiKey': process.env.SPARKPOST_API_KEY
}))

Notice how I am reading the API Key from an environment variable. It’s still a best practice to never put that directly in your code.

There are several options that can passed into the SparkPost transport, like campaign id and whether or not the message is transactional. Take a look at the README.md for all the options.

Here’s how you’d send the same message above using Nodemailer:

transporter.sendMail({
  from: 'test@your-sending-domain.com',
  to: 'someone@somedomain.com',
  subject: 'Hello from nodemailer-sparkpost-transport',
  html: '<p>Hello world</p>'
}, (err, info) => {
  if (err) {
    console.error(err);
  } else {
    console.log(info);
  }
})

Double Bonus: Sending Email with notif.me

We all know that email is king of communication but sometimes you want to be able to reach people via multiple channels. notif.me is a Node.js library for sending all kinds of transactional messages. Whether you want to send an email, sms, push, or webpushes, you can do it with ease. It also has built in fall and round robin strategies for multiple providers. We recently worked with the creators to build a SparkPost provider. You’ll need to install the notifme-sdk package in your project.

npm install notifme-sdk --save

You can now create a notifme instance with the SparkPost provider:

const NotifmeSdk = require('notifme-sdk').default
const notifmeSdk = new NotifmeSdk({
  channels: {
    email: {
      providers: [{
        type: 'sparkpost',
        apiKey: process.env.SPARKPOST_API_KEY,
      }]
    }
  }
})

Again, we are pulling the API Key from an environment variable. We’ve said it three times — it’s that important. 🙂

Now let’s repeat this same example, this time using notif.me:

notifmeSdk.send({
  email: {
    from: 'test@your-sending-domain.com',
    to: 'someone@somedomain.com',
    subject: 'Hello from the SparkPost notif.me provider',
    html: '<p>Hello world</p>'
  }
}).then(console.log)

It’s really easy to use and I recommend looking at the other features.

There’s no wrong way to Node

When it comes to sending email using Node.js, you have many options. We’ve worked hard to help make it as painless as possible. If you run into any issues or have any questions, feel free to submit an issue on our node-sparkpost or nodemailer-sparkpost-transport github repos or join us on our community Slack team in the #node channel. I look forward to hearing from you.

This post was originally published on sparkpost.com

Delivering Sweet Tunes with SparkPost and the Spotify API

Aydrian — Thu, 15 Jun 2017 20:06:28 +0000

One of my favorite things about my job is getting to take existing APIs and figure out ways to mix and match them with SparkPost to create cool and interesting apps. Thanks to our friend Todd Motto, there’s a Github repo full of public APIs for me to choose from. Using APIs from this repo, I’ve created apps like Giphy Responder and many others that didn’t quite make it to Github.

SparkPost recently sponsored ManhattanJS, which happened to be hosted at Spotify Headquarters in New York. This inspired me to take a look at the Spotify Web API and come up with something I could demo at the meetup. Their web API allows you do many things, such as search for a song, get information about artists and albums, manage playlists, and so much more. Given that set of functionality, how could I combine it with sending or receiving email to create an engaging demo?

I love music. I was a Punk/Ska DJ in college. When I owned a car, I would sing in it (I still do when I rent one!). I’ve also been a Spotify Premium member since 2011. Now that I live in NYC and travel mostly underground, I rely heavily on my offline playlists. But here’s the problem: I’m not hip or cool, and since I no longer listen to the radio, I don’t know a lot of new music. This usually results in me sitting in a subway car listening to early 2000’s emo bands or sobbing silently to myself while listening to the cast recording of A New Brain.

So yeah... I need suggestions. Spotify has a great social experience but sadly, not everyone has Spotify. But wouldn’t it be cool if you could email songs to a public collaborative playlist? I’m pretty sure everyone has access to email. This would also be a great way to create a playlist for an event. So I set out to create JukePost.

The idea was simple. First I’d create an inbound domain (listen.aydrian.me) that would allow me to send an email to {playlist}@listen.aydrian.me with a list of songs. (Note: {playlist} has to be an already-existing, public, collaborative playlist.) Then I’d create a Node.js app using Express.js to process the relay webhook, search for the song and add it to the specified playlist, and reply with a confirmation that included the songs added and a link to the playlist.

Webhooks, You Gotta Catch’em All!

For this application, I decided to use Firebase, a real time noSQL database. I like to use it for a lot of my demo apps because it makes receiving webhooks extremely easy. It will even receive them when your app isn’t running. You just need to set the target of your webhook to your Firebase URL + store name + .json.

So, let’s set up an inbound domain and create a relay webhook to point to a Firebase database. I’m going to use the SparkPost Node CLI, but you’re welcome to use your favorite way to access the SparkPost API.

Setup the MX records for your inbound domain. I’ll be using listen.aydrian.me
Create your inbound domain: sparkpost inbound-domain create listen.aydrian.me
Create a relay webhook for your inbound domain targeting your Firebase URL. I’ll be using https://jukepost.firebaseio.com/raw-inbound.json

sparkpost relay-webhooks create --name JukePost --target https://jukepost.firebaseio.com/raw-inbound.json --match.protocol SMTP --match.domain listen.aydrian.me

At this point, you should be able to send an email to {anything}@listen.aydrian.me and see an entry under raw-inbound.

Playing with the Playlist

Now that we’re catching the incoming emails, we need a broker app to parse out the data, handle the interactions with Spotify, and trigger a response email.

First, we need to handle authenticating to Spotify using Oauth 2.0. This was my first time doing that and luckily I found the spotify-web-api-node npm package and a great blogpost that assisted me in creating the login, callback, refresh_token routes needed to get everything going. Once the application is authenticated, we can pull the user’s public playlists, filter out the collaborative ones, and save them for later.

Now we can use the firebase npm package to listen for new inbound messages and process them accordingly. Because Firebase notifies us of new messages in real time, we can set up a listener.

const firebase = require('firebase')
const relayParser = require('./relay_parser')

firebase.initializeApp({
  apiKey: process.env.FIREBASE_API_KEY,
  authDomain: process.env.FIREBASE_AUTH_DOMAIN,
  databaseURL: process.env.FIREBASE_URL
})

const ref = firebase.database().ref('raw-inbound')
ref.on('child_added', snapshot => {
  snapshot.forEach(item => {
      const data = relayParser.processRelayMessage(item.val().msys.relay_message)
      // Search for Tracks, Add to Playlist, Send Email
})

You can take a look at relayParser.js to see how I grab the relevant data from the relay message. Based on the information we parsed from the message body text, we now know who sent the message, which playlist to add songs to, and what songs to search for. We now have everything we need to find the songs and add them to the playlist. Be sure to add the song information to a substitution data object, as we’ll use that for the confirmation email.

I chose to get a little fancy with my confirmation email. I decided to use a stored template that would return a link to the playlist and the songs that were added along with artists, cover art, and a 30 second sample. I put my template html and a sample json object in the resources folder of the github repo for your convenience.

This was a fun little project and the demo went over quite well. Want to try it for yourself? Send an email to sparkpost@listen.aydrian.me with the subject line Add and then add your favorite songs to the body in the format {title} by {artist}. Let’s build an awesome playlist together.

This is just the tip of the iceberg for what this app can do. Have a cool idea for an addition? Feel free to create an issue or submit a pull request.

P.S. Want to do more with inbound email processing? Check out my post about inbound email and other cool things you can do with it.

This post was originally published on the SparkPost blog.

DEV Community: Aydrian

Scaling Access Control: Leveling Up Roles with Relationships Using ReBAC

The Starting Point: Homemade RBAC

Migrating to Oso: Streamlining RBAC

The Dual Writes Pattern

Centralizing Permissions with Oso

The Benefits

When Roles Aren’t Enough: The Expense-Sharing Challenge

Introducing ReBAC: Relationship-Based Access Control

Extending Oso: From RBAC to ReBAC

Adding New Facts

Updating the Policy

The Developer Experience

Conclusion

Current Trends in Authorization: Simplifying Access Control

Introduction

Why Traditional Authorization Falls Short

Lack of Scalability

No Support for Fine-Grained Access Control

Fragmented Authorization Across Systems

Inability to Handle Real-Time Context

Current Trends in Authorization

Policy-as-Code: Simplifying Authorization Management

Context-Aware Authorization: Adapting to Dynamic Environments

Why Context Matters

How It Works

Real-World Use Cases

Zero-Trust Authorization: Trust Nothing, Verify Everything

Real-World Use Cases

Getting Started with Zero-Trust Authorization

Fine-Grained Access Control: Precision in Authorization

Why Fine-Grained Access Control?

How Fine-Grained Access Control Works

Real-World Use Cases

Getting Started with Fine-Grained Access Control

Modern Authorization Services: Simplifying Access Control

Why Authorization is Challenging for Developers

What Modern Authorization Services Offer

How Developers Can Benefit

Conclusion

Best of Courier at Hackabull 2021: Speed Friender

How We Kept Datadog From Blowing Up Our AWS Bill

Table Of Contents

What are our real costs?

Identifying the problem

The impact of removing the Datadog layer

Twitch notifications (part three): How to create and notify a list of subscribers using Courier

How to create and notify a list of subscribers using Courier

Prerequisites

Step one: Update your code to send to a list

Create a List in Courier

Add a new subscriber to the List

Swap out the Courier Send

Step two: Add Discord to your notification in Courier

Configure the Discord integration in Courier

Design the Discord notification

Step three: Subscribe a Discord channel to the List

Putting it all together: Full application code

So, what's next?

Twitch notifications (part two): How to send notifications when your Twitch stream goes live

How to send notifications when your Twitch stream goes live

Prerequisites

Step one: Subscribe to the Twitch EventSub online event

Configure the Twitch CLI

Subscribe to the Stream Online Event

Step two: Capture the event and send it to Courier

Gather your environment variables

Process the stream.online event

Step three: Create your notification in Courier

Configure Twilio as your SMS provider

Design your SMS notification

Map your notification to the event specified in your Send call

Putting it all together: Full application code

So, what's next?

Twitch notifications (part one): How to handle real-time events from Twitch

How to handle real-time events from Twitch

Prerequisites

Creating a basic Express Application

Handling the Twitch POST request

Handling callback verification

Process the `stream.online` event