Event-Driven Magic: Exploring AWS EventBridge

#aws #eventbridge #eventdriven #architecture

I kept thinking about what Werner Vogels said in his speech at AWS re:Invent in December 2022,

the world is event-driven

Even though I’ve involved on a few event-driven architectures before, his words made me reconsider how we use event-driven architectures.

The result? I’ve developed some impressive reference architectures utilising modern cloud-based tools and services. Let’s begin by reviewing the fundamentals.

Event-driven architecture is all about events. Simply put, an event is a change in state. Picture it like this: swiping a credit card at a payment machine is an event. Our world is full of these actions and events, and they play a big role in how everything functions. Think of events as signals that indicate when something changes in a system, like saving information to a database, uploading a file into storage, or a sensor being triggered on an IoT device.

A key point to remember about events is their immutability; once they occur, they cannot be altered. This concept mirrors everyday life — once a moment passes, it is unchangeable. For instance, I wished for the All Blacks to win RWC 2023. Regardless of how much I wanted it, there was no way to go back and change the outcome once the event had taken place and was recorded in history.

Event Driven Architecture Components

Event Producer: These entities generate events based on specific actions or triggers within the system and are solely aware of the event router, not the event consumers.

Event Broker: The event broker is responsible for receiving and filtering events, ensuring that each event is directed to the correct consumer. It serves as a mediator, enabling the smooth transmission of events throughout the architecture.

Event Consumer: Event consumers either subscribe to particular events or monitor event streams, reacting to events based on their individual needs or initiating further actions within the system.

Event-Driven Architecture Patterns

There’s much discussion surrounding event-driven architecture patterns across various categories. Below are some of the most popular patterns:

Communication Patterns: publish/subscribe, point-to-point, event streaming, request/reply

Consumption Patterns: guaranteed delivery, hierarchical topics, event filtering, backpressure, and push

Event Generation Patterns: ECST, event sourcing, CQRS, CDC

Governance Patterns: event catalog, event APIs, intermediated, disintermediated, access control, and authorization

Let’s talk about EventBridge

Let's keep our attention on the main topic. We should delve into what EventBridge is and explore the reasons behind its growing popularity.

EventBridge, introduced in July 2019, is a serverless service that leverages events to seamlessly connect components of software applications. This facilitates the creation of modern, scalable, and loosely coupled event-driven applications.

Its robust positioning within this architectural domain stems from its seamless integration capabilities with numerous other AWS services, including Step Functions, SQS, SNS, and Lambda functions, all of which inherently complement event-driven architectures.

In the middle of the illustration, we have AWS EventBridge, a serverless event bus. EventBridge is designed with a pub-sub architecture pattern in mind, allowing external sources to ingest events as ‘publishers.’ These sources can range from bespoke applications and various SaaS applications to a wide array of AWS services.

Events

An event is a real-time change in a system, data, or environment. This change can occur in your application, an AWS service, or a SaaS partner service. While events can have similar structures, they must all include the top-level fields: detail, source, and detail-type. Custom events must, at a minimum, contain these fields.

Custom Event Sample

{
    "version": "0",
    "id": "665f15b9-bbbb-ffff-0000-ee92777062fb",
    "detail-type": "product.update",
    "source": "your source app",
    "account": "123456789012",
    "time": "2024-07-16T05:24:42Z",
    "region": "us-east-1",
    "resources": [],
    "detail": {
        "action": "product update"
    }
}

Rules

Once the events are ingested into EventBridge, rules redirect the events to appropriate targets based on their content. EventBridge has a soft limit of 300 rules, which can be increased via a support ticket. These rules can be connected to multiple targets, such as AWS services like Lambda functions, Step Functions, or SQS queues.

Event Schema

EventBridge is capable of handling event schemas, which represent the structure of events. A schema typically includes details such as the title and type of each data element. For example, on an online survey website, a schema might include fields for the survey name, description, reviewer’s name, user ID, and reviewer’s comments and ratings. These attributes are defined with specific types, such as the name being a string and the user ID being a GUID, etc.

Schema Registry

A schema registry serves as a centralised repository for managing a collection of schemas. It allows you to search, locate, and track various schemas utilised and generated by your applications, ensuring consistency and compatibility across your data infrastructure.

Schema Discovery

Schema discovery automates the identification and registration of schemas, streamlining their integration into your registry. When enabled for an EventBridge event bus, schema discovery automatically records the schema of each event sent to the bus in the registry. If an event’s schema changes, schema discovery promptly creates a new version in the registry. After a schema is registered, you can generate a code binding for it, facilitating seamless interaction with the schema.

Pipes

EventBridge buses facilitate the routing and distribution of events to services using the publish/subscribe pattern, while EventBridge Pipes manage direct integrations between services through the point-to-point pattern. Both patterns are essential for constructing event-driven architectures. EventBridge Pipes allow for direct integrations without needing specialised knowledge. These pipes consist of a Source and a Target, and an enrichment step can be employed to enhance the payload data before it is sent to the target.

EventBridge Use Cases

There are numerous use cases for event-driven architecture where we can effectively apply AWS EventBridge. Let’s explore some of these use cases.

SaaS Product Integration

Imagine your organisation has a CRM system that is a SaaS partner with AWS EventBridge. The architecture below can be used to integrate this CRM with other business activities. The CRM system can send events to the EventBridge whenever there are changes in the system, data, or environment. Based on the rules defined in the EventBridge, messages are distributed to the appropriate tasks. For example, when a new customer registers in the CRM, the customer banking integration Step Function subscribes to the event and performs the relevant tasks.

Bespoke System Integration

When developing a bespoke software application that requires integration with external entities in an event-driven manner, the following architecture would be ideal.

Consider a scenario where a customer requests custom-made furniture. The request is received by the company’s sales system, which then communicates the specifications to the factory. Throughout the various phases, the customer needs to be kept informed, and once the furniture is ready, the designated courier company must be notified for pickup and delivery. This integration can be handled more reliably and efficiently using an event-driven architecture with AWS EventBridge, rather than a polling architecture.

After diving into a multitude of theories, it’s great to get some hands-on experience. Part two of this article, featuring a near real-world EventBridge integration, will be released soon. Stay tuned!