Shehan Ravindu

Posted on Dec 3, 2020

Event-Driven Architecture

What is Event Driver Architecture

An Event-Driven Architecture is a software architecture pattern built upon the production, detection, consumption & reaction to events. An Event-Driven Architecture is including of event series that create event and event consumers that use up & react to events. An event is anything that occurs at a well-defined time and is possible to record. As an example, one person can order something by clicking one button on a web page, or a sensor recorded a measurement.
An EDA includes event producers that generate a stream of events & event consumers that listen for the events.

An event-driven architecture is able to use an event stream model or pub/sub-model.

Event Stream model: Events must be written to a log. Events are strictly ordered & durable. Clients do not subscribe to the stream instead a client is able to read from any part of the stream. The client is responsible for advancing its position in the stream. That means a client can join at any time & possible to replay events.
Pub/Sub Model: The messaging infrastructure keeps track of subscriptions. When an event is published that event sends the event to each and every subscriber. After an event is received it is impossible to replay & new subscribers don't see the event.
There are some common variations on the consumer side.
Simple Event processing: An event immediately triggers an action in the consumer. As an example, you could use Azure Functions with a Service Bus trigger so that a function executes whenever a message is published to a Service Bus topic.
Complex Event Processing: A consumer processes a series of events looking for patterns in the event data using a technology such as Azure Stream Analytics or Apache Storm. As an example, you could aggregate readings from an embedded device over a time window & produce a notification if the moving average crosses a certain threshold.

Description of Event-Driven Architecture

An Event-Driven Architecture is used to develop several real-world streaming use cases such as financial trading, cyber-security, fraud detection, e-commerce, Internet of Things preventative maintenance.
From both technical and strategic perspectives, Event-Driven Architecture can be broken with the traditional Data Warehouse architecture. Data is currently stored in the Data Warehouse and analyzed to generate reports, dashboards, or other tools used to guide management decisions. However, this is not a fast process that is completely useless for an on-demand service in which the environment changes every couple of minutes.

One of the biggest resistance when implementing an event-driven architecture based solution is that yet there isn't any dominant development framework. But software engineers are able to integrate solutions from various technology suppliers to produce Event-Driven Architecture based projects.

Event capturing with Mulesoft, SnapLogic, Dell Boomi, Apache Apex, Hortonworks Dataflow.
Routing with Apache Kafka, RabbitMQ & ActiveMQ.
Processing with Apache TIBCO BE, Azure Stream Analytics, Amazon Kinesis, Apache Storm, Flink.

Event Stream Processing is the general idea of data integration and analysis. The Event Stream Processing flow can be divided into four steps of event processing those events are Collect, Enhance, Analyze, and Dispatch. The whole flow produces a nonhomogeneous process in which events are inherited by the system and generate a certain output response thanking the processing of different services.

The typical event-driven development makes use of an event bus and various simultaneous services accessing the bus. Data can be stored in replicated for every service & local memory so that they can work freely. Every service can inherit and produce event notifications. The architecture can be extended to the use of different event buses & specified interface services.

Once an event gets closer to the event bus the flow of actions from the different services is difficult to control and predict. But every service is easily replaceable and modifiable since the architecture is loosely tied. Every service is developed to do the specific task following various priorities or timing restrictions & the change in any other process won't affect them. In this manner, long term adaptation & optimization of the system are completely feasible.

The event-driven architecture is great & it has a lot to offer for microservices systems. Let's not step into the trap of thinking that all communication should take place via events there are several clear facts why this is an antipattern.
The key characteristics are that Event-Driven Architecture communicates through APIs & events & the actors are ephemeral and stateless. Event-Driven architecture produces a separation of logic from data, cache, and state.

Significance of Event-Driven Architecture

Almost every application or service built currently assumes that you are able to do whatever you need as soon as possible get instant feedback on the status of your request & interact in real-time with the request & anyone involved in the process. Whether it is ordering products or manufacturing, taxi, financial processes, messaging, or chat everything is event-driven today.

Conceptually This Event-Driven Architecture is any loosely coupled middleware that considered handling events. However, in real-world scenarios, Enterprise Service Bus (ESB) provides a platform on top of which Event-Driven Architecture is easier to develop. Then we are possible to realize in a practical perspective, an Event-Driven Architecture is an event enabled Enterprise Service Bus.

An Event-Driven Architecture moves event processing out of hardcoded existing applications to the enterprise level. That enables a software engineer to write enterprise spanning applications to combine suppose the alert that an important sale has taken place & the alert that you are out of inventory in that product. By writing applications on top of this distributed platform, the software engineer is able to select which events to handle. As an example, complex event processing. A software engineer is able to add alerts and real-time critical reports to existing applications that have nothing. Even more interesting they are able to do things like setting up an automated response loop to detect take initial actions & alert people to handle critical business situations faster than a human-based business workflow.

I have mention below several benefits of Event-Driven Architecture in brief,

Consumers and producers are separated.
Highly distributed and scalable.
Consumers are able to respond to events immediately as they are coming.
Subsystems have independent views of the event stream.
No point to point integrations. It is easy to add new consumers to the system.

Importance of EDA in the Era of Microservices

First, let's see what Microservices is. "Microservices is a software development technique - a variant of the service-oriented architecture architectural style that structures an application as a collection of loosely coupled services. In a microservices architecture, services are fine-grained and the protocols are lightweight." -Wikipedia

There is a problem with the role of messaging in microservices. Microservices are no different from regular services & microservices are the ideal way to design services. In normal services should be as small as practical as possible and still be useful.

It is certainly the case that if you have a microservices architecture you will still need to access those microservices from an Enterprise Service Bus. Enterprise Service Buses are perfect as if your microservice uses a special protocol or interface the Enterprise Service Bus is able to talk to it more easily than most things.

A standard RESTful API could be created as a facade over the microservice to make it easier for other applications to use the microservice if some chosen apps for performance want to speak directly to the microservice that is possible done so that the backlog produced is minimized.
When EDA is used to implement monolithic applications we have to face several challenges. They are,

Scalability
Reliability and availability
Agility

In an event-driven architecture, an agent is activated by the arrival of an event but in microservices architecture, an event can be activated by an explicit call. As an example, if I need to calculate the balance of my bank account I am calling a service that does this calculation and shows me the result or just brings it from a database. In contrast in event-driven architecture the different functions aren't explicitly called but triggered by the occurrence of a specific event (e.g. in connected cars. the velocity exceeds the speed limit).

Microservices are the way forward Automation and decentralization

As we all know microservices separate more traditionally structured applications up into controllable pieces that are possible to maintained and implemented independently. Because these smaller components are much lightweight the codebase for each is able to significantly simpler to understand leading to a more agile development cycle.

In addition, microservices are often decoupled allowing for updates with little to without downtime as the other components are possible to continue running.

Event-driven Triggering adaptation

The event-driven architecture is hardly a new idea. Software engineers who are in the database background have been using database triggers for many years. The idea is simple whenever you change, add, or delete data an event is triggered to do a variety of functions. The new thing is the proliferation of these kinds of triggers and events in applications outside of the traditional relational database management systems.

Mediator Topology & Broker Topology

In Event-Driven Architecture we can see two major topologies.

Mediator Topology
Broker Topology

Mediator Topology

Mediator topology consists of one event queue & one mediator that points each of the events to relevant event processors. Normally events are fed into the event processors passing through an event channel to preprocess events or filter.

The development of the event queue could be in the form of a simple message queue or even a message passing interface in a large distributed system that involves complex messaging protocols such as Rabbit MQ. The diagram above demonstrates the architectural development of the mediator topology.

To show how the mediator topology works, suppose you are insured through an insurance company and you decide to move. In this scenario, the starting event might be called something like a relocation event. The steps involved in processing a relocation event are contained within the event mediator. For each initial event step, the event mediator creates a processing event (examples: change address, recalc quote, etc.), sends that processing event to the event channel & waits for the processing event to be processed by the corresponding event processor (example: customer process, the quote process, etc.). This process continues until all of the steps in the staring event have been processed. The single bar over the recalc quote and update claims steps in the event mediator indicates that these steps are able to run at the same time.

Broker Topology

Broker topology does not involve an event queue. Event processors take accountability for obtaining events, publishing, and processing another event showing the end. Because the name of the topology implies event processors act as brokers to chain events. Once the processor processes an event another event is published then another processor has a chance for proceeding.

According to the above diagram shows some event processors process and leave no trace and some tend to publish new events. This is the same as what we can see in a NodeJS application. Steps of certain tasks are chained in the manner of callbacks when one task ends the callback is triggered & all the tasks remain asynchronous in nature.

To show how the broker topology works, we will use the same example as in the mediator topology (an insured person moves). As there is no central event mediator to receive the starting event in the broker topology, the customer-process component gets the event directly, changes the customer address & sends out an event saying it changed a customer's address (example: change address event). In this example, there are two event processors that are interested in the change address event. The quote process & the claims process. The quote processor component recalculates the fresh auto-insurance rates based on the address change & publishes an event to the rest of the system indicating what it did (example: recalc quote event). The claims processing component, on the other hand, gets the same change address event, but in this case, it updates an awesome insurance claim & publishes an event to the system as an update claim event. These new events are then picked up by other event processor components & the event chain continues through the system until there are no more events are published for that particular initiating event.

Use Cases of Event-Driven Architecture

Trading Floors

A trading floor consists of a lot of information from several sources. These information providers aggregate content from a lot of and different kinds of sources. After that feed this information as a stream of subject-oriented feeds. Building a trading floor using EDA architecture involves building an extremely performing infrastructure consisting of a number of services that must be able to sustain data rates well in excess of thousand transactions per second. Each and every process is provided in a cluster or set of clusters and usually, an active method of fault tolerance is employed. A message broker is used for trades and things related to auditable entities. Topics are used to distribute market data. Systems are monitored using Activity Monitor and Metrics produced. Data also needs to be reliably sent to Risk Analysis which computes credit limits and other limits the firm has for trading operations in real-time. Complex event processing is used to detect anomalous events, security events or even opportunities.

Health Care

Provider, Consumer, Transaction, Patient health data are the four systems of MMIS systems. Each of these systems is integrated and each requires its own EDA architecture. Standards in the health industry include HL-7 for message format and coding. Important standards to be supported in any system include HL-7 EHR standards ICD coding standards and numerous other changing specifications. Systems want to help strong privacy, authentication, and security to protect individuals.

Taxi Service

Online taxi service has several applications that all talk directly to an ESB hub in the cloud or an API management service. A message broker is added for queuing and creating a publish-subscribe framework in the backend infrastructure. This gives chance a new pickup to alert several support services and tracking. It also consists of an API Store for external developers who need to integrate Ufer service into their apps making it easier to arrange to pick up or drop off from any event bar or business that needs.

Key Design Considerations of EDA

The major characteristics of Event-Driven Architecture are communicating via events and APIs. & the actors are stateless and ephemeral. This produces a separation of logic from cache, data & state. Here I am able to mention the main three design considerations for an Event-Driven Architecture. They are Payload design, Event stores, and Standard routers.

Payload design

The software engineer will put the payload on the object itself. The downside is you are handling much more data. Now you have to consider security constructs with what actually goes into the event & how it gets passed forward.

In simple terms, it is just another way of encoding JSON object and use that encoded object as access tokens for authentication from the server. It is compact and self-contained.

There are main three parts of JWT. They are Header, Payload, & Signature.

Header: The header contains the metadata for the token & it minimally contains the type of signature and the encryption algorithm.
Payload: In the context of JWT, a claim can be defined as a statement about an entity as well as additional metadata about the token itself. The claim includes the information we want to transmit &that the server is able to use to properly handle JSON Web Token authentication. There are multiple claims we are possible to provide. These include registered claim names, public claim names, and private claim names.
Signature: The JWT standard follows the JSON Web Signature specification to generate the final signed token. It is produced by combining the encoded JWT Header and the encoded JWT Payload & signing it using a strong encryption algorithm, such as HMAC SHA-256. The signature's secret key is held by the server so it will be able to verify existing tokens & sign new ones.

In Amazon, Order microservice sends an event order canceled and Customer service listens to that event.

Now there could be two ways of sending the event data:

Send complete order data in the Event Pros: Listener services do not need to query Order Service for their functioning.

Cons: Lots of data will be passed in the event even though only 10 % is used lots of I/O.

Send only order id, cancel reason, customer id, date in the event

Pros: If data chosen carefully very little data is flown in the event. Cons: If data chosen incorrectly then lots of API requests

Event Stores

There are two basic event store models. They are streams and queues. The benefit to the streams model is that our events are possibly processed in order & multiple consumers can be processing the same event list. The queue is acting as a durable stowaway. These streams and queues give us a durable & reliable way for events to be replayed recreated & otherwise to move forward.

Events are persisted in an event store. Not only does the event store act as a database of events. It also behaves like a message broker. It supplies an API that enables services to subscribe to events. Each event that is persisted in the event store is delivered by the event store to all interested subscribers. The event store is the backbone of an event-driven microservices architecture.

In this event-driven architecture, requests to update an entity (either an external HTTP request or an event published by another service) are handled by retrieving the entity's events from the event store, reconstructing the current state of the entity, saving the new events & updating the entity.

Standard Routers

The third key consideration to be made regards the possible use of routers. The developer wants to think about what capabilities are needed for a router to be present. A lot of the event-driven architecture capabilities are implicitly bound to the down-stream actor. In many cases, you find service suppliers embodying these capabilities in one form or another. Lambda allows running pub/sub-construct on top. This gives you the ability to securely associate multiple sources to multiple destinations.

In a world where events are flowing all over the place. It is essential to make sure that anyone who is making an event is authorized to do so, & anyone that's consuming an event is authorized to do so. Also, verify that the person who's emitting it is the person who says it is.

As an example, "you do not want to have someone say 'Here is a hundred thousand dollar credit to my account' & slip that into the event stream without anyone else knowing about it." It is critical to make sure that the mapper itself has the ability to map securely between those two.