DEV Community: Shredded Mustard

"Listen to Yourself". Event sourcing for Domain Driven Design ... One Domain Event to Rule Them All

Shredded Mustard — Mon, 02 Dec 2024 12:30:46 +0000

Event-driven architecture is evolving rapidly. Software architects have been developing event-driven architectural patterns to solve the problems that come with such a robust design. Executing a non-idempotent operation might not be as simple as publishing a message to a queue. A non-idempotent operation is an operation that changes the state of our application. There are numerous patterns that dictate when and how to publish a non-idempotent event. When a "Fire and Forget" event happens, we must be careful in dealing with the side effects.

One problem to deal with is the Dual-Write problem; Writing to two external systems in one transactional context. Often times a process involves writing to the database and then publishing an event to a message broker like Kafka. For example, in Event-Sourcing, after adding or updating a an entity to the database, a message is published to advertise the new state of the application.

This problem can be solved programmatically by implementing rollback mechanisms or by applying patterns like the Outbox pattern for example. The Listen to Yourself pattern is also one of the solutions to such a problem, and it solves it in a very simple way.

Instead of updating the database and then publishing an event, we only publish the event. The same component that publishes the event consumes the published event; hence the name "Listen to Yourself". In other words the component responsible for doing the operation listens to its own topic and consumes the messages that it sent to update its database. This means that when the event is published, the database update is still pending and may or may not happen.

Listen to yourself

Before diving into this, make sure you know what Event-driven architecture is and some common patterns. You can read about it here Event Driven Architectural patterns.

The "Listen to yourself" pattern adds the event publisher to the List of event consumers. You heard it right. The same component that is responsible for updating the database and firing the event becomes one of the event consumers. In other words, the publisher listens to itself. However, the database update will be postponed to after firing the event.

Why is this useful?

Minimize side effects
Executing all side effects concurrently minimizes the problems that would arise from a faulty message broker. If for example the message broker failed to write the event to the message queue for any reason, there will be no event and therefore no side effects as opposed to writing in the database first and then firing the event. This is useful for event-sourcing which treats an event as state change. If there is no event, there is no state change.
Easy replay implementation
Event replays in event-sourcing is a very useful technique. If you want to implement a replay for the primary domain, it will be as easy as editing the primary domain consumer implementation, or extending its behavior. This way, we only update the database in the consumer implementation.
Separation of concerns
When you structure your domain model carefully, the domain event becomes a single source of truth that can provide great consistency across your services. This means that if you want to move some subdomains to a separate services, it can be done in a very short time and the logic will not change as the subdomains are already embedded in the domain event. You can then easily replay all domain events from the new services and tada; You have a new consistent subdomain database. This is the most important aspect of the "Listen to Yourself" pattern and arguably the part where you should spend most of your time contemplating.

Listen to Yourself in practice

Consider an e-commerce store that allows the user to acquire free $10 in credits as soon as they sign up using a gift card.

Whenever a new user is created, $10 is added to his wallet.

In this scenario we have two domains.

The first domain is the "User".
The second domain is the "Gift".

The "Wallet" is a subdomain of the User.

Before you dive in make sure you have:

Java 11 or above installed on your machine
Apache Maven installed
Running instances of Kafka and MongoDB (We will use Docker compose for running both instances so if you have docker installed you do not need to install anything else)
Any IDE or Text Editor

You will find the full code source here https://github.com/Shredded-Mustard/ltys-user-svc

Let us take a look at our dependencies which we will use our pom.xml file to define

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-mongodb</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-validation</artifactId>
            <version>3.3.5</version>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.12.0</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
    </dependencies>

We will be Using the Spring framework's Web package along with Spring-Kafka.

Our controller will look like this

package com.noketchup.shop.user.controller;

import com.noketchup.shop.user.controller.dto.UserRequest;
import com.noketchup.shop.user.controller.dto.UserResponse;
import com.noketchup.shop.user.domain.UserDomainModel;
import com.noketchup.shop.user.domain.WalletDomainModel;
import com.noketchup.shop.user.producer.UserProducer;
import com.noketchup.shop.user.service.UserService;
import com.noketchup.shop.user.service.WalletService;
import com.noketchup.shop.user.service.mapper.UserMapper;
import jakarta.validation.Valid;
import lombok.RequiredArgsConstructor;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;

import java.util.List;

@RestController
@RequiredArgsConstructor
@RequestMapping("user")
public class UserController {

  private final UserService userService;
  private final WalletService walletService;
  private final UserProducer userProducer;
  private final UserMapper mapper;

  @GetMapping
  public ResponseEntity<UserResponse> getUserByUniqueIdentifier(
          @RequestParam(value = "email", required = false) String email,
          @RequestParam(value = "phoneNumber", required = false) String phoneNumber,
          @RequestParam(value = "username", required = false) String username
  ) {
    UserResponse userResponse = userService.getUserByUniqueParam(email, phoneNumber, username);
    return ResponseEntity.ok(userResponse);
  }

  @PostMapping
  public ResponseEntity<String> createNewUser(@RequestBody @Valid UserRequest userRequest) {
    //Validate all fields before taking any action
    userService.validateUser(userRequest);

    //Map Request to the Domain Model
    UserDomainModel userDomainModelObject = mapper.mapToUserDomain(userRequest);
    WalletDomainModel wallet = walletService.createNewEmptyWalletDomainObject();
    userDomainModelObject.setWallets(List.of(wallet));

    //Send Domain event
    userProducer.sendUserDomainEvent(userDomainModelObject);
    return ResponseEntity.accepted().body("User creation in progress");
  }
}

We have two main endpoints.

The GET endpoint gets a single user by email
The POST endpoint Adds a new User to the database

Our producer interface needs only to have one method which is produceUserDomainEvent

package com.noketchup.shop.user.producer;


import com.noketchup.shop.user.domain.UserDomainModel;

public interface UserProducer {
  void sendUserDomainEvent(UserDomainModel userDomainModelObject);
}

We also have to have a Consumer Interface which should also have just one method consumeUserDomainEventModel

package com.noketchup.shop.user.consumer;

import com.noketchup.shop.user.domain.UserDomainModel;
import org.apache.kafka.clients.consumer.ConsumerRecord;

public interface UserConsumer {
  void consumeUserDomainEventModel(ConsumerRecord<String, UserDomainModel> userDomainModel);
}

We also have to implement both interfaces like this

package com.noketchup.shop.user.producer;

import com.noketchup.shop.user.domain.UserDomainModel;
import lombok.RequiredArgsConstructor;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.header.Header;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;

@Component
@RequiredArgsConstructor
public class UserProducerImpl implements UserProducer {
  @Value("${config.user.domain.topic}")
  private String userDomainTopic;


  private final KafkaTemplate<String, UserDomainModel> userDomainKafkaTemplate;

  public void sendUserDomainEvent(UserDomainModel userDomainModelObject) {
    ProducerRecord<String, UserDomainModel> record = new ProducerRecord<String, UserDomainModel>(
            userDomainTopic,
            userDomainModelObject.getId().toString(),
            userDomainModelObject
    );

    record.headers().add(new DomainEventHeader("NAME", "CREATE_NEW_USER"));
    userDomainKafkaTemplate.send(record);
  }

  public static class DomainEventHeader implements Header {

    private final String key;
    private final String value;

    public DomainEventHeader(String key, String value){
      this.key = key;
      this.value = value;
    }

    @Override
    public String key() {
      return key;
    }

    @Override
    public byte[] value() {
      return value.getBytes();
    }
  }
}

package com.noketchup.shop.user.consumer;

import com.mongodb.MongoException;
import com.noketchup.shop.user.domain.UserDomainModel;
import com.noketchup.shop.user.exception.RetryableException;
import com.noketchup.shop.user.service.UserService;
import lombok.RequiredArgsConstructor;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Transactional;


@Component
@RequiredArgsConstructor
public class UserConsumerImpl implements UserConsumer {
  private final UserService userService;

  @Transactional
  @KafkaListener(topics = "${config.user.domain.topic}", groupId = "${spring.kafka.consumer.group-id}")
  public void consumeUserDomainEventModel(ConsumerRecord<String, UserDomainModel> userDomainModel) {
    UserDomainModel userDomainModelObject = userDomainModel.value();
    try {
      userService.commitDomain(userDomainModelObject);
    } catch (MongoException e) {
      throw new RetryableException("Retryable Exception", e.getMessage());
    }

  }
}

And to define our Mongo Repository as well

package com.noketchup.shop.user.repository;

import com.noketchup.shop.user.model.User;
import org.springframework.data.mongodb.repository.MongoRepository;
import org.springframework.stereotype.Repository;

import java.util.Optional;
import java.util.UUID;

@Repository
public interface UserRepository extends MongoRepository<User, UUID> {
}

Make sure to download the full code from https://github.com/Shredded-Mustard/ltys-user-svc or you can define the model and implement the User Service and Mappers in your own way.

And our properties file may look like this

spring.application.name=User Service

# Database Configuration
spring.data.mongodb.host=localhost
spring.data.mongodb.port=27017
spring.data.mongodb.database=userdb
spring.data.mongodb.username=noketchupadmin
spring.data.mongodb.password=sausage
spring.data.mongodb.authentication-database = admin
spring.data.mongodb.auto-index-creation=true

# Kafka Configuration
spring.kafka.bootstrap-servers=localhost:29092
config.user.domain.topic=user

spring.kafka.consumer.group-id=@artifactId@-group
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.consumer.enable-auto-commit=false
#spring.kafka.listener.ack-mode=MANUAL
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.springframework.kafka.support.serializer.JsonDeserializer
spring.kafka.listener.fixed-backoff.max-attempts=4
spring.kafka.listener.fixed-backoff.interval=1000
spring.kafka.consumer.properties.spring.json.trusted.packages=*

spring.kafka.producer.acks=all
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.springframework.kafka.support.serializer.JsonSerializer

To run Kafka and Mongo DB we will add a Docker Compose file

version: '2'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.3.2
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    ports:
      - 22181:2181

  kafka:
    image: confluentinc/cp-kafka:7.3.2
    depends_on:
      - zookeeper
    ports:
      - 29092:29092
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
  mongo:
    image: mongo:latest
    restart: always
    ports:
      - 27017:27017
    environment:
      MONGO_INITDB_DATABASE: userdb
      MONGO_INITDB_ROOT_USERNAME: noketchupadmin
      MONGO_INITDB_ROOT_PASSWORD: sausage
    volumes:
      - ./mongo-init.js:/docker-entrypoint-initdb.d/mongo-init.js:ro
  mongo-express:
    image: mongo-express:latest
    restart: always
    ports:
      - 9081:8081
    depends_on:
      - mongo
    environment:
      ME_CONFIG_MONGODB_AUTH_USERNAME: noketchupadmin
      ME_CONFIG_MONGODB_AUTH_PASSWORD: sausage
      ME_CONFIG_MONGODB_URL: mongodb://noketchupadmin:sausage@mongo:27017/
      ME_CONFIG_MONGODB_ENABLE_ADMIN: true

Now let us start our containers and run our application

> docker compose up
> mvn compile
> mvn exec:java -Dexec.mainClass=com.noketchup.shop.user.UserServiceApplication

Now we can add a new User

curl --location 'http://localhost:8080/user' \
--header 'Content-Type: application/json' \
--header 'Cookie: lng=en' \
--data-raw '{
    "username": "mustard",
    "dateOfBirth": "1996-03-02",
    "mobileNumber": "0123456789",
    "email": "mustard@onthebeat.com"
}'

We can also fetch our user like this

curl --location 'http://localhost:8080/user?email=mustard%40onthebeat.com' \
--header 'Cookie: lng=en'

and we get

{
    "id": "74b1e501-5d5c-4e38-9257-4654c301fd34",
    "username": "mustard",
    "dateOfBirth": "1996-03-02",
    "mobileNumber": "0123456789",
    "email": "mustard@onthebeat.com"
}

To recap

We first exposed our GET and POST endpoints
We created the main domain event producer and consumer
We Defined our Mongo Repository
We configured Kafka and Mongodb

Not only do we have our addNewUser command implemented, we also have a free Event that we can consume from any other service to hook another command to an operation chain at any point in time.

One event to Rule them all

The event is already there, all you have to do is implement your consumer

In the Gifts service we will implement our consumer the same way we did in the User Service

@Component
@RequiredArgsConstructor
public class UserConsumerImpl implements UserConsumer {
  private final UserService userService;

  @Transactional
  @KafkaListener(topics = "${config.user.domain.topic}", groupId = "${spring.kafka.consumer.group-id}")
  public void consumeUserDomainEventModel(ConsumerRecord<String, UserDomainModel> userDomainModel) {
    UserDomainModel userDomainModelObject = userDomainModel.value();
    // Add gift credits to the newly created user or DO WHATEVER YOU WANT based on that information
  }
}

Event Sourcing

Shredded Mustard — Mon, 04 Nov 2024 12:45:58 +0000

Imagine data flowing continuously from one source to multiple destinations, passing through various stages of processing.

This is how we can visualize many modern applications. In building scalable systems, data management is central—along with preserving the history of that data. In traditional systems, when we update an entity's state in a database, the previous state is overwritten, and we lose any record of its past.

Consider a banking service that records a user’s account balance. Without logging each transaction, we wouldn’t know how the current balance was reached. When users review their balance, they expect to see a history of transactions that led to the current amount. Without this history, there's no clear way to explain how they arrived at their present balance.

Another example is a shipping service that allows users to track parcels. Once a package is shipped, tracking its journey is crucial for both user experience and problem-solving if the package gets lost. By tracing its path, we can identify exactly where things went wrong.

An event represents a fact, action, or change.
In this approach, each event marks a state change in the application. Storing these events enables us to capture the state of the application at any given time. By replaying these events, we can even restore the application to a previous state, almost like rewinding through a series of snapshots.

To understand event sourcing, consider breaking down an application into domains—distinct business areas or groupings of related processes. When a domain’s state changes, it emits an event to signal this update.

For example, an Account domain might publish an [Account Created] event when a new account is added for a user. This event is saved in an append-only log, and other domains can interpret it as they see fit.

By storing each event, we can replay them at any time to rebuild our application state. This replay feature is invaluable. Say we introduce a new service interested in past events—it can simply replay the event log to establish its state. For instance, if only the account service exists initially and a new User Service is added later, this service can replay the event log to create an up-to-date list of accounts linked to each user.

This also simplifies migrating or replatforming applications. If we decide to rebuild the application with a new technology, we only need to replay the event log to restore the data without worrying about database migrations or state loss.

Picture it like this: a user logs into a banking app, registers an account, and opens a new debit account. Two primary domains are involved here—the User domain and the Account domain. As the user is created and stored, the User domain publishes a [UserCreatedEvent], while the Account domain follows with an [AccountCreatedEvent].

Many other interconnected domains may find these events useful. For example, if a fraud detection feature is added to the Security domain, it can process the entire event log to analyze user behavior and account activity, identifying potential fraudulent actions. In scenarios like this, retaining event history proves immensely valuable.

Event Sourcing vs CRUD

In typical CRUD systems, updating an entity overwrites previous states without keeping a historical record. Event sourcing addresses this by storing each event in an append-only log, preserving the complete change history.

However, CRUD systems generally ensure strong consistency, meaning all parts of the system see updated data immediately. In contrast, event sourcing provides eventual consistency. This means that if your application requires strong consistency across multiple domains in real time, event sourcing may not be the best fit.

Event sourcing and CQRS

Event sourcing and CQRS (Command Query Responsibility Segregation) work well together. By separating read and write operations, the write services can be considered the source of truth and scaled as needed. The query services only read the event log to update their state without directly changing the application's state.

Benefits

Audibility and traceability
Event sourcing allows seamless auditing across domains. Instead of implementing auditing logic in every service, an Audit service can be created to consume events directly from the event log. Additional information, like trace IDs or timestamps, can be added to event messages to enhance traceability with minimal effort.
Scalability
Combining event sourcing with advanced storage technologies and partitioning techniques makes it easier to scale and test the application, especially in high-demand environments.

Trade-offs

Eventual consistency
As mentioned, event sourcing only provides eventual consistency. This may not suit applications needing immediate data consistency, like stock trading platforms.
Storage concerns
Events can accumulate rapidly, leading to significant storage demands. Careful management of event storage and strategies like snapshotting may be required to handle data efficiently.
Complexity in querying
While simple events are easy to store, complex queries across large event logs can be challenging.

CQRS

Shredded Mustard — Sun, 03 Nov 2024 12:16:57 +0000

Distributed systems have long been evolving. Most distributed systems use the Database per Service concept, which means that each service has its own database that it is fully responsible for, and only the service can access this database to perform CRUD operations. CRUD stands for Create, Read, Update, and Delete. In most modern applications, there is a Data Access component, which acts as a contract between the application and the database. It is responsible for modeling the database and managing querying, inserting, updating, and deleting operations, all encapsulated within the CRUD Data Access Component.

If our application needs to fetch some data from the database, for example, it communicates with the Data Access component to retrieve the required data. Most read operations may be straightforward, like "find entity by ID" or "find by Username." However, when dealing with write operations, things are often not as simple. Before performing write operations, a series of checks and validations may be necessary. For example, if we have a Tickets database and want to purchase a new ticket for User "X," we might first validate that the user meets the age requirement [findUserAgeByID()], then check ticket availability [countTicketsByEventId()], and only then proceed with updating the ticket status to "Pending." After that, we credit the User's bank account with the ticket's value and finally change the Ticket status to "Acquired." This chain of operations is unnecessary in a read flow, where, for example, the user simply wants to view their purchased tickets. In that case, retrieving data is as straightforward as [findTicketsByUserId()]. This highlights that most application logic complexity lies in write operations, while read operations remain relatively simple.

Another consideration with read operations is the frequent need for joins. Join operations are essential in relational databases. However, when applying the Database per Service principle, most join operations are eliminated, requiring communication with the responsible service to fetch the necessary data. For example, if an event organizer wants to view a list of each ticket along with the corresponding phone number of the user who purchased it, the Tickets Service would first fetch all tickets for the event. Then, for all associated users, we would need to communicate with the Users Service to retrieve their phone numbers. Although we could add a PHONE_NUMBER field to the TICKET table, such a design becomes cumbersome as the application evolves. For instance, if we want to add a feature where the vendor also sees users' emails, the database design grows increasingly complex, making it challenging to extend our service. If you've worked extensively with microservices, you may recognize this as a common problem. The simplest solution in practice is to accept the latency and fetch each user’s details separately or in batches by passing a list of IDs.

Simple Write flow

Simple Read flow

To conclude, write operations are significantly more complex than read operations, and each microservice typically contains far more write logic than read logic. The CQRS pattern can be beneficial in addressing this complexity.

CQRS pattern

CQRS stands for Command Query Responsibility Segregation. As the name suggests, this pattern segregates commands and queries. Commands are for insertion, update, and deletion operations, while queries are purely for reads and joins. Here’s how it works: each microservice handles write logic within its own database, as in traditional distributed systems. However, we introduce a separate service solely for querying, which has a different data source and contains groups of relevant resources often joined to construct a single entity. The Query service's database can differ completely from the Command database. The Query service is responsible only for fetching data. When the command service updates its database, it publishes an event containing the new or updated model. The Query service consumes these events and updates its database accordingly. When data needs to be fetched, the application queries the Query service, which can efficiently retrieve data and perform joins as needed.

This model offers numerous advantages:

Optimized Datastore
We can select different database technologies for Command and Query databases. For example, if we find that a NoSQL database is more suitable for fetching data, we’ll use a NoSQL database for the Query service. Conversely, if a well-structured, constraint-oriented database is more appropriate for the Command service, we can use a relational database like MySQL or Oracle.
Better scalability
Consider a Posts service in a social media platform like Instagram. The read-to-write load ratio is enormous. A typical user might create a new post once a week, month, or even year, but could view dozens, hundreds, or even thousands of posts daily. It doesn’t make sense to scale a service that handles both reading and writing equally. With CQRS, we can scale each microservice according to its load, leading to higher performance and reduced costs.
Single Responsibility
Each service’s responsibility is clearly defined and distinct from others. A good engineer will know exactly where to make changes when a new requirement arises.

CQRS in Practice

In this example, we delegate all Command responsibilities to the Tickets Service and Users Service, as before, but we introduce a third service dedicated to querying data. We’ll use a MySQL database for the Tickets and Users services and MongoDB for the Query service

Now, whenever a user purchases tickets, the Query database will have all the necessary information.

Trade-offs with CQRS

There is a trade-off, of course. Unlike traditional CRUD operations, CQRS provides eventual consistency rather than immediate consistency, meaning that data will eventually be consistent across services but not necessarily right away. If you cannot accommodate this delay, CQRS may not be suitable for your application.

We have learned how the CQRS pattern works and when to use it. Next, we will explore Event Sourcing, which often goes hand in hand with CQRS.

SAGA Pattern

Shredded Mustard — Mon, 28 Oct 2024 13:42:33 +0000

When dealing with scalable applications, things can get messy quickly. If you don’t follow established programming principles and structured design patterns, adding new features can become hectic and time-consuming. You may not realize exactly when things went wrong, and you may be uncertain about the steps needed to fix them. This is especially true in distributed systems, where following well-structured design patterns is essential.

In distributed systems, a crucial concept in practice is Database per Service. This means each microservice has its own exclusive database, for which it alone is responsible, and no other microservice can access or modify it in any way. This provides significant programming benefits like improved abstraction, loose coupling, separation of concerns, and adherence to the single responsibility principle. However, this can lead to common challenges when managing transactions.

In traditional single-database systems, we often perform multiple database operations transactionally. A transaction has a beginning and an end, and as long as the transaction remains uncommitted (unfinished), everything done within it can be rolled back. All operations within a transactional context will either succeed together or fail together. Once we commit the transaction, there is no going back. This is known as an ACID transaction. It ensures consistency across our system and guarantees that all tables or entities are updated correctly, preventing misinformation. If we want to undo a committed transaction, we must manually perform another transaction to reverse it.

In distributed systems, things become more complex due to the principle of separation of concerns. Often, we cannot maintain a transaction across multiple microservices. The best solution to this is to avoid using transactions altogether, though designing a database to eliminate transactions entirely can be complex and demands critical thinking. However, complete avoidance of transactions is not always feasible.

There are mechanisms to maintain a single transaction across multiple microservices, such as two-phase commits, but these are often slow, require heavy implementation, and can become complex and impractical.

If strong consistency is essential for your application, you must adhere to atomic transaction rules. However, if you can allow for eventual consistency—where data becomes consistent over time rather than immediately—then there’s more flexibility.

A popular design pattern for handling this is the SAGA Pattern.

Consider a ticket reservation web application. A user logs in and wants to return a ticket they’ve already purchased. To process the return, the server must perform three operations:

Update the status of the Tickets to "Available"
Remove the ticket from the user’s acquired tickets list
Issue a refund request

If our application is monolithic, all operations would be performed in a single transactional context, meaning they would either all succeed or fail together. However, in a microservices setup, we must wait for each process to finish to determine success or failure. If the request fails, we need to manually roll back previous operations.

Let’s first visualize our scenario in a distributed system.

The user requests a refund through the API Gateway, which sends a request to the Tickets microservice. The Tickets service changes the ticket status and sends a request to the User Service. The User Service updates the user’s ticket list and sends a request to the Payment Service to issue the refund. The Payment Service communicates with a third-party payment gateway provider to transfer the funds back to the user and updates its own database. While the flow may vary depending on the implementation, the issue of transaction management remains the same.

If any request fails, we need to roll back previous operations to maintain consistency.

The SAGA Pattern

The SAGA pattern addresses this by introducing a rollback mechanism across microservices. It is commonly used in distributed systems where a transactional context spans multiple microservices. The SAGA pattern breaks down a transaction into a series of smaller, individual steps, each managed by a single service. Together, these steps form a distributed transaction. If a step fails at any point in the SAGA, compensating actions (or rollback steps) are used to undo previous actions, ensuring data consistency. This approach applies to transactions that follow a sequence of operations.

There are two types of SAGA patterns.

Orchestration-Based

In an orchestration-based SAGA, a dedicated SAGA orchestrator service manages the transaction. The orchestrator knows which services to call in sequence. If one operation fails, all previous operations are rolled back, and the user is informed of the failure’s reason. To adapt our example to an orchestration-based SAGA, we’d add an Orchestrator service to control the flow.
The Orchestrator executes each step sequentially. If a step succeeds, it moves to the next. If a step fails, it rolls back all previous actions.

Here is how it goes

The Orchestrator service now has control over the entire process and knows how to handle any failure, allowing the user to see exactly what went wrong. However, this pattern introduces complexity and makes the orchestrator a single point of failure. Additionally, any new operation must be configured in the Orchestrator, including its rollback call.

To reduce this coupling, we can use the Choreography-based SAGA pattern with event-driven architecture.

Choreography-Based

As the name suggests, the Choreography-based SAGA pattern lets multiple services work together in a choreographed manner, with no central coordinator. Each service involved in the transaction triggers the next step and listens for events to decide its next action.

If an operation fails, it triggers an event to roll back previous operations. For example, if the Users service fails to return the ticket, it publishes a "User Ticket List Update Failed" event, which only the Tickets Service listens to and uses to perform its rollback. If the Payment Service fails, it triggers a "Payment Refund Failed" event, which both the Tickets and Users services listen to, performing their respective rollbacks. To achieve this, we need a Message Broker.

Now our happy flow looks like this

Now to add the SAGA pattern mechanism

Now, in the successful flow, each service listens for the preceding service’s success event. For instance, the Users service listens for a "Ticket Return Successful" event, while the Payment Service listens for a "User Ticket List Update Success" event. After the Payment Service completes processing, it notifies the user by publishing to the notification service.

In the failure flow, each service listens for the failure event of the preceding service, triggering a chain of rollback events as needed.

SAGA Pattern review

The SAGA pattern divides a transaction into distributed steps, where a failure in one service triggers events that roll back previous steps, ultimately achieving consistency across the system.

The SAGA pattern is ideal for microservices architectures requiring eventual consistency and flexibility in managing long-running, distributed transactions, especially in domains like e-commerce, travel, banking, and other transactional applications. However, the added complexity in designing and monitoring SAGA workflows should be considered based on your application’s requirements.

When dealing with distributed systems, cross-microservice transactions should generally be avoided. While design solutions exist, they are often complex and add application overhead. Unlike in monolithic transactions, rolling back a distributed transaction is not straightforward and requires careful implementation and thorough testing. Therefore, whenever you apply such a pattern, proceed with caution and thoroughly test your implementation.

The next pattern we will review is CQRS

Introduction to Event Driven Architecture

Shredded Mustard — Sun, 27 Oct 2024 09:25:57 +0000

Event-driven architecture has gained significant popularity in recent years. Advancements in technologies like message brokers and distributed systems have enabled architects to design software in an event-driven manner. But what exactly is event-driven architecture, and why is it so impactful? To understand its purpose and benefits, we first need to review the traditional request-response model.

Traditional Request-Response Model

Let’s take a basic example of a ticket reservation web application. Users log in to purchase tickets for an event they want to attend. The browser collects information from the user and sends a request to the server. The application then performs the following operations in sequence:

Checks if the requested number of tickets is available.
Locks the tickets for the user, so no one else can request the same tickets simultaneously.
Sends a request to a payment service and waits for a response confirming the payment.
Changes the status of the tickets to "Acquired."
Assigns the tickets to the user’s account.
Finally, returns a response to the user with the ticket details and confirmation.

In a typical distributed microservices environment, this flow might look like this:

We assume that two microservices are running on our backend. The Tickets microservice accepts the user’s HTTP request, queries a Mongo database to check ticket availability, and updates the ticket status to "Pending". It then sends a payment order to the Payment microservice over HTTP. The Payment service issues a payment request to a third-party payment gateway provider. Once confirmed, it notifies the Tickets service, which finalizes the transaction by updating the database and assigning the tickets to the user. Finally, the Tickets service responds to the user with a confirmation.

All these operations must be completed for the ticket to be successfully reserved.

While this approach works well in theory and often in practice, a few challenges arise:

User wait time: The user must wait for all operations to complete before receiving a response.
Thread blocking: In a multithreaded environment, each thread is locked for several seconds or minutes while waiting for the processes to complete. This can cause issues during periods of high traffic, such as when tickets are first released, potentially leading to thread exhaustion.
Unpredictable latency: The payment gateway, especially if it involves user authentication via OTP, can introduce unpredictable delays.

For small-scale applications, these issues may not be critical, and adopting an alternative approach could seem unnecessary. However, as our application scales, latency and performance bottlenecks can emerge.

We could reduce User wait time by responding to the user before sending the payment request.

While this approach solves one problem, the server still remains blocked until the entire process completes, meaning it doesn’t solve the thread-blocking issue. We are still processing the request synchronously.

Another approach to mitigate this is horizontal scaling. Horizontally scaling microservices might alleviate some issues, but it introduces cost concerns. Additionally, it doesn’t fully address database congestion, especially when dealing with large datasets or additional operations like checking a user's age for an age-restricted event or updating loyalty points. Also, we can only scale our own microservices but we have no control over third party service providers.

The flow becomes more complex if we want to add new operations. Suppose we wanted to add a recommendation service that suggests new events based on the user's previous selections. Now, we need to send another request to a Recommendation microservice. Each new operation adds complexity and tight coupling, leading to a highly interconnected application.

This is where event-driven architecture becomes a game-changer.

What is an Event

An event represents a fact, action or a state change. Imagine you're a manager at a financial company. One morning, you ask an employee, "I need a report on the quarterly income for Company X". The employee replies, "It will be ready in two hours". You go to your desk and proceed with your daily tasks, eventually forgetting about the report until you receive an email from the employee two hours later with the requested report. This scenario can be considered event-driven; you didn’t wait idly for the task to complete.

Asking the employee for the report can be imagined as triggering an event.

An event is always immutable. Events cannot be changed once they are sent. Events can also be stored indefinitely, and can be consumed by multiple consumers concurrently.

What is a Message Broker

A message broker is a piece of intermediary software between applications and services, allowing them to exchange messages and communicate seamlessly. Unlike protocols like HTTP, message brokers format the message and store it in a message queue. This queue acts as a buffer, storing messages until consumed by the receiving application. The key components of a An event-driven model are:

Message Producers produce the event (send the message to the message queue).

Message Queues store the message

Message Consumers retrieve and process messages from the message queue. Multiple consumers can read the message concurrently from the queue.

Message Brokers Facilitate communication between producers and consumers, adding features like message routing, filtering, delivery acknowledgment, and transformation.

Applying Event-Driven Architecture to Our Example

Now, let’s revisit our ticket reservation web application. If we could notify the user that we’re processing their request and promise to inform them once processing is complete via email or notification, we could solve the earlier issues and process the logic asynchronously.

To reconstruct our ticket reservation flow as event-driven, let’s break down each step to understand how the new model operates.

Step 1: Ticket Reservation Request
The user sends the request to an API gateway. The Request is processed by the Tickets service. The service queries the database to check the tickets' availability and change the tickets status to "Pending". The service then sends a message to the message queue. The message can be formatted as JSON or any other formatting technology according to the message broker technology. Finally, the server responds to the user saying that his reservation is "Accepted" and being processed.

The above diagram may seem unusual since nothing yet comes out of the message queue. However, this illustrates a core concept in event-driven architecture: when a producer sends an event, delivery is only guaranteed within the context of the message queue; which means that the consumers are decoupled from the producers. The broker acknowledges only that the message has been stored , and the producers don’t need to know which service will consume it or how it will be processed. This decoupling enables flexible, scalable systems, as we’ll see later on.

Step 2: Payment Processing
The Payment service consumes the message from the queue, initiates a payment request to the third-party service, and waits for confirmation. Once received, it produces an event indicating successful payment.

Step 3: Ticket Confirmation
The Tickets service consumes the event (Reads the message from the queue), changes the status of the tickets to "Acquired," and completes the reservation process by associating the tickets with the user’s account. Here, the Tickets service acts as a consumer.

Step 4: User Notification
Although the ticket reservation is now finalized, the user still needs to be informed. The final step in the process could involve producing a notification event, which can be sent to an email service or another system to alert the user.

Request-Response vs Event-Driven

Now that we reviewed each approach, let us compare between them and see what we gained.

1. Synchronous vs Asynchronous

In the request-response model, the processing happens synchronously, which means that the user has to wait for the response, and if the server never replies this usually indicates a problem.

In the event driven model, the producer doesn't need to wait for the response or even know how and by who the event will be consumed. The delivery of the message to the consumer is outside of its responsibility and control. This allows the producer to move on to the next task instead of waiting for a response it doesn't actually need.

2. Inversion of Control

In the request-response model, the sender, first, needs to know about the receiver. If the sender needs to communicate with multiple receivers, it needs to send a request to each receiver each with their own unique parameters. This makes the sender depend on the receivers of the request.

In an event-driven model, the producer is not even aware of the consumers of the event. This completely decouples the sender from the receivers.

3. Loose Coupling

The reason event-driven architecture and microservices go very well together is the fact that event-driven architecture changes the dynamics of dependencies dramatically. As our application logic gets more complicated, we can add more and more consumers that consume the same event with the same structure. The producer service is completely decoupled from consumers.

Event-driven architecture offers numerous benefits, changing the way we view traditional applications. However, improper implementation can lead to significant issues. Before switching to such a model, it’s important to assess its suitability for your application. In the next articles, we’ll dive into popular event-driven architectural patterns and when to apply them. Read about the first important design pattern, "The SAGA Pattern," in the next article.