DEV Community: Alex Alves

An Overview of Distributed Programming and Systems

Alex Alves — Mon, 29 Jan 2024 13:25:12 +0000

A simple way to define/describe is the same system can be executed in many environments, where you can distribute the execution/process of some flow. Besides this, you need to build algorithms that can be executed in this environment. Let’s see some overview concepts, below!

Why is this important?

When you work with high processing power, distributing it is an excellent way to improve the utilization of computational resources without spending all your resources on the machine. You allow your system to process things in different environments/nodes/machines/servers.

When we use a non-distributed architecture, we have just one POD/VM (the first called) to execute all the flow:

On the other hand, when we use distributed architecture, we can divide the responsibility to execute all the processes. Where we use all PODs/VMs to do this:

Distributed programming is the art of solving the same problem that you can solve on a single computer using multiple computers. [Mikito Takada]

Pros

Horizontal Scalability
Availability: failures do not impact other modules/services
Performance: distributed tasks may get a great result
Resilience: distributed processes and data may be more failure resistance

Cons

Complexity: requires advanced knowledge about distributed programming, network, and resource manager
Consistency: mainly in a high concurrency
Communication: may introduce latency when transmitting some data
Infrastructure: you should know about networking to define the best way to reduce latency, cost, and degradation

Topologies

When we talk about distributed systems, we talk about strategies for following, and we need to answer the following question by ourselves:

* Are we confident in the system/service, to process some piece of journey/flow correctly?

If the answer is “yes”, we don’t need some app to control the flow, we need to accept it, and we will get a decentralized system. If the answer is “no”, we need something to control all the journey/flow, to tell which service/environment will execute something, and then we'll get a centralized system. Both are distributed but in different ways (or Topologies). Let’s talk about it!

Broker (Decentralized)

The message flow is distributed across the journey/components. This topology is very useful if you do not want a central event orchestration. You only need to broadcast the message, someone consumes it and propagates another message, without other components knowing about the process. It’s all about the chaining of events. And, to see what happens if an event is published, you need an “alive documentation” to demonstrate it, like an EventStorming.

Points to Consider
🔴 Monitoring complexity
🔴 Debbuging complexity
🔴 Error handling
🟢 Decoupling
🟢 Flexibility
🟢 Scalability
🟢 High throughput

Mediator (Centralized)

It is useful if you need some level of orchestration, like when you will do a rollout process (migrate functionalities of legacy to new application). Here, you have to pay attention to SPOF (Single Point of Failure), because all the processes will depend on your orchestration. Here we maybe see the Request-Reply pattern, in which the orchestration needs to wait for the service reply when he calls it.

Points to Consider
🔴 Single Point of Failure (SPOF)
🔴 Scalability limitations
🔴 Performance overhead
🔴 Over-coupling
🟢 Clearer visibility
🟢 Simplified error handling
🟢 Improved security

Sagas

This is a pattern that helps us to do rollbacks in “distributed transactions”, used in both topologies like Choreographic (to broker topology) and Orchestration (to mediator topology).

Choreographic
When a process fails, the service publishes an event of failure, and all interested services consume it to do something (like "revert"), using the "compensation event" concept.

Orchestration
Is almost the same thing as Choreographic, except it controls the failure service and calls directly all other points to "revert".

In other words, we don’t have a real rollback, just a “compensation event”, because the side effects have already happened. And, looking at this we observe how this strategy impacts the system coordination and consistency. As the topology models the components' interaction, let’s see below how can we use the concurrency control to guarantee the consistency and integrity of each service.

Concurrency Controls

It's something like a mechanism that ensures to control of multiple concurrent transactions. It thinks about the conflicts and how we can maintain data consistency.

Optimistic

This strategy considers the best scenarios possible because it assumes will happen infrequently and allows transactions to happen unsynchronized. However, you need to guarantee that the conflicts that will have don’t affect your system. Whether through exceptions or the ordering of actions that will happen, for example.

Pessimistic

On the other hand, you can use a blockade strategy. In this case, we assume that conflicts will happen frequently and we block the part of the code for the first client to use it. We can use thread local safe or distributed lock for this, using something like Redis.

The best practice to choose will depend of to know which how you will send and receive requests. For this, let’s see some high-level protocols.

Communication Protocols

Let's see some useful protocols that we can use in Distributed Architecture:

HTTP/HTTPS

Client-Server communication: allows to segregate tasks between client and server
Stateless: each request is independent
Scalability: easy server replication

RPC

Transparent Integration: call remote functions as local functions
Network Abstraction: you do not need to care about the subjacent networks
Concurrency Manager: allows control of transactions and distributed locks

MQ (Message Queue)

As AMQP or MQTT
Decoupling and Asynchronism: A component can send a message without waiting for your processing
Fault Tolerance: Ensure message delivery even if systems are temporarily unavailable
Load Balance: Allows to distribute messages in many server instances
Integration of Heterogeneous Systems: Programming language agnostic

Now, let’s see how we can use all these things together in an architecture/code design

Architectures/Patterns

Client-Server

It is when you divide the application into two, client and server. The client is the part that is responsible for interacting with the end user, and the server provides all the client requests. The client can be software in a mobile phone or web application, and the server can be a dedicated machine. We need to pay attention to a single point, which every request on the server needs a response, so the clients wait for the server to process the request.

Service-Oriented Architecture

A way to goal organize a system with interconnected services. Service in this context is autonomous, independent, and exposed. Following the Distributed System, we can distribute different tasks to different services.

Microservices

This pattern almost follows the same way as SOA, but the difference is how the Consumer Layer uses the services. In this case, we do not need the ESB, the services are so independent that the Consumer Layer uses it directly and their infrastructure is just for itself.

Event-Driven Architecture

We can do this pattern in different scenarios, from Monolithic to Microservice. The idea is to use the MQ protocol to do async communication, in which we can use the load balance strategy and redirect all the charges to the infrastructure, like Message Brokers. This is useful for distributing the same task process to different server instances.

Actor Models

This pattern is based on Actors, in which each actor is an independent unit and your interactions are by messages. This kind of architecture works with a stateful strategy, and for this is very performative.

Conclusions

We need to recognize this way brings us some benefits, like resilience, scalability, and performance. But we need to see the complexity and challenges that exist, too, like architectural decisions, concurrency controls, and protocol management. For this, we can look at the CAP Theorem, which may help us choose between consistency, availability, and fault tolerance.

The comprehension of these trade-offs helps us to decide the best way to follow and the best alignment between the tech team and product team.

PS: This article brings us an overview of the distributed system, we did not go deep into any concepts. For this, we require a deeper study.

Abbreviations / Significates

SPOF: Single Point Of Failure
POD: Kubernetes abstraction that represents a group of one or more application containers, and some shared resources for those containers
VM: Virtual Machine
RPC: Remote Procedure Call
AMQP: Advanced Message Queuing Protocol
MQTT: Message Queuing Telemetry Transport
ESB: Enterprise Service Bus

References

Aplicando Conceitos TDD em .NET

Alex Alves — Thu, 04 Jan 2024 18:17:26 +0000

O Test-Driven Development é uma prática onde desenvolve-se um projeto ou parte dele pensando, primeiramente, em testes. O próprio nome traduzido já é uma auto-explicação, Desenvolvimento Dirigido por Testes. Tal prática tornou-se popular e, consequentemente, muito utilizada devido a garantia de funcionamento e qualidade de itens de desenvolvimento entregues.

Neste contexto, surge algumas perguntas, tais como:

Qual a vantagem em desenvolver assim?

Maior cobertura de testes no código
Garantia que os requisitos solicitados funcionem
Qualidade do código (onde o próprio código de testes traz informações sobre a qualidade do código produzido)
Garantia da entrega de um produto com maior qualidade

Qual a diferença entre utilizar TDD e criar os testes após o desenvolvimento?
A qualidade final de um código está com base nos testes criados. A diferença entre a maneira que cria-se os testes está no retorno de feedback do teste para o desenvolvedor.

Se um programador começa a criar os testes primeiro, ele recebe feedback constantemente, pois o mesmo consegue dividir seu trabalho em pequenas tarefas. Onde o mesmo consegue ter os seguintes passos:

Escrever um teste
Implementar um pedaço da funcionalidade
Executar o teste

A cada teste escrito ele recebe um feedback. E, com isso, fica fácil de alterar o código feito.

Quando o programador desenvolve primeiro e depois, ao final, cria os testes, ele não recebeu nenhum feedback. E ao criar os testes, caso tenha que realizar alguma alteração, a mesma pode ser complexa e trabalhosa e levar um tempo maior para ser feita, custando cara.

Isso irá diminuir minha produtividade?
Onde pratica-se o TDD, obviamente o desenvolvedor irá gastar um maior tempo criando testes, mas não significa ser menos ou mais produtivo.

Tal tempo gasto, é gasto de maneira inteligente. Pois os testes criados, irão perdurar até o fim do projeto/código, ou seja, irá durar por muito tempo. E, além isso, podem ser executados a qualquer momento e quantas vezes for necessário, visando a garantia do código feito.

Também, ao criar/pensar em testes primeiro, facilita as modificações no código que for preciso, como citado acima.

Como funciona o TDD?
TDD é uma técnica de desenvolvimento baseado em um ciclo, independente da linguagem de programação, composto por passos que são:

Criar um cenário de teste baseado em uma regra de negócio
Criar o teste baseado no cenário
Executar o teste criado (irá falhar)
Escrever o código até que o teste apresente sucesso
Refatorar o código desenvolvido
Executar o teste novamente

Basicamente, é o que está apresentado na imagem abaixo:

🔧 Mão na massa!

Regra de negócio
Antes de tudo, é preciso ler e entender qual é a regra de negócio que será solicitada. Sendo assim, tem-se:

Dado itens de produtos, é necessário permitir o cadastro de tais itens. Onde cada produto terá as seguintes informações com seus respectivos tipos:

Nome [string]

Código do produto [string]

Descrição [string]

Valor de custo [decimal]

Valor de venda [decimal]

Deve-se considerar as seguintes regras, ao realizar o cadastro de algum item:

Os campos obrigatórios serão: 'Nome', 'Código do produto' e 'Valor de custo'

O campo 'Valor de custo' deve ser maior do que zero

A quantidade de caracteres do campo 'Nome' deve ser inferior a 100

A quantidade de caracteres do campo 'Descrição' deve ser inferior a 500

O campo 'Código do produto' deverá conter, exatamente, 6 caracteres

O 'Valor de venda' será calculado automaticamente, onde constituirá do 'Valor de custo' + 15% de seu valor

Cenário de teste
Dado a regra de negócio acima, necessita-se criar os cenários de teste. No caso, cria-se apenas um como demonstrativo, sendo ele:

Deve-se testar um produto válido, onde apresenta-se um item composto pelos seguintes dados:

Nome: Notebook Acer

Código: 849071

Valor de custo: R$ 1.843,25

Deve-se validar todos os campos, de acordo com as regras, onde o resultado deverá ser sucesso.

Construção do teste
Com o cenário de teste já definido, pode-se criar os testes via código, como é exibido abaixo:

public class ProdutoTest 
{
    [TestMethod]
    [DataRow("Notebook Acer", "849071", 1843.25)]
    public void SalvarProduto_ComDadosValidos_Sucesso(string nome, string codigo, decimal valor)
    {
        //action
        Produto produto = new Produto(nome, codigo, valor);

        //assert
        Assert.IsNotNull(produto);
        Assert.IsTrue(produto.Nome == nome);
        Assert.IsTrue(produto.Codigo == codigo);
        Assert.IsTrue(produto.ValorCusto == valor);
        Assert.IsTrue(produto.ValorCusto > 0);
        Assert.IsTrue(produto.Nome.Length <= 100);
        Assert.IsTrue(produto.Codigo.Length == 6);
        Assert.IsTrue(produto.ValorVenda == 2119.7375);
    }
}

Analisando o código, nota-se o seguinte:

Para que um produto exista é necessário informar o nome, código e valor de custo do mesmo. Com isso, tais campos são cobrados no construtor da classe e a mesma não deve permitir ser instanciada sem tais campos, atendendo a primeira regra de validação.
É testado se o campo referente ao Valor de Custo é maior do que zero, atendendo a segunda regra de validação
É testado se o campo referente ao nome tem o número de caracteres menor ou igual a 100, atendendo a terceira regra de validação
É testado se o campo referente ao código do produto tem o número de caracteres igual, exatamente, a 6, atendendo a quinta regra
É testado se o campo referente ao valor da venda é igual a 2119.7375, valor obtido pela fórmula descrita na regra de negócio, atendendo a sexta regra

Execução do teste
Após o teste já estar desenvolvido, pode-se executá-lo. Como ainda não tem-se nenhum código criado, o teste não será executado e irá dar erro de compilação.

Escrita do código
Após analisar o teste criado, deve-se desenvolver o código que irá atendê-lo, ou seja, criar a classe Produto com todos os atributos esperados.

public class Produto 
{
    public Produto(string nome, string codigo, decimal valor) 
    {
        Nome = nome;
        Codigo = codigo;
        ValorCusto = valor;
    }

    public string Nome { get; }
    public string Codigo { get; }
    public decimal ValorCusto { get; }
    public decimal ValorVenda { get; } = ValorCusto * 1.15;
}

Refatoração do código escrito
Como o código desenvolvido está atendendo perfeitamente as regras de negócio, não será necessário refatorá-lo. Apenas se houver alguma mudança na própria regra.

Execução novamente do teste
Por fim, ao executar o teste novamente, nota-se que o mesmo irá ter um resultado de sucesso.

Conclusão

Seguindo tais práticas, garante-se que o código desenvolvido sempre estará coberto por uma certa quantidade de testes. Onde, a cada alteração do código pode-se assegurar que o mesmo estará em funcionamento.

É válido, também, considerar que os teste não são imutáveis. Pois as regras de negócio podem sofrer alterações.

Referências

User Secrets in your .NET Project

Alex Alves — Thu, 11 May 2023 21:07:42 +0000

Genarally, we need to include some sensitive data in our appsettings.json file, like a connection string that content a username and password or a something kind of private key:

{
  "ConnectionStrings": {
    "MongoDb": "mongodb://mongoadmin:secret@127.0.0.1:27017/aurora/?authSource=admin"
  }
}

However, the exposure of these credentials is not safe and is not a good practice either. And, when a developer get your repository, maybe they need to do adjust your docker images or change the sensitive data/connection strings to attend the project.

Let's see how can we solve this! 🤓

User Secrets

This is a way that allows us to manage sensitive data for .NET on local development.

So, to configure it in your project, you'll need something like below in your Startup/Program class:

WebApplication 
    .CreateBuilder(args)
    .Host
    .ConfigureAppConfiguration((context, configurationBuilder) =>
    {
        configurationBuilder
            .AddUserSecrets(Assembly.GetExecutingAssembly())
            .AddEnvironmentVariables();
    })

Now, we need to configure our secrets, right?
First, let's open the startup project folder and, next open the terminal, and follow the commands below:

To enable secret storage:
dotnet user-secrets init
To add a secret:
dotnet user-secrets set "ConnectionStrings:MongoDb" "[YOUR CONNECTION]"
To add a secret in a list:
dotnet user-secrets set "PrivateKey[0]" "[YOUR KEY]"
dotnet user-secrets set "PrivateKey[1]" "[YOUR KEY]"
To list the secrets:
dotnet user-secrets list
To remove a secret:
dotnet user-secrets remove "ConnectionStrings:MongoDB"

Conclusion

Never store passwords or other sensitive data in source code
Production/Staging/Development should not be used for local test/development
Sensitive data should not be deployed with the app
You should store sensitive data in a protected environment/tool, like Azure Key Vault

References

https://learn.microsoft.com/en-us/aspnet/core/security/app-secrets?view=aspnetcore-7.0&tabs=windows

Message Types in Domain-Driven Design

Alex Alves — Mon, 02 Jan 2023 12:19:51 +0000

Before we start to talk about message types, let's talk about what is a message.

A message is any intention of transmitting and/or processing information.

A message, you can produce and consume from message brokers, message queues, log streams, or in-memory. But, the way to do it is not important, but the message intention, and information.

Having that clear, let's identify what types of messages we have and how we can produce them! 🔥

Command Messages 🎯

The goal of this message type it invokes a functionality in another application. Usually, it represents an action or intention to produce some effects in our system and is started for an external agent, like a user.

The important concept it's that this message has a specific recipient. And every command generates a side effect in Aggregate/Context

Event Messages 🔊

A difference between commands and events is that events do broadcast to anyone interested.

Domain Events

When we talk about DDD we talk about Domain/Business. So, when we triggered a command, this command generates a side effect in our System/Domain. This side effect we may call Domain Events, which represents business facts.

Relevant points:
- Into a system, this event type may sensibilize other contexts or your own context/aggregate.
- Our Aggregate may be produce 1-N business facts/events.

Integration Events

Generally, if other external agents were interested in something of any journey, it's common for the context to produce an event and contains all the necessary information. An example of it is when our system needs to produce events in a Data Lake. The Data Lake is out of our domain and tech.

Notification Events

Similar to Integration Events, the difference is the content. The Notification Event has just an ID, and your goal is to notify all interested that something happened. If the interested wants to get more information or trigger another process, doesn't matter.

Query Messages 🔛

When we need to recover some data from our system, we need to ask our application to recover it. We can use messages to do it. But, this is a little strange if we think about the async environment. So, for async applications, we can use the request-reply async pattern for this.

Conclusions

Messages is technology agnostic
It's import to identify your message intention
Your journey may be more fluid and clear
If you decide to use a Message Broker, Queue, or Log Stream, it's important to direct each message to your specific channel, liek:
- Message Brokers is an excellent tool for Commands and Domain Events
- Log Stream is an excellent tool for Integration Events
Request-Reply Async pattern is a good idea for ascyn environment

References

Modeling your Domain with Event Storming workshop

Alex Alves — Thu, 17 Nov 2022 20:43:09 +0000

Event Storming is a technique that helps us to see our business from another optical. Well, I want to say that Event Storming shows us a map that contains: product intents, their comportments, and how the interactions occur... all of this in our software.

So, I think you understand a little, right? Don't worry, we'll talk more about it in this article.

Continuing... why do I talk about Event Storming? First, because I believe that the best way to build and model a software/product is by following the DDD (Domain-Driven Design) philosophy; and second, because I think that DDD and Event Storming fit perfectly.

Let's Start!

In a few words, Event Storming helps us to map the actions that occur in our software, including which side effects these actions produce. And, consequently, these side effects may trigger other actions. So, we'll build a flow, that helps us to see the boundaries of business intentions, what kind of intentions we have, and more.

The image below is an example of Event Storming output:

Let's explain this image:

🟡 Yellow cards:
To start a flow/journey in our system we need some sensibilization, which is triggered by some users or other external systems. This sensibilization occurs externally. So, to represent it we use this card type named "Actors".
🔵 Blue cards:
When the actor uses our system he triggers some actions, like making a login, or buying a shirt in an e-commerce system. So, to represent this comportment, we use this card type named "Commands".
👛Light Pink or Light Yellow cards:
When actions/commands are triggered, these commands will enter our system/domain, causing some sensibilization. To represent this behavior/domain we use this card type and we named it "System" or "Aggregate".
🟠 Orange cards:
Pause for this card!⚠️ As I said before, the commands will enter our system, right? So, when this happens, these actions will generate some side effects which are the most important thing that we need to map, because this show us that our system did something, something changed... So, when something (command) happens, our system runs some comportment, i.e. it execute a specific situation that exists in our business rules, our domain, and then a side effect will be generated. So, this type of card represent these side effects, and we name it "Events".
🟣 Purple cards:
After these events occurred, they may trigger other specific commands. For it, we have some rules to decide it. So, these rules are represented by this card named "Policy".
🟢 Green cards:
Beyond policy, after the events occurred, they may sensibilize a UI either. So, something happened and whoever started the journey may know what happened, e.g. see some data modified. To represent it, we use this card named "View", "Projection" or "Read Model".
🔴 Pink/red cards:
When we have doubts about some behaviors in our journey, we can pin an alert flag and postpone them. To represent it we use the card type named "Hotspot".It's important to solve these doubts at some point.

That's it, now we know more about Event Storming. But running an Event Storming session may have its difficulties like: not all participants will be kept engaged, sometimes may be boring, not everyone will know what they want from the product... and, explaining how Event Storming works may be confusing at the beginning and achieving this flow draw may be complex and difficult, but it's worth it! Let's talk about the workshop!

The Workshop

Exists many other techniques to do this workshop, but in this particular session, I'm going to talk step-by-step about an experience that works for me.

Instead of running it with many people, from different areas, try to make a short group with people of the same area, including stakeholder, tech people, and managers.
When you gather this short group, do an overview about the Event Storming, show the goals, the advantages, and the difficulties that may occur .
From here, I think that we have a key to running it successfully:

3.1. Instead of trying to start to draw the flow, propose to your group to brainstorm the commands/actions that may occur in the system/product. For non-technical people is easier thinking about the actions, as many of they start from a user and tend not to be a technical biased, than about the flow itself. Below you can see an example:

3.2. After you have mapped your commands, you can order them chronological. This will help you to see where would start and end the flow:

Note that, in this example after we ordered the commands, we have two paths. It means that looks like we can have two different flows/journeys in this case (although we thought that it was just one).

3.3. Finally, we can adjust these commands and use all other Event Storming cards type to fill the flow. Take this this moment to explain more about the Event Storming, specially the cards meaning and importance, and to envolve more the group to help you to build the flow:

Look that we have two different contexts. This happens because after we mapped the commands, we identified two flows. It's a strong signal that may be different things, but it's not a sure think. Only after discussing it in the Event Storming session we agreed that made more sense segregated into two contexts.

So, that's how we can map our journeys!

PS: This Event Storming is a reference to the Aurora Project

Domain-Driven Design & Event Storming

Why is this a perfect match?

DDD shows us how we can build better products, for that we need:

Ubiquitous language: is a unique language to enable developers, stakeholders and managers understand each other in prol to find the product actions. The Event Storming cards helps this understanding.
Boundaries: is how we separate contexts. Note that when finished the workshop, we can see two different contexts, which show us what your boundary is.
Context Map: we have all the context mapped and how they communicate with others.

Conclusions

Event Storming is an excellent tool to help us to get a better look at our product and what it does. Additionally, we can use this generated document to build our solution architecture, and how we can transform each context in a project/code. In my opinion, it increases our chance of creating something evolving, and easy to maintain.

Challanges / Next steps

Think about an architecture to attend the Event Storming
Segregate our domain to attend the contexts mapped
Transform all journeys in code

Finally, for enterprises... think about how the squad can work together in this environment. The book "Team Topologies" may help us.

Acknowledgment

None of that would be possible without the incentive, help and teachings of Antonio Falcão.

References

Getting started with .NET Core API, MongoDB, and Transactions

Alex Alves — Sat, 20 Nov 2021 22:46:09 +0000

MongoDB is a kind of NoSQL database. NoSQL is a document-oriented database that is organized as a JSON. Some points about MongoDB:

Performance: It stores a majority of data in RAM, so the query performance is much better here than in a relational database. But it requires more RAM and precise indexes.
Simplicity: Some users consider the query syntax is simpler here than in relational databases. The installation, configuration, and execution are simple and quick to do. And the learning curve is shorter than in the others.
Flexibility: It’s dynamic because it doesn’t have a predefined schema.
Flexibility: It’s dynamic because it doesn’t have a predefined schema.
Transaction: v3.6 and beyond allow using the transaction concept. And v4.0 and beyond allow using the multi-document transaction concept.

Now that you know a little more about MongoDB, let’s go to our goal! This article proposes to teach you how you can build a .NET Core API connecting with MongoDB and use the Transactions with Dependency Injection.

#ShowMeTheCode!

For this example, don’t consider the structure and the architecture, it was built in this way just because I think that it’s easier to explain. So, I won’t explain the layers, folders, and other things in detail, except those needed by the use of MongoDB and the transactions.

The project

In this case, I used the Visual Studio 2019, Community version. So, with the VS installed, we select the “ASP.NET Core Web Application” option and select the API type.

About the base structure

After creating the project, we create five folders like below:

Controllers: responsible to answer the requests
Entities: Domain classes
Interfaces: like contracts, we’ll use it to do the DI and IoC
Models: Classes that we’ll use to receive and return data on controllers
Repositories: Classes with methods that contain the implementation of MongoDB operations

We’ll focus on just Controllers and Repositories folders and the Startup class. If you want to see the complete code, wait for the end of this article.

Installing and configuring MongoDB

Now, we need to install the more important package for our project, that is the MongoDB.Driver.

After installed, we need to modify the Startup class, on the ConfigureServices method, specifically.

Let’s analyze this code:

public void ConfigureServices(IServiceCollection services)
{
    // ...

    services.AddSingleton<IMongoClient>(c =>
    {
        var login = "";
        var password = Uri.EscapeDataString("");
        var server = "";

        return new MongoClient(
            string.Format("mongodb+srv://{0}:{1}@{2}/test?retryWrites=true&w=majority", login, password, server));
    });

    services.AddScoped(c => 
        c.GetService<IMongoClient>().StartSession());

    // ...
}

The ConfigureServices method is where we do our IoC. So, the first statement is where we make the MongoDB connection. Pay attention in this step because we make a Singleton, we do this because the MongoClient instance, in MongoDB, is already a pool connection, so if you don’t use a Singleton, a new pool connection will always be created.

And the second statement is where we declare the IoC, which we’ll use to start a session of the MongoDB Transaction. Notice that, in this case, we make a Scoped, because the transaction life cycle will be equal to the request life cycle.

Creating a base repository

So now, let’s make a base repository that we will use whenever we want to do some operations. First, the base repository class will receive a Generics to identify the entity in our code that represents a collection in the database, like below:

After creating the class, let’s go to declare some attributes that will be important to us:

public class BaseRepository<T> : IRepositoryBase<T> where T : BaseEntity
{
    // some code
}

We have four attributes:

public class BaseRepository<T> : IRepositoryBase<T> where T : BaseEntity
{
    private const string DATABASE = "poc_dotnet_mongodb";
    private readonly IMongoClient _mongoClient;
    private readonly IClientSessionHandle _clientSessionHandle;
    private readonly string _collection;
}

DATABASE: it’s the Constant that represents the name of our database.
_mongoClient: it’s the client interface to MongoDB. Using it we do some operations on the database.
_clientSessionHandle: it’s the interface handle for a client session. So, if you start a transaction, you should pass the handle when you do some operation.
_collection: it’s the name of the collection used by the Generic class.

All these attributes will receive a value on a class constructor:

public class BaseRepository<T> : IRepositoryBase<T> where T : BaseEntity
{
    private const string DATABASE = "poc_dotnet_mongodb";
    private readonly IMongoClient _mongoClient;
    private readonly IClientSessionHandle _clientSessionHandle;
    private readonly string _collection;

    public BaseRepository(IMongoClient mongoClient, IClientSessionHandle clientSessionHandle, string collection)
    {
        (_mongoClient, _clientSessionHandle, _collection) = (mongoClient, clientSessionHandle, collection);

        if (!_mongoClient.GetDatabase(DATABASE).ListCollectionNames().ToList().Contains(collection))
            _mongoClient.GetDatabase(DATABASE).CreateCollection(collection);
    }
}

Look at the constructor parameters. We receive these parameters by dependency injection. We get the values of parameters and we assign them to the attributes that we created.

Another very important thing is the code below of the assignments. If you work with transactions, on MongoDB, and work with a multi-document transaction, we need to create the collection first of all. And for it, we verify if the collection exists in our database and, if not, we create it. If we try to create a collection that already exists, the flow will break and throw an exception.

Now, we will do an important code too, specifically a virtual property that we use to facilitate to do some operations:

public class BaseRepository<T> : IRepositoryBase<T> where T : BaseEntity
{
    // ...some attributes and the constructor...

    protected virtual IMongoCollection<T> Collection =>
        _mongoClient.GetDatabase(DATABASE).GetCollection<T>(_collection);

    // ...some other codes...
}

From the _mongoClient attribute, we retrieve the database and, from the _collection attribute, we retrieve the collection and associate it with the Generic class.

Finally, we build some base operations to change the data in the database like Insert, Update, and Delete:

public class BaseRepository<T> : IRepositoryBase<T> where T : BaseEntity
{
    public async Task InsertAsync(T obj) =>
        await Collection.InsertOneAsync(_clientSessionHandle, obj);

    public async Task UpdateAsync(T obj)
    {
        Expression<Func<T, string>> func = f => f.Id;
        var value = (string)obj.GetType().GetProperty(func.Body.ToString().Split(".")[1]).GetValue(obj, null);
        var filter = Builders<T>.Filter.Eq(func, value);

        if (obj != null)
            await Collection.ReplaceOneAsync(_clientSessionHandle, filter, obj);
    }

    public async Task DeleteAsync(string id) =>
        await Collection.DeleteOneAsync(_clientSessionHandle, f => f.Id == id);
}

Some important things:

We have to pass the _clientSessionHandle for all the methods that do, in fact, the operation, like the InsertOneAsync method.
To do an Update operation, we build a Lambda Expression, using the ID property. And then, we use a Reflection to retrieve the ID value and we make the filter that will use to do, in fact, the operation.

Example of a specifical repository

Now I show you how we can use the Base Repository to do some other specifically repository. In this case, we’ll build an AuthorRepository.

First, we build the class with the inheritances and the constructor:

public class AuthorRepository : BaseRepository<Author>, IRepositoryAuthor
{
    public AuthorRepository(
        IMongoClient mongoClient,
        IClientSessionHandle clientSessionHandle) 
        : base(mongoClient, clientSessionHandle, "author")
    {
    }

    // some code...
}

When we inherit the BaseRepository class, we force it to make a constructor as shown. Look at how we pass the collection name using a string “author”.

Finally, we build some other MongoDB operations, here we build only specific query operations to get data. It happens because we can get specific data in each moment, but the operations that change the database data (like insert, update, delete) will be always the same in this example.

public class AuthorRepository : BaseRepository<Author>, IRepositoryAuthor
{
    // ... constructor code

    public async Task<Author> GetAuthorByIdAsync(string id)
    {
        var filter = Builders<Author>.Filter.Eq(s => s.Id, id);
        return await Collection.Find(filter).FirstOrDefaultAsync();
    }

    public async Task<IEnumerable<Author>> GetAuthorsAsync() =>
        await Collection.AsQueryable().ToListAsync();

    public async Task<IEnumerable<Book>> GetBooksAsync(string authorId)
    {
        var filter = Builders<Author>.Filter.Eq(s => s.Id, authorId);
        return await Collection.Find(filter).Project(p => p.Books).FirstOrDefaultAsync();
    }

    public async Task<IEnumerable<Author>> GetAuthorsByNameAsync(string name)
    {
        var filter = Builders<Author>.Filter.Eq(s => s.Name, name);
        return await Collection.Find(filter).ToListAsync();
    }
}

In this code, I show you some query operations examples:

We can recover one author searching by id.
We can recover all authors.
We can recover all books of a specific author, searching by author’s id, using the Project method.
We can recover some authors searching by name.

Business rules examples with transactions

Let’s use these repositories to do some business rules and use, in fact, the transactions! For this, we’ll make all business rules in the classes located in the Controller folder.

So, we build a class called BusinessController, like below:

[Route("api/[controller]")]
[ApiController]
public class BusinessController : ControllerBase
{
    private readonly IRepositoryUser _repositoryUser;
    private readonly IRepositoryAuthor _repositoryAuthor;
    private readonly IClientSessionHandle _clientSessionHandle;

    public BusinessController(
        IRepositoryUser repositoryUser, 
        IRepositoryAuthor repositoryAuthor, 
        IClientSessionHandle clientSessionHandle) =>
        (_repositoryUser, _repositoryAuthor, _clientSessionHandle) = 
            (repositoryUser, repositoryAuthor, clientSessionHandle);
}

I’ll don’t talk about the Annotations and the inheritances class, because it’s specific for the API. So, look that we use the repositories that we created before and a session _clientSessionHandle that we’ll use to make the transactions. All these attributes we’ll receive on the constructor parameters by dependency injection.

So now, let’s show how we can use a transaction, in fact:

[Route("api/[controller]")]
[ApiController]
public class BusinessController : ControllerBase
{
    // ... some code

    [HttpPost]
    [Route("user")]
    public async Task<IActionResult> InsertUser([FromBody] CreateUserModel userModel)
    {
        _clientSessionHandle.StartTransaction();

        try
        {
            var user = new User(userModel.Name, userModel.Nin);
            await _repositoryUser.InsertAsync(user);
            await _clientSessionHandle.CommitTransactionAsync();

            return Ok();
        }
        catch (Exception ex)
        {
            await _clientSessionHandle.AbortTransactionAsync();

            return BadRequest(ex);
        }
    }

    // ... some code
}

In this case, it’s a POST method, which means that we want to insert a new record on the database. In the method beginning, we need to start a transaction (line 11), then we can make our business rules and, if there is nothing wrong, we can Commit all changes we did on the database. But if something break, an exception will be thrown and the flow will begin on the Catch statement, thereby, we’ll abort all the transaction and nothing in the database will be changed.

Below you can look another example using multi-document transactions. Where we use more than one operation on the database:

[Route("api/[controller]")]
[ApiController]
public class BusinessController : ControllerBase
{
    // ... some code

    [HttpPost]
    [Route("authorAndUser")]
    public async Task<IActionResult> InsertAuthorAndUser([FromBody] CreateAuthorAndUserModel authorAndUserModel)
    {
        _clientSessionHandle.StartTransaction();

        try
        {
            var author = new Author(authorAndUserModel.AuthorModel.Name, new List<Book>(authorAndUserModel.AuthorModel.Books.Select(s => new Book(s.Name, s.Year))));
            var user = new User(authorAndUserModel.UserModel.Name, authorAndUserModel.UserModel.Nin);

            await _repositoryAuthor.InsertAsync(author);
            await _repositoryUser.InsertAsync(user);
            await _clientSessionHandle.CommitTransactionAsync();

            return Ok();
        }
        catch (Exception ex)
        {
            await _clientSessionHandle.AbortTransactionAsync();

            return BadRequest(ex);
        }
    }

    // ... some code
}

The logic is the same as the code shown before.

I expect that you liked this article! If yes, give a like to it. If you have any doubt or any questions comment below.
You can find the complete code here!

Thank you!

DEV Community: Alex Alves

An Overview of Distributed Programming and Systems

Why is this important?

Topologies

Broker (Decentralized)

Mediator (Centralized)

Sagas

Concurrency Controls

Optimistic

Pessimistic

Communication Protocols

HTTP/HTTPS

RPC

MQ (Message Queue)

Architectures/Patterns

Client-Server

Service-Oriented Architecture

Microservices

Event-Driven Architecture

Actor Models

Conclusions

Abbreviations / Significates

References

Aplicando Conceitos TDD em .NET

🔧 Mão na massa!

Conclusão

Referências

User Secrets in your .NET Project

User Secrets

Conclusion

References

Message Types in Domain-Driven Design

Command Messages 🎯

Event Messages 🔊

Query Messages 🔛

Conclusions

References

Modeling your Domain with Event Storming workshop

Let's Start!

The Workshop

Domain-Driven Design & Event Storming

Conclusions

Challanges / Next steps

Acknowledgment

References

Getting started with .NET Core API, MongoDB, and Transactions

#ShowMeTheCode!

The project

About the base structure

Installing and configuring MongoDB

Creating a base repository

Example of a specifical repository

Business rules examples with transactions

References: