DEV Community: Tural Suleymani

Why You Should Use Domain-Driven Design in Your Real Projects?

Tural Suleymani — Tue, 27 May 2025 09:30:09 +0000

In this article, I am going to talk about why you should use Domain-Driven Design (DDD) in your practice. Does it really make sense to use Domain-Driven Design in your practice? This is the most fundamental, the most important question when you start thinking about applying Domain-Driven Design.

Before reading article, it is worth to check my youtube video :

Well, let’s get started.

The first fundamental problem is the two-language problem.

We have the language of business and software development. Business people work in a problem space. They try to learn more about the business, gather more information about it, and pass this information on to software developers.

Software developers, on the other hand, try to understand the business, ask more relevant questions about it, and translate the business language into software development.

Business people speak in a completely different language, but you have a completely different language in your code. So we have two languages. We are working in one company, but we are speaking two different languages. This is the most fundamental problem of classical software development.

When you use Domain-Driven Design, your business language becomes the language of software developers , and you don’t need any translation. You have zero translation — completely zero translation. You’re speaking the same language. We have ubiquitous language in Domain-Driven Design, which helps us speak in a single language. You have business terms, and these terms will be mapped exactly as-is to your software development. You don’t need to think about how to rename a class or how to name a method. What type of signature should it have etc.

This is a big problem because you are translating the business language. But Domain-Driven Design helps you not to translate, to have zero translation, and to use the same language.

Why is using the same language important?

Because the business helps software developers write really good code. On the other hand, developers help the business understand the domain because we are using the same language. We are helping each other.We, as developers, help the business. The business helps developers.

Another benefit is understanding the code. Even business people with limited technical knowledge can easily understand your code and easily identify where to make changes. As a software developer, it won’t take you a long time to add features or make changes to your code because you already know where to make the updates.

Understanding the code is really important — even for new engineers.

Let’s say in our company, we hire a new software developer. I don’t need to explain everything related to the code or the business or provide a lot of documentation, because our code is our documentation.

If you understand our code, you already understand our documentation. You understand our business. Because business is code. Design is code. Code is design.

Domain-Driven Design also helps us centralize business knowledge.

The business helps developers write good code. Developers ask a lot of detailed questions about the business. And the business also learns more about the business — because business people don’t know everything either.

We, as developers, help the business to learn more about the business. And in the end, both developers and business people gain deep knowledge about the business.

This is really important because knowing the business will help you write even better, more understandable, and more readable code.

On the other hand, Domain-Driven Design is an agile, iterative process. In modern software development, it is really important to have an iterative approach. DDD is a 100% iterative process.

And the last benefit for me is, of course, having a toolbox or toolset for development.

We have cool design patterns — strategic patterns, tactical patterns — you can apply. You can use ubiquitous language, bounded context, context mapping, entities, value objects, aggregates, domain events, domain services, modules, repositories, and more as design patterns when using Domain-Driven Design.

So for me, you should apply Domain-Driven Design when you have a complex business domain — like a banking domain or an insurance domain. Please don’t use Domain-Driven Design for technical complexities.

Conclusion

Domain-Driven Design (DDD) is not just a technical framework — it’s a strategic approach to building software that mirrors the business it serves. By eliminating the communication gap between business experts and developers, DDD enables teams to speak the same language, build systems that reflect the domain accurately, and iterate efficiently. Whether you’re building complex domains like banking or insurance, DDD equips you with the right mindset and a powerful set of tools to create scalable, understandable, and maintainable software.

🔑 Key Takeaways from the article

1.The Two-Language Problem

Traditional development suffers from a disconnect between business language and software code.
DDD bridges this gap by unifying both into a shared language.

2.Ubiquitous Language

Everyone (business and developers) speaks the same language.
Reduces confusion and eliminates the need for translation.

3. Improved Code Understanding

Even non-technical business people can read and understand the code.
New developers need less onboarding because the code is self-explanatory.

4. Code as Documentation

The code itself acts as living documentation of the business rules and structure.
Reduces the need for redundant external documentation.

5. Mutual Growth and Collaboration

_Developers help the business refine their domain knowledge.
Business helps developers write better, domain-aligned code.

6. Centralized Business Knowledge

The domain model becomes the single source of truth.
Both technical and non-technical teams gain deep insights into the business.

7. Iterative and Agile-Friendly

DDD naturally supports agile and iterative development.
Makes it easier to adapt to evolving business needs.

8. Rich Toolbox for Modeling

DDD offers powerful design patterns:
Strategic: Bounded Contexts, Context Mapping
Tactical: Entities, Value Objects, Aggregates, Domain Events, Services, Repositories

9. Suitable for Complex Domains

Best applied in domains with inherent business complexity (e.g., banking, insurance).
Should not be used for purely technical complexity.

Want to dive deeper?

Regularly, I share my senior-level expertise on my TuralSuleymaniTech(English version) YouTube channel, breaking down complex topics like .NET, Microservices, Apache Kafka, Javascript, Software Design, Node.js, and more into easy-to-understand explanations. Join us and level up your skills!

NoSQL vs. SQL: Which One to Use

Tural Suleymani — Sun, 04 May 2025 17:52:17 +0000

If you’ve participated in many interviews, you’ve probably heard the question: “What are the differences between NoSQL and SQL databases?” This article will help you clearly understand these differences and when to choose one over the other.

As a software developer, you may have already worked with various SQL databases such as T-SQL, PostgreSQL, MySQL, and others. What’s the first thing you’ve likely noticed? The “rules” or the “schema,” of course. SQL databases have highly structured data models. You create tables with defined columns and rows, strictly following a predetermined schema. Breaking this structure means violating the fundamental principles of SQL. Tables, columns, rows, and data types form the essential building blocks for organizing your data.

On the other hand, when you work with NoSQL databases (non-relational databases) like Azure Cosmos DB, Aurora, or MongoDB, you have the flexibility to frequently modify your data model. NoSQL databases don’t enforce a rigid structure. They provide an “elastic” schema, allowing you to store data in various formats. Instead of sticking to the traditional table representation, you can use document-based, key-value-based, graph-based, or column-based models, among others.

For relational databases, developers usually apply normalization (such as first normal form, second normal form, etc.) to ensure a clear and balanced data structure. As a result, relational databases often rely heavily on multi-table JOINs, aggregations, and complex relationships to retrieve data. However, when datasets become large, it can be challenging and inefficient to manage and retrieve data quickly from relational databases.

Unfortunately, relational databases aren’t inherently designed to handle massive volumes of data. They follow a “scale-up” approach, meaning they require more resources such as RAM, CPU, and GPU to handle increased data.

NoSQL databases, however, are designed for “scaling out.” This means you can distribute and handle data across multiple servers without negatively impacting performance. Many people associate “NoSQL” with “Big Data,” often using these terms interchangeably. Indeed, you can consider the term “NoSQL” a buzzword frequently associated with solving big data challenges.

Behind NoSQL Lies the 3V Principle

Volume
Velocity
Variety

Let’s examine each of these elements individually to understand their importance.

Volume refers to handling massive datasets, reaching terabytes, petabytes, and beyond. Thanks to the “scale-out” design, NoSQL databases comfortably manage vast amounts of data without issues. SQL databases, by comparison, often struggle with such extensive data sets due to limitations in hardware scaling and structured data constraints, making them less efficient for extremely large data scenarios.

Velocity is about throughput — handling massive amounts of simultaneous requests quickly and efficiently. NoSQL databases excel at processing high-velocity data streams, which is crucial for applications like social media feeds, real-time analytics, IoT applications, and more. SQL databases may experience bottlenecks due to their rigid schemas and transaction overhead, slowing down performance in high-throughput situations.

Variety emphasizes schema flexibility. You can utilize any of the schema forms mentioned previously or even choose a schema-less approach entirely. This schema flexibility means NoSQL databases can easily accommodate rapidly evolving data requirements, different data formats, and unstructured or semi-structured data like images, videos, and sensor data. Conversely, SQL databases are best suited for structured and consistent data that doesn’t frequently change.

Let’s explore more internal details between them.

**1. Transactions and ACID Compliance: **SQL databases generally offer strong consistency and ACID (Atomicity, Consistency, Isolation, Durability) compliance. NoSQL databases often sacrifice strict ACID compliance for scalability and flexibility, adopting eventual consistency models.

Complex Queries and Reporting: SQL databases excel in executing complex queries, and multi-table joins, and providing extensive reporting capabilities. NoSQL databases might require additional processing layers or specialized query mechanisms for complex analytical queries.

3. Scaling Approaches: SQL databases typically scale vertically (adding more resources to a single server), while NoSQL databases scale horizontally (adding more servers), providing more flexibility and efficiency for handling large datasets.

Understanding these differences and key characteristics will help you choose the right database solution based on your specific requirements. The best measure for your application is your context. The application context defines which one is perfect for you.

Want to dive deeper?

Regularly, I share my senior-level expertise on my TuralSuleymaniTech(English version) YouTube channels breaking down complex topics like .NET, Microservices, Apache Kafka, Javascript, Software Design, Node.js, and more into easy-to-understand explanations. Join us and level up your skills!

The best way of implementing Domain-driven design, Clean Architecture, and CQRS

Tural Suleymani — Sat, 28 Dec 2024 13:39:58 +0000

Today, I am going to show you the best way learning Domain-Driven Design, Clean Architecture, CQRS, and Software Design Principles in practice. As you might know, there are many interesting books out there that help us gain valuable information about Domain-Driven Design, Clean Architecture, CQRS, and other topics. But nowadays, most developers don’t have enough time to read books or may not want to spend time reading books. Of course, I don’t admire this approach, but it is what it is. And in the era of AI, we want to learn everything fast. That’s why I think using tutorials and real-world examples from GitHub or other sources will help you reach your destination.

For that reason, I’m going to show you one of the best domain-driven design, CQRS, a clean architecture repository. And I hope you will like this repository. I have provided almost all possible best ways of implementing domain-driven design and clean architecture in this repo. You can just check the readme file here.

If you don’t like huge articles, jusat check my video where I explained this article’s content in more detail:

Of course, in GitHub, you may find a lot of interesting repositories which demonstrate Domain-driven design, clean architecture, CQRS, and other stuff. But fortunately, most of them are just garbage for me because they don’t demonstrate real-world examples, they are just pet projects.

But for me, the best way of implementing domain-driven design, is clean architecture should look like this. And that’s why I implemented this repository. So here we have a lot of interesting features.

✨ Features

*Basket Management:
*

Create baskets with basket items.
Underlying architecture for Add, update, and remove items.
Underlying architecture for Calculate total amounts including tax and shipping.

Coupon System:

Underlying architecture for Apply and remove discount coupons.
Underlying architecture for Support for fixed and percentage-based discounts.

Clean Architecture:

Separation of concerns with clearly defined layers.
Independent and replaceable infrastructure.

Domain-Driven Design:

Focus on business rules encapsulated in the domain layer.
Events for tracking domain state changes.
Tactical and Strategical DDD patterns: Ubiquitous Language,Bounded Context,Value Objects, Entities, Aggregates, Domain Events, Domain Services, Application services, Repositories, Factories,Modules.

Cross-Cutting Concerns:

Logging: Centralized and consistent logging for debugging and monitoring.
Validation: Reusable validation logic using FluentValidation to ensure data integrity.
Exception Handling: Unique exception handling to provide meaningful error messages and prevent crashes.

I tried to implement almost all possible patterns from domain-driven design like ubiquitous language, bounded context, value objects, entities, aggregates, domain events, domain services, application services, repositories, factories, modules, and more.

The development process in our case starts from Event Storming.

Before diving into the details of the source code, please check the Event Storming Board. It will help you to understand why we implemented such type of features and Event Storming will help you to understand the core ubiquitous language used in our application.

Our Application was implemented using clean architecture and I applied best practices to demonstrate the usage of clean architecture in rich business applications.

Here is the simple explanation for the diagram:

First, we have a user. User sends request to the web API. We have the application layer which handles the API query using CQRS and it forwards, it acts like orchestrator between domain models. And we have domain models that we are retrieving, we are manipulating and domain models raises domain events.

These events will be caught by the event dispatcher. And for the event dispatching, we have events and event handlers. Our events will be caught by event handlers. And if you want to go outside of your bounded context, we have integration events that will help you to publish you event outside. For example, in our context, we have Apache Kafka implementation. You can push your event to the outside using Apache Kafka.

So as you see, we have approximately more than 10 Domain-Driven Design patterns here, but that’s not all. Of course, I have used other Design patterns, let’s say from the GoF Patterns. For example, we have the Greg Young’s CQRS here.

We have the Result pattern from the functional programming. We have a decorator, mediator, publisher-subscriber, strategy, template method, factory method, chain of responsibility, unit of work, and more. If you really want to see all these patterns in practice in real-world examples, this repository is one of the best to investigate and learn all these concepts using only one project.

So now let’s dive into the details of this project using Visual Studio. In our application, you might guess, we have two main folders. They are source(src) folder and tests folder.

In our source folder, we have clean architecture and the shared library functionality. And in tests folder, we have unit tests plus the test data.

Of course, I have just implemented unit testing here and I will add integration tests, but for now, I approximately implemented all possible unit test scenarios for this project. Let’s start from the source(src) folder. You should first try to understand the domain layer here.

In our domain design applications, you are implementing microservices using a domain-driven design. In this approach, you know, we have special naming rule. In our case, we have VOEConsulting.Flame.BasketContext.Domain. ‘VOE consulting’ is a company name. ‘Flame’ is the application name. ‘BasketContext’ is a bounded context name and the ‘Domain’ is the layer name.

Lets start from the domain layer.

We have Basket, Coupon and we have Common folders. In our basket, you will find all the entities, aggregates, value objects, events, and services related to the basket. And the same is applicable for the Coupon also. After reviewing our event storming, you will understand why we have basket and coupon implementation.

public sealed class Basket : AggregateRoot<Basket>
{
    public IDictionary<Seller, (IList<BasketItem> Items, decimal ShippingAmountLeft)> BasketItems { get; private set; }
    public decimal TaxPercentage { get; }
    public decimal TotalAmount { get; private set; }
    public Customer Customer { get; private set; }
    public Id<Coupon>? CouponId { get; private set; } = null;

    private Basket(decimal taxPercentage, Customer customer)
    {
        BasketItems = new Dictionary<Seller, (IList<BasketItem>, decimal)>();
        TaxPercentage = taxPercentage.EnsurePositive();
        TotalAmount = 0;
        Customer = customer;
    }

    public void AddItem(BasketItem basketItem)
    {
        if (BasketItems.TryGetValue(basketItem.Seller, out (IList<BasketItem> Items, decimal ShippingAmountLeft) value))
        {
            value.Items.Add(basketItem);
        }
        else
        {
            BasketItems.Add(basketItem.Seller, (new List<BasketItem> { basketItem }, basketItem.Seller.ShippingLimit));
        }

        RaiseDomainEvent(new BasketItemAddedEvent(this.Id, basketItem));
    }

    public static Basket Create(decimal taxPercentage, Customer customer)
    {
        var basket = new Basket(taxPercentage, customer);
        //basket.RaiseDomainEvent(new BasketCreatedEvent(basket.Id, customer.Id));
        return basket;
    }
}

We have implemented approximately all possible domain-driven design, strategical, and tactical patterns. For example, our Basket is an Aggregate Root. Our Coupon is an Aggregate Root also. Aggregate root is a type of entity that has possibilities related to events. You can add events, clear events, pop events ,and more.

Just go to the definition. I have implemented approximately all the base classes using by myself. I haven’t used any libraries here. So you see we have aggregate root, it is inherited from the entity and it has this sort of functionalities.

public abstract class AggregateRoot<TModel> : Entity<TModel>, IAggregateRoot
    where TModel : IAuditableEntity
{
    private readonly IList<IDomainEvent> _domainEvents = new List<IDomainEvent>();
    public IReadOnlyCollection<IDomainEvent> DomainEvents => _domainEvents.AsReadOnly();

    public IReadOnlyCollection<IDomainEvent> PopDomainEvents()
    {
        var events = _domainEvents.ToList();
        ClearEvents();
        return events;
    }

    public void ClearEvents()
    {
        _domainEvents.Clear();
    }

    protected void RaiseDomainEvent(IDomainEvent domainEvent)
    {
        domainEvent.EnsureNonNull();
        _domainEvents.Add(domainEvent);
    }
}

The best way of exploring this repository is you don’t need to implement everything from scratch. I will have series of tutorials which help you to understand this project in more detail.

This is just an overview of the application which help you to understand the high level implementation of domain driven design, clean architecture, CQRS and the software development best practices. So just go to VOEConsulting.Flame.Common.Domain. And here we have all sorts of contracts, abstract classes for our application.

All sorts of aggregates inherited from Aggregate Root. If you have entity, your entity should inherit from the abstract entity class. If you have value object, your value object should inherit from the abstract value object class. Also, I have mentioned that if you want to use records, there is no need to implement such type of class because the records will handle all the equality component functionality.

In our domain layer, I’m throwing exceptions, but catching these sort of mechanisms using result pattern in a domain layer is also completely okay. I mean, you don’t need to throw exceptions. You can just easily wrap them to the result pattern and send it to the application layer. This is also completely okay, but I have used the exception-throwing mechanism here.

We validate entities, aggregates, and value object through these extension methods. For example, in our basket, we have, for example, ensure positive.

private Basket(decimal taxPercentage, Customer customer)
{
    BasketItems = new Dictionary<Seller, (IList<BasketItem>, decimal)>();
    TaxPercentage = taxPercentage.EnsurePositive();
    TotalAmount = 0;
    Customer = customer;
}

Ensure positive is a checking mechanism will help us to check if the given value is positive or not. So I have approximately all sorts of extension methods for all sorts of scenarios, but in the future, I may add or remove some of them.

Long story short, in our VOEConsulting.Flame.BasketContext.Domain, we have domain oriented logic,on the other hand, in VOEConsulting.Flame.Common.Domain we have shared domain layer features.

I will create a separate NuGet package that will help you to just easily install and use these features. But right now, you can just by yourself create your own private NuGet repository and use it.

The next layer in our application is going to be our Application Layer. The responsibility of our application layer is to handle commands and queries using CQRS. In our Web API, we have request mechanism and we are requesting some data and we are forwarding it to our application layer using CQRS.As you might guess, I have future based mechanism and it looks like a vertical slice architecture. For instance, we have create basket functionality. It has Command, Command handler and validator; like all in one.

The command handling mechanism is really easy because we have common handler base, which we have wrapped all the repeated steps into one abstract class and using template method pattern. You can just inherit and rewrite the exact **ExecuteAsync **method. So every time when we are handling a request, we have a special core handling mechanism.

I don’t have any validation code here because I have cross-cutting concerns in our behaviors on the application. So you see we have exception handling pipeline behavior and this pipeline behavior will help you to handle the exception stuff. And we have logging and we have validation.

We have a domain model orchestration like aggregate/entity orchestration in our application layer.

Of course, the domain layer throws exceptions, but I mentioned, exception handling pipeline behavior will catch it and will provide the value for you result.

The domain events will help us to notify the changes about our aggregate inside our bounded context. But if you want to go outside of our bounded context, you should map it to the integrated event and raise the integrated event.

public sealed class BasketCreatedIntegrationEvent : IntegrationEvent
{
    public BasketCreatedIntegrationEvent(Id<Basket> basketId, Guid customerId)
        : base(basketId)
    {
        CustomerId = customerId;
    }

    public BasketCreatedIntegrationEvent() { }

    public Guid CustomerId { get; set; }
}

You have orchestration mechanism because in one domain model you are storing a part of business related to use case in another domain model you are storing another part of business so we need to orchestrate them to get one use case that’s why we are using application layer and it is responsible for publishing our domain events and of course.

So integration events will be handled in our infrastructure layer because we are going outside of our bounded context. For example, we need to publish an event to message queue and we will implement the exact implementation of message queue in our infrastructure layer. And we have repository interfaces. Why I haven’t put these interfaces to domain layer because I don’t think that it is the best way of implementing repository interfaces in our case. Why? Because in my domain layer, I’m not using these interfaces. This is a bit design decision. If you have the exact implementation for your interfaces, if you are actually using these interfaces in the domain layer, you should put them to domain layer. If you are not using them in your domain layer, it would be better to move them to the upper layer like the application layer. In my case, I put them into our application layer.

Now lets talk about our infrastructure layer. The responsibility of infrastructure is to work with outside elements. If you have mail sending, you have working with GraphQL, Apache Kafka, external services, databases, you should put all these mechanisms’ implementation inside your infrastructure. For example, I have put my interfaces to the application layer, but this interfaces implementation will be stored in our infrastructure layer.

So,as you see, I have persistence, have all sort of entity framework configurations, I have all sort of entities here, profiles for mapping and the exact unit of work implementation. I have DB context here. I have all sorts of repository implementations.

In our infrastructure, we are handling all the exact implementations related to the outside world.

And the higher one is our API layer.

We have simple **BasketController **here. The responsibility of basket controller is actually getting the request data from the user and forward it to the lower layers. In my case, for example, we have create basket.

[ApiController]
[Route("api/[controller]")]
public class BasketController(ISender sender, ILogger<BasketController> logger) : BaseController(logger)
{
    private readonly ISender _sender = sender;

    // GET api/basket/{id}
    [HttpGet("{id}")]
    public async Task<IActionResult> GetBasket(Guid id)
    {
        var result = await _sender.Send(new GetBasketQuery(id));
        return result.IsSuccess ? Ok(result) : HandleError(result.Error);
    }

    // POST api/basket
    [HttpPost]
    public async Task<IActionResult> CreateBasket([FromBody] CreateBasketCommand command)
    {
        var result = await _sender.Send(command);
        if (result.IsSuccess)
            return CreatedAtAction(nameof(GetBasket), new { id = result.Value }, result.Value);
        return HandleError(result.Error);
    }
}

basket creation is a command, not a query. That’s why we have a create basket command. Using ISender’s send, we are forwarding it to the application layer, actually. And the combination of application + infrastructure will handle all this stuff for us. I have implemented base controller that will help us easily work with controllers. I don’t have too much logic because you should not put logic to your API layer if they are not related to the API layer. For example, if I need to wrap some data from the lower layer to the bad request ,I don’t now, to the 404, 400, 500, of course, I should interact with API layer. Otherwise, you should not add any logic to this API layer. So just check this controller. I will add more features. Of course, we haven’t implemented all possible endpoints for basket, for coupon, but I will add them one by one. This application will help you to understand domain driven design, clean architecture, and of the best way of implementing software. And of course, we have tests. How it is possible to write application without tests. I have almost implemented

all possible best ways of using unit tests in this application. Starting from the next tutorials, I will create a series of articles to explain almost all possible parts of this application. I will start from the unit tests and using these implementations, you can easily understand and adapt it to your current projects.

So that’s what I have implemented and I hope you will love this repository. Just go to Tural Suleymani’s GitHub and in my git, you will find the real domain-driven design, CQRS and clean architecture.

🌟 If you find this repository helpful or interesting, please don’t forget to give it a star! It really helps and motivates me to keep improving and sharing more projects. Thank you! 🌟

See you in our next tutorials, bye.

Want to dive deeper?

Regularly, I share my senior-level expertise on my TuralSuleymaniTech youtube channel, breaking down complex topics like .NET, Microservices, Apache Kafka, Javascript, Software Design, Node.js, and more into easy-to-understand explanations. Join us and level up your skills!

I’m Preparing for FAANG : Getting Started with Essentials

Tural Suleymani — Thu, 26 Dec 2024 19:27:21 +0000

Data structures and algorithms are important, but it is not always easy to grasp their main ideas. This “I’m Preparing for FAANG “series will help you not just learn data structures and algorithms but also dive into their philosophy. The final purpose of this series is to help you achieve your goal: To pass FAANG interviews.

This is the first and most foundational lesson from the series. In this tutorial, you’ll understand what exactly a data structure is and how we interact with it.

Getting started with Data structures
Data structure is simply a way of structuring data. What does it mean? It means they are essentially how data is stored, retrieved, and organized.

Data is always important. In software development, we usually work with different data structures, and using different algorithms, we interact with this data. Whether it is a small application like a utility tool, a huge web application that serves billions of people, or a machine learning model, the manipulation and management of data is a crucial process of our system.

The purpose of interacting with data defines its structure. You may need the fastest way of getting data, or you don’t care about retrieving it, but your focus is on storing the data/ writing the data.

When you first learn data structures, you may come up with a foundational question: Why do we have so many data structures out there?

Well, the choice of a data structure depends heavily on how the data will be used. So, The actual reason is your business/requirements. Consider the following cases:

If you need the fastest way of retrieving data, you can pick tables or arrays
If your main focus is storing the data, linked lists or compressed tries may be a relevant choice for you
If you need to maintain the order, data structures like binary search trees or sorted arrays are often used.
If your data grows unpredictably, you may pick dynamic arrays or linked lists. As you might guess, the requirement for the data structure is the main case here. That is why we have a lot of data structures, and that is why you should deal with so many data structures throughout your learning and working process.

Getting started with Algorithms
Data structures alone are not sufficient. What is the purpose of picking up the most suitable data structure if you don’t have a way of interacting with it? Well. That is why we need algorithms to interact with this data effectively.

We encapsulate the following logic inside our algorithms:

How quickly can new data be inserted into the structure? For example, inserting data into an array is faster than inserting it into a sorted linked list.
Can we remove elements efficiently?
How about sorting and searching? Algorithms like binary search, merge sort, and quicksort are important when it comes to large datasets.
How fast can we access the element? Searching for an element in a table is faster than a sequential search through an array. Random Access memory We mostly structure data inside a RAM. RAM stands for Random Access Memory. The performance of our application heavily depends on how well the data structure is utilized by RAM. There is no single way of interacting with RAM when it comes to data structure and RAM communication. Almost every data structure interacts with RAM in a unique way that affects performance, memory usage, and system efficiency.

We temporarily store data and instructions inside RAM for quick access to the CPU.

Data structures stored in RAM are accessed faster compared to other storage like hard drives, etc. So, RAM’s speed and organization directly affect how effectively data structures perform operations like read, search, delete, etc.

As you already know, RAM is measured in bytes (e.g. 16 GB RAM or 32 GB RAM), which represents the total amount of data it can hold at once.

All data in RAM, whether text, numbers, or instructions, is stored in binary format. For example, the symbol “A” ( in ASCII) is stored as 01000001 (1 byte). Each byte in RAM is associated with a unique memory address. In the end, the CPU uses these addresses to locate and interact with the data stored there.

In our next tutorial, we will talk about arrays, but let’s focus on storing the name ‘Tural’ in RAM. When it comes to storing the ASCII symbols, we all know that every ASCII symbol is 1 byte. Almost in all programming languages, when you specify a string, it acts like an array of chars, so they are the contiguous block of information. In RAM, the system will represent it sequentially. Every symbol of the string is one byte, so the next address in RAM will be increased to 1.

But byte is not the smallest representation of data. Originally, RAM stored a combination of 0 and 1, which we call binary representation.

After getting the binary representation, let’s see how it will be displayed in RAM.

Oh, yes, I forgot to mention, that preparing for FAANG doesn’t start from the exact questions, it starts from the foundations. When you move forward, you will understand that the fundamentals are the most important ones. Well, let’s wrap up what we have learned so far:

A bit is the smallest unit of data, and a byte is a combination of 8 bits. RAM is a temporary data storage where we store our arrays, linked lists, hash tables, and other data structures. Efficient use of RAM is crucial for optimized data handling.

Don’t worry. We will learn all these data structures in more detail.

Data, such as the name “Tural” is represented in memory as binary sequences. In ASCII, every symbol is 1 byte, so the name “Tural” will be 5 bytes. In total, we have 8*5=40 bits for this name.

Understanding binary representation in most cases helps to optimize our software design, whether working with low-level memory or preparing for interviews with FAANG companies.

See you in our next lessons. If you like my content, you can follow me on Linkedin too.

Want to dive deeper?

Regularly, I share my senior-level expertise on my TuralSuleymaniTech YouTube channels, breaking down complex topics like .NET, Microservices, Apache Kafka, Javascript, Software Design, Node.js, and more into easy-to-understand explanations. Join us!

The Cynefin framework with Domain-driven Design

Tural Suleymani — Mon, 25 Nov 2024 17:26:57 +0000

I first met the term ‘Cynefin’ when learning Domain-driven Design. I have seen multiple mentions of this framework, but not all of them were related to software development, especially DDD. The thing here is that the Cynefin framework is not directly related to the software industry. It is more than that. In simple terms, the Cynefin framework, developed by Dave Snowden back in 1999, is a decision-making and sense-making tool designed to help leaders and organizations deal with complex problems and make better decisions.

I know you may have tried to learn it before this article, and because of its complexity and complex explanations, you may fail. This article aims to try to explain it as simply as possible.

If you learn DDD, you may notice that understanding the domain is important and trying to understand how complex the domain is really complex.

Cynefin framework and domain-driven design(DDD) address complexity, deal with complex systems, and aim to provide frameworks for managing uncertainty. Cynefin categorizes problems to determine how to address them. On the other hand, DDD focuses on managing complexity in business domains by modeling them effectively.

Using Cynefin, we can identify which domain(s) in a system are Complex or Complicated, influencing how DDD might be applied.

Cynefin’s primary purpose is to guide decision-makers in understanding the context of a situation and applying appropriate strategies.

So far, so good. Now, we have some basic understanding of the Cynefin framework, and it is time to dive into the five realms of the framework.

The Cynefin framework categorizes problems into five domains, each representing a different type of system or context. It doesn’t matter if you’re a developer, manager, or a non-IT guy, throughout the working process you always deal with some problems. While the Cynefin framework helps categorize problems and provides general guidance on how to approach them, it does not describe the exact “steps of operations” to deal with issues. Instead, it offers principles or strategies adapted to the characteristics of each domain. You should remember that Cynefin is about adapting strategies to fix the context, not about following strict, predefined steps. While a “step-by-step” procedure might work in the Clear domain of Cynefin, in the Chaotic domain, quick action to stabilize the situation is prioritized.

The first domain in Cynefin is the Clear/Obvious/Simple domain. This domain typically deals with well-defined, predictable problems where ”best practices” are sufficient. In contrast, DDD is designed for complex and complicated domains, where understanding and modeling the domain requires collaboration, expertise, and iterative refinement. The main approach in this domain is “sense-categorize-respond”.

Sense: Before categorizing the problem, we observe and recognize the situation to identify the problem. In the Clear domain, the situation is easily understandable because it fits into established patterns or rules. You don’t need to apply deep analysis or experimentation. These sorts of problems don’t even require a domain expert. A simple example may be an email validation. We observe that the system needs to validate an email address provided by a user. The problem is really simple: ensure the email format meets a standard pattern.

Categorize: We have a category of “solutions” here, and we match the observed situation to a known category or pattern. We choose the corresponding best practice or standard procedure to address the issue. In terms of email validation, this problem is a well-understood problem and has established patterns and best practices. We need to categorize the issue as “input validation using predefined standard” and apply a regular expression or use a library function for validation.

Respond: Based on categorization, we need to implement the pre-defined solution. There is no need for innovative thinking- just follow the existing rules or steps. In terms of email validation, use the best practice for email validation(Simple domain). Implement a regex or library-based validator in the code and provide feedback to the user if the email doesn’t match the expected format.

using System;
using System.Text.RegularExpressions;class Program
{
    static void Main()
    {
        Console.WriteLine("Enter an email address:");
        string email = Console.ReadLine();if (IsValidEmail(email))
        {
            Console.WriteLine("The email is valid.");
        }
        else
        {
            Console.WriteLine("The email is invalid.");
        }
    }

    static bool IsValidEmail(string email)
    {
        if (string.IsNullOrWhiteSpace(email))
        {
            return false;
        }
        // Simple email validation regex
        string emailPattern = @"^[^@\s]+@[^@\s]+\.[^@\s]+$";
        return Regex.IsMatch(email, emailPattern);
    }
}

A real-world example of a clear-domain problem involving a car wash where the water supply runs out might look like this:

Sense: Recognize the problem: The car wash system detects that the water supply has been depleted (e.g., through a water level sensor or manual observation). The cause is obvious and easily identifiable: there is no water left to continue operations.

Example : The water tank for the car wash is empty, and the system cannot proceed with washing cars.

Categorize: Match the situation to a known category: Categorize the problem as “resource depletion.”

Determine the standard operating procedure (SOP) for refilling the water tank. This is a routine task with established steps.

Example : Check the water tank level. Confirm it needs refilling. Identify the nearest water source or supplier for replenishment.

Respond: Apply the known solution: Follow the established procedure to resolve the problem. Notify the operator to refill the tank or automatically trigger the water refilling mechanism if the system is automated.

Example : Refill the water tank from the designated source. Ensure the tank is full. Resume car wash operations once the water is replenished.

The next domain is the Complicated domain. The main approach here is “sense-analyze-respond”. A complicated domain represents situations where problems are more challenging than in the Clear domain, but they are still solvable with expertise and analysis. The cause-and-effect relationships exist, but they are not immediately obvious and require deeper investigation.

Any characteristics of the Complicated Domain? Of course:

We don’t have best practices. Instead of this, we have multiple solutions -> good practices.
It requires expertise and specialists.
The system is stable enough to allow analysis, but understanding it requires more effort than simple categorization.
The Recommended Approach for Complicated domains is Sense-Analyze-Respond.

Sense: Gather data and observe the situation to identify the problem.
Analyze: Use expertise, tools, or frameworks to investigate the cause-and-effect relationships. Explore different options and evaluate their potential outcomes.
Respond: Implement the solution deemed most effective based on the analysis.
In terms of the car domain, say we need to design a car’s engine. To build a car engine, we need expertise in engineering, as there are many variables to consider, like fuel efficiency, durability, emissions, etc.

Differences between domains:

In terms of DDD, we can/should apply DDD for such types of problems. A complicated domain requires expert input to understand and analyze problems when DDD centers on domain experts collaborating with developers to create models that accurately represent the business domain. Long story short, the relationship between the Complicated domain and DDD lies in their shared focus on dealing with complex problem-solving scenarios.The both:

Emphasize the need for expert input.
In the complicated domain, good practices guide decision-making, and solutions are not obvious but discoverable. By its nature, DDD introduces structured methodologies to tackle complexity.
The iterative analysis in the complicated domain aligns well with DDD’s emphasis on continuously evolving the domain model.
In a complicated domain, experts might segment the problem into understandable parts to apply solutions effectively. Similarly, DDD introduces bounded contexts to isolate complexities within specific subdomains. As an example, the tax domain can be a complicated domain example in C#. Besides the language, we need to understand the tax domain. It means you need to involve someone from this domain to understand and solve the problem:

using System;
using System.Collections.Generic;
class Program
{
    static void Main()
    {
        Console.WriteLine("Enter your income:");
        decimal income = decimal.Parse(Console.ReadLine());
        Console.WriteLine("Enter the number of dependents:");
        int dependents = int.Parse(Console.ReadLine());
        Console.WriteLine("Are you self-employed? (yes/no):");
        bool isSelfEmployed = Console.ReadLine()?.Trim().ToLower() == "yes";
        var taxCalculator = new TaxCalculator();
        decimal tax = taxCalculator.CalculateTax(income, dependents, isSelfEmployed);
        Console.WriteLine($"Your calculated tax is: {tax:C}");
    }
}
public class TaxCalculator
{
    private readonly List<ITaxRule> _taxRules;
    public TaxCalculator()
    {
        _taxRules = new List<ITaxRule>
        {
            new BasicIncomeTaxRule(),
            new DependentDeductionRule(),
            new SelfEmploymentTaxRule()
        };
    }
    public decimal CalculateTax(decimal income, int dependents, bool isSelfEmployed)
    {
        decimal tax = 0;
        foreach (var rule in _taxRules)
        {
            tax += rule.Apply(income, dependents, isSelfEmployed);
        }
        return Math.Max(tax, 0); // Ensure tax is not negative
    }
}
public interface ITaxRule
{
    decimal Apply(decimal income, int dependents, bool isSelfEmployed);
}
public class BasicIncomeTaxRule : ITaxRule
{
    public decimal Apply(decimal income, int dependents, bool isSelfEmployed)
    {
        // Example: Flat 20% tax for incomes above 50,000
        if (income > 50_000)
        {
            return (income - 50_000) * 0.20M;
        }
        return 0;
    }
}
public class DependentDeductionRule : ITaxRule
{
    public decimal Apply(decimal income, int dependents, bool isSelfEmployed)
    {
        // Example: $2,000 deduction per dependent
        return -dependents * 2000;
    }
}
public class SelfEmploymentTaxRule : ITaxRule
{
    public decimal Apply(decimal income, int dependents, bool isSelfEmployed)
    {
        if (isSelfEmployed)
        {
            // Example: 15% self-employment tax on all income
            return income * 0.15M;
        }
        return 0;
    }
}

The third domain is the** Complex domain.** The complex domain involves systems where the cause-and-effect relationship can only be understood in retrospect. Solutions arise through experimentation, probing, and feedback loops. There are no best and good practices. You must experiment with small changes to understand how the system behaves and adapt accordingly. Collaboration is very important in identifying emerging patterns. The main approach here is “probe-sense-respond”. Because relationships between actions and outcomes are not directly observable or predictable. The solution evolves over time through experimentation and feedback. The other important fact is: that complex domains often require inputs from multiple disciplines to approach solutions. You must be careful because a small change in one part of the system can lead to significant or unpredictable effects elsewhere.

Long story short, here is the most understandable approach:

Introduce small, safe-to-fail experiments to observe how the system behaves.
Observe and analyze the outcomes of the experiments.
Adapt and take action based on what you’ve learned from probing and sensing. If you think about complexity in DDD, it addresses it through Bounded contexts, clearly defined Ubiquitous language inside the context, and applying tactical design inside bounded context. Breaking the domain into Bounded Contexts helps teams focus on smaller, more manageable areas, enabling gradual understanding and refinement. DDD does not assume that all requirements or domain insights can be known upfront, making it a natural fit for complex systems. In the end, DDD enables teams to build systems that adapt and thrive in unpredictable environments.

The chaos domain in the Cynefin Framework represents situations where cause-and-effect relationships are completely unclear and immediate. Decisive action is required to stabilize the system. Unlike other domains, the chaotic domain is marked by extreme unpredictability and the need for rapid response to avoid further damage. Our main approach is “act-sense-respond”. It’s impossible to identify a pattern or determine how actions influence outcomes.

There’s no time for analysis or experimentation; No way to apply DDD. Quick action is necessary to bring the situation under control.

Our main purpose is to move the situation into a more manageable domain (e.g., complex or complicated). Without decisive action, the situation may spiral out of control.

While DDD is not inherently designed for chaotic environments, aspects of its principles can help navigate and transition chaos into manageable domains.

While DDD typically operates in complex and complicated domains, its principles can indirectly support efforts to manage and recover from chaos:

In chaotic situations, maintaining focus on a system’s Core Domain is critical. DDD emphasizes prioritizing the core business logic, ensuring that during a chaotic event, resources and actions target the most business-critical systems or processes.

Bounded Contexts help isolate parts of a system. In chaos, this separation enables teams to address and stabilize affected areas without disrupting unrelated parts.

Chaos often leads to miscommunication under pressure. DDD’s emphasis on a Ubiquitous Language fosters clear communication among developers, domain experts, and other stakeholders, ensuring alignment during chaotic responses.

DDD mostly applies event-driven architecture, which supports handling chaos. Systems designed with event sourcing and CQRS can respond to unexpected situations more flexibly. Also, DDD principles can guide the transition from chaos to the complex domain, where experimentation and adaptive solutions are possible. In the end, After an initial stabilization of chaotic failures, teams can use DDD techniques like Event Storming to model and understand emergent patterns, enabling iterative improvement.

**The disorder domain **in the Cynefin framework represents situations where it’s unclear which of the other four domains applies. This domain is marked by confusion and lack of clarity.

When the domain is poorly understood, and there is no agreement on the business goals or technical approach, you may end up with a disordered domain in the context of DDD. Well, then, how can DDD help in the disorder domain?

The disorder domain then arises from miscommunication and inconsistent terminology. UL will guide you in fixing this problem. Also, trying to solve too many problems at once or overlapping responsibilities can be a good “reason” to be in the disorder domain. From the DDD perspective, splitting the domain into well-defined Bounded Contexts can help us to move from disorder to other domains.

The other reason for being in the disorder domain is the lack of clarity on which parts of the domain are most important. In the context of DDD, strategic design helps identify core domains to prioritize and focus efforts where they bring the most value.

Conclusion
The Cynefin Framework and Domain-Driven Design (DDD) complement each other by providing a structured way to navigate complexity in software systems and business domains. While Cynefin offers a lens to categorize problems and determine suitable decision-making approaches, DDD provides concrete practices for solving problems in complicated and complex domains. Together, they form a powerful toolkit for building robust, adaptable, and business-aligned software systems.

SQL Server Execution Plans

Tural Suleymani — Wed, 13 Nov 2024 18:52:04 +0000

There are some questions that those who are familiar with the T-SQL dialect of SQL ask themselves:

Why is my query working slowly?
Does the query I wrote use an index?
Why is the index I created not used?
Why is this query slower (faster) than the other one, despite returning the same result?

If you want answers to these questions, one of the tools that must be used to understand them is Execution plans.

What is an Execution Plan?
EThe executionPlan is a description of the most efficient execution of the query we wrote, calculated by the optimizer. That is, Execution Plan shows us in which version your query is implemented, and using which algorithm it is executed.

In short, how SQL Server will or has executed our query.

Let's look at everything from the beginning,

After we write a query, the execution of the request goes through several stages. Those stages are concentrated into 2 large groups:

Events in relation engine (RE)
Events in Storage Engine (SE) In the RE(relation engine), the query is parsed and executed by the query optimizer. The query optimizer prepares an execution plan. This Execution Plan is then sent in binary format to the SE(storage engine), which uses this plan during the execution of queries. Transaction and index operations are realized in the storage engine.

Let's first open SQL SERVER and let such a request be executed. (Queries will be applied to the AdventureWorks2012 database)

use AdventureWorks2012
GO
SELECT
         hremphist.businessentityid
       , person.person.firstname
       , person.person.lastname
       , humanresources.employee.jobtitle
       , humanresources.department.name AS DepartmentName
       , hremphist.startdate
FROM (
                 SELECT
                         businessentityid
                       , departmentid
                       , startdate
                       , ROW_NUMBER() OVER (PARTITION BY businessentityid ORDER BY startdate DESC) AS rn
                 FROM humanresources.employeedepartmenthistory
         ) AS hremphist
         JOIN humanresources.department
                 ON humanresources.department.departmentid = hremphist.departmentid
         JOIN humanresources.employee
                 ON humanresources.employee.businessentityid = hremphist.businessentityid
         JOIN person.person
                 ON person.businessentityid = hremphist.businessentityid
WHERE hremphist.rn = 1;

We can see the Execution Plan of the query by pressing Ctrl+L.

Let's see together what execution processes the above request goes through,

1) Query Parsing

As soon as we execute the query we wrote, it is taken by RE and checked whether it is true or false from the syntactic side (Parsing). If there are no syntactic problems, the query has been successfully parsed. It should be remembered that if the query is DDL, it is not optimized! Because CREATE TABLE, which is a DDL operation of the example, has only one syntax. Only DML operations can be optimized. The result of the query parsing process gives us a parse tree (sequence tree, query tree).

2) Algebrizer
If the DML request has passed the parsing process successfully, it is sent to Algebrizer, a special mechanism. Algebrizer performs logical processes such as the existence of objects used by us, the names of table columns, compatibility of types, etc. As a result of these processes, Algebrizer returns us a binary format. Returns a "query processor tree", which results are processed by the query optimizer.

3) Query Optimizer prepares Execution Plan for us.

4) Storage Engine

Physical processing of the request is performed at this stage, and the request is executed based on the Execution Plan.

The SQL Query Optimizer performs the generation and preparation of Execution Plans. When preparing an Execution Plan, it necessarily refers to statistics and indexes and uses the information collected there by the execution plan maker. In the next step, the Execution Plan given by the Query Optimizer is sent to the storage engine; based on this, our query is physically executed. It should be remembered that when an Execution Plan is prepared, the prepared plan is added to the cache (processor cache) so that it is stored in the memory. During the next similar Execution Plan preparation, the Optimizer does not have to do additional work and can use the previously generated plan.

Depending on the complexity of the query, T-SQL QUERY OPTIMIZER generates several Execution Plans for us and tries to choose the fastest one among them. However, it is necessary to consider that there are often queries for which Execution Plans can be generated for their implementation, which may take several minutes. Therefore, Query Optimizer makes the necessary selection among the first generated plan combinations, not all. Gave us the perfect Execution Plan. Our EPs will be stored in the plan cache not to generate a plan for similar questions every time.

To delete cached data, run the following command:

DBCC FREEPROCCACHE

We can also delete the required plan by passing sql_handle and plan_handle specifically.

What can cause the current Execution Plan to change?

Execution of parallel queries
The statistics had changed or became outdated when the Execution Plan was made.
Entering information into the temporary table

It should be remembered that there are 2 main forms of EP(Execution Plan) in T-SQL,

The estimated Execution Plan (Estimated execution plan) is the plan the SQL Optimizer estimates before the query is executed.
Actual Execution Plan (Real execution plan) – is a plan received only after the request is released for execution.

Although these plans store a completely different set of data, they do not differ from each other at first glance.

There are 3 main forms of execution plans,

Graphical description (Graphical plans)
Description in text format (Text Plans)
In XML format (XML Plans)

The most commonly used representation form of execution plans is a Graphical plan. Although the graphic description form does not reflect all the details at first glance, detailed information can be seen in the outer panel.

Although Text Plans have been declared "deprecated" by SQL Server, we can still use them. In the Text version, you can get detailed information about the plan the first time, speed it up, and edit it in a text editor.

SET SHOWPLAN_ALL ON;
GO
SELECT TOP 100 [BusinessEntityID]
      ,[NationalIDNumber]
      ,[LoginID]
      ,[OrganizationNode]
      ,[OrganizationLevel]
      ,[JobTitle]
      ,[BirthDate]
      ,[MaritalStatus]
      ,[Gender]
      ,[HireDate]
      ,[SalariedFlag]
      ,[VacationHours]
      ,[SickLeaveHours]
      ,[CurrentFlag]
      ,[rowguid]
      ,[ModifiedDate]
FROM [ADV_Works].[HumanResources].[Employee]

In the test code example above, we enabled the detailed description of the "estimated" plan in text form. To disable this mode, write OFF instead oExecutionrun the query.

SET SHOWPLAN_TEXT ON;--active
GO
SET SHOWPLAN_TEXT OFF;--deactive
GO
SET STATISTICS PROFILE ON;--active text mode for actual plan

XML Plans - XML plans have 2 forms of description:

SHOWPLAN_XML-is generated until execution
STATISTICS_XML is generated after query execution

The most used form of EPs is a graphical form (Graphical Execution plans).

Ctrl+M (for Actual) Ctrl+L (for Estimated)
By selecting the "Include Actual Execute plan" or "Display Estimated Execute Plan" button on the toolbar
By right-clicking on the session, we wrote the query and selected "Include Actual Execution Plan" or "Display Estimated Execute plan" from the drop-down menu.

Want to dive deeper?

Regularly, I share my senior-level expertise on my TuralSuleymaniTech(English version) and TuralSuleymaniTechRu (Russian version) YouTube channels, breaking down complex topics like .NET, Microservices, Apache Kafka, Javascript, Software Design, Node.js, and more into easy-to-understand explanations. Join us and level up your skills!

Understanding API Gateways in Microservices

Tural Suleymani — Fri, 01 Nov 2024 08:22:16 +0000

In Microservice development, we mainly use interesting techniques and patterns. One of the interesting patterns in the microservice environment is called API Gateway. It describes a pattern for managing client requests for multiple services and offers a structured solution to various challenges in distributed systems and service-oriented architectures.

This article will help you understand the real value of an API gateway and will describe how to implement a simple API gateway in practice.

I always prefer to explain things in detail, let’s say in theory, before implementing it in practice. The case here is that if you don’t have theoretical fundamental knowledge or an understanding of the thing, it doesn’t make too much sense to code it, and it is mostly not possible to get the most out of the implementation.

Let’s first answer the question: Why API Gateway?

In your microservice environment, the natural microservice-oriented design ends up with multiple microservices where all these services are separated and have their address to reach out to and invoke them. Using API Gateway, you don’t need to remember their address. API gateway acts as a Single Entry Point for such types of services. Instead of managing multiple endpoints for different services, our clients can now connect to one endpoint( to our API gateway), and it means we don’t need to keep track of multiple services’ addresses.

API Gateway service is a great place to implement our authentication, authorization, and SSL termination. Instead of implementing all these stuff for different services, now we can write them and manage them from one single point. As you might guess, API Gateway is a fundamental place to implement other cross-cutting concerns as well.

Can I implement logging in my API gateway? Of Course, it is a good design decision to implement one of the popular cross-cutting concerns called logging here. API Gateways can log requests and provide insights into usage patterns, performance, and errors. We can use this data not just for monitoring but also for improving our services.

When you have API Gateway, you can get responses from multiple services and aggregate them to provide only one single response to the client

You can implement data transformation inside your API Gateway. Data transformation logic here will act as an architectural decorator over you services.

Load balancing is another interesting functionality that you can integrate into your API gateway. Via Load Balancing, API gateways can distribute requests across multiple instances of a service, providing load balancing and improving systems’ scalability and resilience.

What if I want to add some limitations for calling the exact services? Well, that is also another interesting feature we implement inside the API gateway, and it is called Rate Limiting. Gateways can limit the number of requests from a client within a specific time frame. This prevents abuse and helps maintain performance levels.

Another must-have behavior for API Gateway is Fault tolerance. In our API Gateways, we can implement retries, circuit breaking, and timeouts. If a service is down or slow, the gateway can provide fallback mechanisms, improving overall resilience.

A small note: Although effective, the API Gateway pattern can introduce some overhead, such as an extra hop in communication and potential performance bottlenecks if not scaled properly. In simpler systems, a Backend-for-Frontend (BFF) pattern is sometimes preferred, where each client type has a dedicated gateway, reducing complexity for specific use cases.

Summary
As a pattern, the API Gateway provides a standardized way to handle client requests and manage cross-cutting concerns in a distributed system. It is a highly effective design pattern for microservices and complex applications that need centralized control and routing for multiple backend services.

Want to dive deeper?

Real-World Expression Trees: Dynamic Filtering in C# with Minimal API

Tural Suleymani — Tue, 22 Oct 2024 19:05:29 +0000

In our previous tutorial, we discussed key points of expression trees, use cases, and limitations. Any topic without a practical example, especially if it is related to programming, doesn’t make much sense. This article will discuss the second part of expression trees in C# and demonstrate the real power of using Expression trees in practice.

What we’re going to build?
Our main purpose is to build an Asp.NET Core web API with dynamic filtering functionality built with minimal API, EF core, and, of course, Expression Trees.

We plan to build filtering over the product database and use Expression trees to show one of the real powers of expression trees when it comes to building complex and dynamic queries.

PS: If reading is boring then you can watch the complete tutorial here where I explain everything from scratch, one by one:

For the Russian version of the video use the following link:

Here is the final example with multiple dynamic filtering arguments:

For a more complete example, please refer to the GitHub repo.

Getting started
First, open Visual Studio and select the Asp.NET Core Web API template with the following configuration:

We use .NET 8.0, but the topic itself doesn’t depend on any .NET version. You can even use classical .NET Framework to use Expression Trees. The project name is “ExpressionTreesInPractice”.

Here is the generated template from the Visual Studio:

To have simple storage, we will use InMemory Ef Core. You can use any other EF Core sub-storage.

Now go to Tool->Nuget Package Manager->Package Manager Console and type the following command:

install-package microsoft.entityframeworkcore.inmemory

Now, let’s create our DbContext implementation. Create a folder called ‘Database’ and add a class called ProductDbContext to it with the following implementation:

using ExpressionTreesInPractice.Models;
using Microsoft.EntityFrameworkCore;

namespace ExpressionTreesInPractice.Database
{
    public class ProductDbContext : DbContext
    {
        public DbSet<Product> Products { get; set; }
        public ProductDbContext(DbContextOptions<ProductDbContext> options) : base(options) { }
        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<Product>().HasData(new List<Product>
            {
                new Product(){ Id = 1, Category = "TV", IsActive = true, Name = "LG", Price = 500},
                new Product(){ Id = 2, Category = "Mobile", IsActive = false, Name = "Iphone", Price = 4500},
                new Product(){ Id = 3, Category = "TV", IsActive = true, Name = "Samsung", Price = 2500}
            });
            base.OnModelCreating(modelBuilder);
        }
    }
}

We just need some basic initialized data when we run our application, and that is why we need to override OnModelCreating from DbContext. A great example of a template method pattern, isn’t it?

We need our Entity model called Product, and you can create a folder called ‘Models’ and add the Product class to it with the following content:

namespace ExpressionTreesInPractice.Models
{
    public class Product
    {
        public int Id { get; set; }
        public string Category { get; set; }
        public decimal Price { get; set; }
        public bool IsActive { get; set; }
        public string Name { get; set; }
    }
}

It is time to register our DbContext implementation in the Program.cs file:

builder.Services.AddDbContext<ProductDbContext>(x => x.UseInMemoryDatabase("ProductDb"));

By the way, Program.cs has tons of unnecessary code snippets that we need to remove. After the cleaning process, our should look like this:

using ExpressionTreesInPractice.Database;
using Microsoft.EntityFrameworkCore;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
builder.Services.AddDbContext<ProductDbContext>(x => x.UseInMemoryDatabase("ProductDb"));

var app = builder.Build();

// Configure the HTTP request pipeline.
app.UseHttpsRedirection();

app.Run();

We don’t want to use controllers because they are heavy and cause additional problems. That is why we choose to use minimal API. If you don’t know what minimal API is, please refer to our video tutorial to learn more.

After understanding it, open Program.cs and add the following code snippet:

app.MapGet("/products", async ([FromBody] ProductSearchCriteria productSearch, ProductDbContext dbContext) => { }

The above code defines a route in a minimal ASP.NET Core API and creates an endpoint for an HTTP GET request to the /products path. The method uses asynchronous programming to handle potentially long-running operations without blocking the main application flow.

ProductSearchCriteria is a parameter passed to the method, which contains the criteria used to filter the products. It's marked with [FromBody], meaning the request body will be bound to this parameter. Usually, GET requests don't use request bodies, but this setup is allowed if you need to pass a complex object.

ProductDbContext is the database context, which represents the session with the database. It's injected into the method, allowing the application to perform operations like querying the database for products based on the search criteria.

The reason for using **ProductSearchCriteria** instead of **Product** is that the query needs to be dynamic. In this case, the user may provide some of the attributes of the **Product**, but not all of them. Since the properties of **Product** are not nullable, the user would be required to provide every property, even if they don't want to filter by all of them.

By using **ProductSearchCriteria**, we allow for more flexibility. It acts as a container for optional and dynamic parameters. The user can choose to provide only the attributes they want to search by, making it a better fit for scenarios where not all product properties are needed in the query.

Here is what our ProductSearchCriteria class looks like in the ‘Models’ folder.

namespace ExpressionTreesInPractice.Models
{
    public record PriceRange(decimal? Min, decimal? Max);
    public record Category(string Name);
    public record ProductName(string Name);
    public class ProductSearchCriteria
    {
        public bool? IsActive { get; set; }
        public PriceRange? Price { get; set; }
        public Category[]? Categories { get; set; }
        public ProductName[]? Names { get; set; }
    }
}

Now, let's focus on our minimal API implementation. Please take into account that the purpose of the current tutorial is not to show the best practices or write clean code. The purpose is to demonstrate Expression trees in practice and after learning the point you can easily refactor the code.

Here is our first code snippet inside the **MapGet **function:

await dbContext.Database.EnsureCreatedAsync();

 ParameterExpression parameterExp = Expression.Parameter(typeof(Product), "x");

 Expression predicate = Expression.Constant(true);//x=>True && x.IsActive=true/false


 if (productSearch.IsActive.HasValue)

 {

     MemberExpression memberExp = Expression.Property(parameterExp, nameof(Product.IsActive));

     ConstantExpression constantExp = Expression.Constant(productSearch.IsActive.Value);

     BinaryExpression binaryExp = Expression.Equal(memberExp, constantExp);

     predicate = Expression.AndAlso(predicate, binaryExp);

 }

var lambdaExp = Expression.Lambda<Func<Product, bool>>(predicate, parameterExp);

var data = await dbContext.Products.Where(lambdaExp).ToListAsync();

 return Results.Ok(data);

This code is using C#'s Expression classes to dynamically build a predicate for querying a database. Let's break it down step by step.

1.await dbContext.Database.EnsureCreatedAsync();
This line asynchronously ensures that the database is created. If it doesn’t exist, it will be created. This is typically used in development or testing environments to ensure the database schema is in place.
2.ParameterExpression parameterExp = Expression.Parameter(typeof(Product), "x");
Here, a parameter expression is created to represent an instance of the Product class. This will act as the input parameter (x) in the expression tree, similar to how you define a lambda expression like x => ....
3.Expression predicate = Expression.Constant(true);
An initial predicate is created as a constant boolean expression with the value true. This is useful for building the dynamic predicate incrementally, as you can use it as a base to add more conditions (e.g., true AND other conditions). It serves as a starting point for combining additional expressions.
4.if (productSearch.IsActive.HasValue)
This block checks if the IsActive property in productSearch is not null, meaning the user has provided a filter for whether the product is active or not.
5.Inside the if block:
MemberExpression memberExp = Expression.Property(parameterExp, nameof(Product.IsActive));
This creates a MemberExpression *that accesses the IsActive property of the Product instance represented by parameterExp (x.IsActive). Essentially, it represents the expression x => x.IsActive.
**ConstantExpression constantExp = *Expression.Constant(productSearch.IsActive.Value);**
A ConstantExpression is created with the value of productSearch.IsActive. This represents the value to compare against (true or false).
BinaryExpression binaryExp = Expression.Equal(memberExp, constantExp);
A BinaryExpression is created to compare the IsActive property with the provided value. This represents x.IsActive == productSearch.IsActive.
predicate = Expression.AndAlso(predicate, binaryExp);
The current predicate (which started as true) is combined with the new condition (x.IsActive == productSearch.IsActive) using a logical AND. This results in an expression that can be used to filter products based on their active status.

Overall, the above code is dynamically building an expression tree that will eventually be used to filter products based on whether they are active. The initial predicate (true) allows for additional conditions to be added easily without special handling for the first condition. If productSearch.IsActive is provided. It adds a condition that checks if the product’s IsActive property matches the given value (true or false).

Then, the lambdaExp variable is assigned a lambda expression that represents a filtering function for the Product entities. This lambda expression is created from the predicate built earlier, which may contain conditions like checking whether the product is active (IsActive). The Expression.Lambda> call generates a Func, meaning a function that takes a Product as input and returns a boolean value, determining whether the product satisfies the filtering criteria.

Next, this lambda expression is passed to the Where method of the Products DbSet in dbContext. The Where method applies this filter to the product records in the database. It creates a query that retrieves only the products matching the conditions defined in the lambda expression.

Finally, the ToListAsync() method asynchronously executes the query and retrieves the matching products as a list. This list is then returned as part of an HTTP 200 OK response using Results.Ok(data). The result is the filtered list of products, which is sent back as the API's response.

In order to test it, just run the application and send the following GET request with Body via Postman:

This approach is useful for us when building queries dynamically, as it allows the flexibility to add conditions based on which filters are provided.

Here is how your LINQ expression should look after compiling your expression tree:
{x => (True AndAlso (x.IsActive == True))}

So far, we have implemented the easiest property, which has two values: true or false. But how about other properties like categories, names, prices, etc.? Users are also able to not pick a product based on whether it is active or not but pick, for example, based on its category field. We allow users to provide multiple categories at the same time. That is why we implemented it as an array in our ProductSearchCategory class.

if (productSearch.Categories is not null && productSearch.Categories.Any())
{
    //x.Category
    MemberExpression memberExp = Expression.Property(parameterExp, nameof(Product.Category));
    Expression orExpression = Expression.Constant(false);
    foreach (var category in productSearch.Categories)
    {
        var constExp = Expression.Constant(category.Name);
        BinaryExpression binaryExp = Expression.Equal(memberExp, constExp);
        orExpression = Expression.OrElse(orExpression, binaryExp);
    }
    predicate = Expression.AndAlso(predicate, orExpression);
}

The code is adding dynamic filtering for product categories. It first checks if the Categories in the productSearch object is not null and contains any items. If so, it proceeds to build a dynamic expression to filter products by category.

It starts by accessing the Category property of the Product class through an expression. This member expression represents x => x.Category, where x is an instance of Product.

An initial orExpression is set to false. This will serve as the base for the dynamic category comparison. It uses a loop to iterate over each category in productSearch.Categories. For each category, a constant expression with the category name is created, and a binary expression checks if the product's Category equals this name.

The binary expressions are then combined using OrElse, meaning that if the product matches any of the given categories, the condition becomes true. After processing all categories, the combined orExpression is appended to the main predicate with AndAlso. This means the overall predicate will now check both the previous conditions and whether the product's category matches any of the categories in the search criteria.

This approach allows for dynamically filtering products by multiple categories, and it integrates the category filtering into the existing predicate.

At the end of the last code, you would get a LINQ expression that represents a lambda function used to filter products based on dynamic conditions. This expression can be translated into a predicate for use in a LINQ query, which can be applied to your ProductDbContext or any IQueryable.

The LINQ expression, in this case, would be a combination of logical operations (AND and OR) that filter products. Specifically, it looks like this in pseudocode:
products.Where(x => (x.Category == "Category1" || x.Category == "Category2" || ...) && other conditions)

If a user provides both (isActive and categories) then we should get the following lambda expression:
{x => ((True AndAlso (x.IsActive == True)) AndAlso (((False OrElse (x.Category == "TV")) OrElse (x.Category == "Some Other")) OrElse (x.Category == "Mobile")))}

We follow the same approach for the Names field. Here is our code snippet:

if (productSearch.Names is not null && productSearch.Names.Any())
{
    //x.Name
    MemberExpression memberExp = Expression.Property(parameterExp, nameof(Product.Name));
    Expression orExpression = Expression.Constant(false);
    foreach (var productName in productSearch.Names)
    {
        var constExp = Expression.Constant(productName.Name);
        BinaryExpression binaryExp = Expression.Equal(memberExp, constExp);
        orExpression = Expression.OrElse(orExpression, binaryExp);
    }
    predicate = Expression.AndAlso(predicate, orExpression);
}

This code snippet dynamically builds a filtering condition for product names using expression trees. It first checks if the productSearch.Names property is not null and contains any items. If there are product names to filter by, it proceeds to build an expression for comparing the Name property of the Product entity.

The memberExp expression refers to the Name property of the Product (x.Name in a lambda expression). An initial expression, orExpression, is created, starting as false. This orExpression will be updated in a loop to accumulate comparisons for each name in productSearch.Names.

Within the loop, for each name in the productSearch.Names collection, a constant expression is created from the product name. A binary expression is then formed to check if the product's Name equals the current name from the search. The loop builds up a series of OR conditions using Expression.OrElse, which creates a logical OR operation between the current orExpression and the new comparison.

After the loop, the final orExpression represents a chain of OR conditions where the product's Name must match one of the names in productSearch.Names. This expression is combined with the existing predicate using Expression.AndAlso, ensuring that the name filter is applied along with any other conditions previously defined in the predicate.

Long story short, our block of code dynamically constructs a query filter that matches products based on their Name, allowing for multiple possible names from the productSearch.Names collection.

If the User provides only Names from the Body of the query, we will get approximately the following lambda expression at the end:

{x => (True AndAlso (((False OrElse (x.Name == "LG")) OrElse (x.Name == "LG2")) OrElse (x.Name == "Samsung")))}

If we get all filter parameters like isActive, categories, and names from the request body, we will get the following lambda expression at the end:

{x => (((True AndAlso (x.IsActive == True)) AndAlso (((False OrElse (x.Category == "TV")) OrElse (x.Category == "Some Other")) OrElse (x.Category == "Mobile"))) AndAlso (((False OrElse (x.Name == "LG")) OrElse (x.Name == "LG2")) OrElse (x.Name == "Samsung")))}

Here is what it looks like when running the application and sending the query:

The final argument for our dynamic filtering is Price. It is a complex object which consists of min and max values. The user should be able to provide any of them, both or none of them. That is why we designed it with nullable parameters.

Here is what our code implementation looks like:

if (productSearch.Price is not null)
{
    //x.Price 400
    MemberExpression memberExp = Expression.Property(parameterExp, nameof(Product.Price));
    //x.Price>=min
    if (productSearch.Price.Min is not null)
    {
        var constExp = Expression.Constant(productSearch.Price.Min);
        var binaryExp = Expression.GreaterThanOrEqual(memberExp, constExp);
        predicate = Expression.AndAlso(predicate, binaryExp);
    }
    //(x.Price>=min && x.Price.Max<=max)
    if (productSearch.Price.Max is not null)
    {
        var constExp = Expression.Constant(productSearch.Price.Max);
        var binaryExp = Expression.LessThanOrEqual(memberExp, constExp);
        predicate = Expression.AndAlso(predicate, binaryExp);
    }
}

This code dynamically constructs a predicate for filtering products based on their Price range using expression trees. It starts by checking if the productSearch.Price object is not null, which indicates that a price filter is applied.

The memberExp expression is created to represent the Price property of the Product (x.Price). This expression is used to compare the product's price against the minimum and maximum values in the productSearch.Price object.

If the minimum price (productSearch.Price.Min) is provided (not null), an expression is built to check if the product's Price is greater than or equal to this minimum value. This condition is added to the overall predicate using Expression.AndAlso, meaning the product must satisfy this condition to be included in the results.

Similarly, if the maximum price (productSearch.Price.Max) is provided, another expression is constructed to check if the product's Price is less than or equal to the maximum value. This condition is also combined with the existing predicate using Expression.AndAlso, ensuring that both the minimum and maximum price conditions are applied.

Long story short, the code builds a predicate that filters products by a specified price range, ensuring that products have a price greater than or equal to the minimum (if provided) and less than or equal to the maximum (if provided).

If the User provides only Price from the Body of the query, we will get approximately the following lambda expression at the end:

{x => ((True AndAlso (x.Price >= 400)) AndAlso (x.Price <= 5000))}

If we get all filter parameters like IsActive, Categories, Names, and Price from the request body, we will get the following lambda expression at the end:

{x => (((((True AndAlso (x.IsActive == True)) AndAlso (((False OrElse (x.Category == "TV")) OrElse (x.Category == "Some Other")) OrElse (x.Category == "Mobile"))) AndAlso (((False OrElse (x.Name == "LG")) OrElse (x.Name == "LG2")) OrElse (x.Name == "Samsung"))) AndAlso (x.Price >= 400)) AndAlso (x.Price <= 5000))}

Here is what it looks like when running the application and sending the query:

The elegant ending

This article serves as a practical continuation of the previous tutorial on C# expression trees, focusing on their real-world usage within an ASP.NET Core web API. It explores the creation of dynamic filtering functionality using minimal API, Entity Framework Core (EF Core), and expression trees.

The project involves building a product database with dynamic filtering capabilities, such as filtering by product attributes like IsActive, Category, Name, and Price. The use of expression trees is highlighted to construct flexible, dynamic queries without hardcoding-specific filters.

The setup begins with an ASP.NET Core Web API using an in-memory database for storage, although other EF Core-supported databases could be used. The article emphasizes using minimal API over traditional controllers for simplicity and performance and guides the user through the necessary steps, including setting up the database context (DbContext) and initializing data.

One of the core features demonstrated is how expression trees are used to build predicates dynamically. For example, when filtering by the IsActive property, the system checks whether the user provided this filter and then dynamically constructs a condition that compares the product's IsActive status with the provided value. The process is extended to handle dynamic filtering of other properties such as Category, Name, and Price, each of which allows flexible criteria for querying.

By using expression trees, the article illustrates how complex and flexible queries can be constructed without writing multiple hardcoded query methods. The example of filtering products by Name and Category demonstrates how logical OR conditions can be combined dynamically, depending on user input, resulting in concise and reusable query logic.

Additionally, the price filtering is handled by checking both minimum and maximum values and dynamically adjusting the predicate to include only those products within the specified price range.

In conclusion, this article demonstrates the power of expression trees in building dynamic, flexible queries in C# applications. It provides hands-on code examples of using expression trees to construct queries for an ASP.NET Core web API, offering a practical way to manage complex, real-world scenarios like filtering product databases based on varying user input.

Solid Introduction to Expression Trees in C#

Tural Suleymani — Sat, 28 Sep 2024 11:15:41 +0000

Expression trees are one of the complex topics in C#/.NET that needs to be understood. They represent code in a tree-like data structure, where each node is an expression (such as a method call, binary operation, or constant value). They allow you to construct, examine, and execute code dynamically at runtime.

Expression trees are particularly useful for creating dynamic code, analyzing code at runtime, and enabling frameworks like LINQ to SQL and Entity Framework to translate C# code into SQL queries or other operations.

Expression trees are composed of nodes, each representing a specific element of a program (e.g., a method call, a lambda expression, or a binary operation like + or -).

Before diving into the details of technical implementation, let’s try to understand the use cases of Expression trees.

1. LINQ Providers: In LINQ to SQL and Entity Framework, expression trees are used to parse LINQ queries and translate them into SQL statements. When you write a LINQ query like dbContext.Products.Where(p => p.Price > 100), the LINQ provider examines the expression tree representing p => p.Price > 100 and translates it into a SQL query (SELECT * FROM Products WHERE Price > 100).
2. Dynamic Query Construction: Expression trees allow developers to dynamically construct queries at runtime. For example, you can build complex search conditions based on user input, dynamically combining predicates using expressions like Expression. Also Expression.OrElse.
3. Meta-Programming: Expression trees enable meta-programming scenarios where you can inspect and manipulate code at runtime. You can analyze expression trees to understand the structure of code, allowing you to write tools that generate or transform code.
4. Building Dynamic LINQ Queries: Expression trees allow you to construct dynamic LINQ queries by building predicates based on conditions at runtime. This is useful when constructing search filters or complex queries based on dynamic user input.

var parameter = Expression.Parameter(typeof(Product), "p"); 
 var property = Expression.Property(parameter, "Price"); 
 var constant = Expression.Constant(100);  
var condition = Expression.GreaterThan(property, constant); 
var lambda = Expression.Lambda<Func<Product, bool>>(condition, parameter);

5. Custom Rule Engines: Expression trees are used in rule engines where business rules are evaluated dynamically. Developers can build, compile, and execute rules represented by expression trees based on data at runtime.

What do we have as advanced Features?

Expression Visitor: An ExpressionVisitor is a class in the System.Linq.Expressions namespace that allows you to traverse and modify expression trees. This is useful for scenarios where you need to analyze or modify parts of an expression tree.

public class CustomExpressionVisitor : ExpressionVisitor
{
    protected override Expression VisitBinary(BinaryExpression node)
    {

        // Example: Change all addition operations to multiplication
        if (node.NodeType == ExpressionType.Add)
        {
            return Expression.Multiply(node.Left, node.Right);
        }
        return base.VisitBinary(node);
    }
}

Expression Trees for LINQ Query Optimization: Expression trees can be used to optimize LINQ queries at runtime. By analyzing the structure of a LINQ query, frameworks can choose to cache certain expressions, rewrite inefficient queries, or perform other optimizations.
Combining Expressions: You can combine multiple expressions dynamically to create more complex queries. For instance, you can dynamically build predicates using Expression.AndAlso Expression.OrElse.

Expression<Func<Product, bool>> expr1 = p => p.Price > 100;

Expression<Func<Product, bool>> expr2 = p => p.Category == "TV";

var combined = Expression.Lambda<Func<Product, bool>>(

Expression.AndAlso(expr1.Body, expr2.Body), expr1.Parameters);

Expression trees are created using the types defined in the System.Linq.Expressions namespace. The most common classes include.

Expression: The base class for all nodes in an expression tree.
LambdaExpression: Represents lambda expressions.
BinaryExpression: Represents binary operations (e.g., +, -, , /).
**MethodCallExpression:* Represents method calls.

How to build them?
You can create expression trees manually using factory methods such as Expression.Add(), Expression.Constant(), and Expression.Lambda(). For example, to represent the expression x + 1 where x is a parameter.

ParameterExpression param = Expression.Parameter(typeof(int), "x");

ConstantExpression constant = Expression.Constant(1);

BinaryExpression body = Expression.Add(param, constant);

Expression<Func<int, int>> lambda = Expression.Lambda<Func<int, int>>(body, param);

Once an expression tree is built, it can be compiled into executable code using Compile().

var compiledLambda = lambda.Compile();

int result = compiledLambda(5);  // result = 6

Long story short, we have the following steps.

Construction: Expression trees are built using System.Linq.Expressions.Expression static methods such as Expression.Add(), Expression.Call(), and Expression.Lambda(). Each method constructs a node in the expression tree.
Compilation: Once the expression tree is built, you can compile it into executable code using the Compile() method, which turns the expression into a delegate that can be invoked.
Execution: After compiling, the expression behaves like any regular delegate, and you can invoke it with arguments.

Any limitations? Of course. Here they are:
No Full Language Support: While expression trees cover a wide range of C# constructs, they don’t support all C# features. For example, loops, try-catch blocks, and certain other flow-control constructs cannot be represented using expression trees.
Performance Overhead: Constructing and compiling expression trees can introduce a performance overhead compared to using compiled code. However, once compiled, the generated delegate executes with minimal overhead.
Complexity: Managing and manipulating expression trees for complex logic can become cumbersome due to their hierarchical and low-level nature. This is why they are often used in combination with higher-level abstractions.

In the end, let’s cover real-world examples of using Expression Trees.

Entity Framework Core: EF Core uses expression trees to translate LINQ queries into SQL queries. The LINQ queries written in C# are parsed into expression trees, and then EF Core translates these trees into the corresponding SQL.
**Custom Query Builders: **You can use expression trees to build custom query builders that generate complex search queries based on dynamic conditions. For example, if you’re building a search filter for a web application, you could use expression trees to construct the query based on the user’s input dynamically.
**Unit of Work and Repository Patterns: **Expression trees are often used in repository patterns to implement dynamic filtering, sorting, and pagination in a reusable way.

Conclusion
Expression trees are an invaluable tool when you need to dynamically create, manipulate, or inspect code in a flexible and efficient way. They are particularly important for frameworks that rely on dynamic code generation, such as LINQ providers and rule engines.

Want to dive deeper?

Regularly, I share my senior-level expertise on my TuralSuleymaniTech YouTube channel, breaking down complex topics like .NET, Microservices, Apache Kafka, Javascript, Software Design, Node.js, and more into easy-to-understand explanations. Join us and level up your skills!

Resolving Race Conditions and Critical Sections in C#

Tural Suleymani — Fri, 30 Aug 2024 08:38:54 +0000

A race condition in C# occurs when two or more threads access shared data simultaneously, and the outcome of the program depends on the unpredictable timing of these threads. This can lead to inconsistent or incorrect results, making race conditions a critical problem in multi-threaded applications.

In this article, we will learn everything in practice and will try not just to understand but also to resolve the race condition and critical section problems in .NET.

How Race Conditions Occur?
Race conditions typically occur when the following conditions are met.

Shared Resource: Two or more threads are trying to read, write, or modify the same shared resource (e.g., a variable, object, or file).
Concurrent Execution: The threads execute concurrently without proper synchronization mechanisms to control their access to the shared resource.
Race conditions occur when multiple threads concurrently access and modify shared data, leading to unpredictable results.

They arise due to the lack of synchronization, where threads interleave in a way that causes operations to be performed out of the expected order.

A critical section in C# refers to a block of code that must be executed by only one thread at a time to prevent data corruption or inconsistent results due to concurrent access. When multiple threads access shared resources, such as variables or objects, and at least one thread modifies those resources, a critical section ensures that only one thread can execute the code block that accesses the shared resource at a time. This is crucial for maintaining the integrity of the shared data.

Let's directly switch to our example below and explore both worlds.

public class Transaction
{
    public bool IsDone { get; set; }
    public void Transfer(decimal amount)
    {
        if (!IsDone)//critical section
        {
            Console.WriteLine($"Transaction operation started in thread number = {Environment.CurrentManagedThreadId}");
            TransferInternally(amount);
            Console.WriteLine($"Transaction operation ended in thread number = {Environment.CurrentManagedThreadId}");
            IsDone = true;
        }
    }
    private void TransferInternally(decimal amount)
    {
        Console.WriteLine($"TransferInternally in thread...{Environment.CurrentManagedThreadId}, the amount = {amount}");
        Console.WriteLine($"TransferInternally is done in thread...{Environment.CurrentManagedThreadId}, the amount = {amount}");
    }
}

In the provided code, the critical section and the race condition problem relate to how multiple threads might interact with the Transaction class, particularly with the IsDone property and the Transfer method.

Critical Section
A critical section is a part of the code that accesses shared resources (in this case, the IsDone property) and must not be executed by more than one thread at a time. The critical section in your code is.

This code block is critical because it checks and updates the IsDone property. If multiple threads access this code simultaneously without proper synchronization, it could lead to inconsistent or incorrect behavior.

Race Condition Problem
A race condition occurs when a program's outcome depends on the timing or sequence of threads executing. In this code, a race condition could happen if multiple threads call the Transfer method at the same time.

Scenario Leading to a Race Condition

Thread 1 checks the IsDone property and finds it to be false.
Thread 1 proceeds with the transaction, outputting the start message and entering the TransferInternally method.
Thread 2 then checks the IsDone property before Thread 1 has finished and still finds it to be false.
Thread 2 also proceeds with the transaction, despite Thread 1 already handling it.

static void Main(string[] args)
{
    Transaction2 transaction = new Transaction2();
    for (int i = 0; i < 10; i++)
    {
        Task.Run(() =>
        {
            transaction.Transfer(3000);
        });
    }
    Console.ReadLine();
}

The provided code demonstrates how a race condition problem can occur when multiple threads simultaneously attempt to execute the Transfer method of the Transaction class.

Explanation of the Code
Transaction Instance

Transaction transaction = new Transaction();

Here, a single instance of the Transaction class is created. This instance is shared among all threads that will be created in the subsequent loop.

for (int i = 0; i < 10; i++)
{
    Task.Run(() =>
    {
        transaction.Transfer(3000);
    });
}

This loop runs 10 times, and each iteration starts a new task using Task. Run. Each task calls the Transfer method of the transaction object, attempting to transfer the amount of 3000.

Task
Each Task. Run spawns a new thread (or reuses one from the thread pool) to execute the code within the lambda expression. Therefore, up to 10 threads might be running the Transfer method concurrently.

Because there’s no synchronization mechanism like a lock statement, multiple threads could be executing this block simultaneously. This leads to several threads performing the transfer, outputting the start and end messages, and setting IsDone to true.

Race Condition Effect
The race condition occurs because the IsDone check and the subsequent operations are not atomic. This means that even if one thread sets IsDone to true, other threads might have already passed the check and are also executing the transfer, leading to multiple executions of what should be a one-time transaction.

The outcome of the Race Condition
As a result of the race condition, you might see output that indicates the transaction was processed multiple times, even though the logic implies it should only happen once. Each thread will print its messages to the console, showing that multiple threads have entered the critical section and completed the transaction.

The output

To prevent race condition problems and capture the critical section, we will use the lock keyword. Of course, we have multiple ways to avoid these problems but the easiest one is using the lock keyword.

public class Transaction2
{
    public bool IsDone { get; set; }
    private static readonly object _object = new object();
    public void Transfer(decimal amount)
    {
        lock(_object)
        {
            if (!IsDone)//should act as a single atomic operation
            {
                Console.WriteLine($"Transaction operation started in thread number = {Environment.CurrentManagedThreadId}");
                TransferInternally(amount);
                Console.WriteLine($"Transaction operation ended in thread number = {Environment.CurrentManagedThreadId}");
                IsDone = true;
            }

        }

    }

    private void TransferInternally(decimal amount)
    {
        Console.WriteLine($"TransferInternally in thread...{Environment.CurrentManagedThreadId}, the amount = {amount}");
        Console.WriteLine($"TransferInternally is done in thread...{Environment.CurrentManagedThreadId}, the amount = {amount}");
    }
}

Transaction2 result

Advanced JS — Refactoring Toward Objects

Tural Suleymani — Sun, 18 Aug 2024 17:27:29 +0000

Javascript has several object creation syntaxes, but object configurations (in a sense, associative arrays) occupy a special place among them. This modeling approach is often referred to as object configurations because it contains more configuration information when used, similar to JSON syntax in practice. When object configurations do not store motion in themselves, they act as an associative array to it. Associative arrays are another form of an array whose index is usually a form of an array containing more meaningful phrases.

PS: You can download the source codes of the project here. Don’t forget to put a star on the repo if you like it.

Author’s note: One of the top 10 JS topics we used most when programming a payment terminal was object configurations. When a new provider was integrated into the terminal, information such as the added provider’s identifier, min-max payment limit, which provider group it belongs to, the names of parameters that will come and go during the request-response, etc., were stored in the object configurations.

{
    "__type": "pages",
    "__objects": [{
        "pageId": "30492001",
        "title": "",
        "useOnline": "true", // Онлай авторизация false-нет, true-да.
        "__objects": [{
            "__type": "controls",
            "__objects": [{
                "header": "Xahiş edirik, 15 rəqəmli kodu daxil edin", // Приветствие
                "footer": "", // Подпись под полем ввода номера
                "mask": "", // Маска поля ввода
                "name": "account",
                "nobr": "false",
                "regexp": "^\\d{15}$",
                "strip": "True", // Удаление доп. символов в маске например кавычки. true - удалить false - нет
                "type": "text_input"
            }, {
                "layout": "DG", // Язык клавиатуры по умолчанию DG - цифровая, Буквенная ( AL - английский язык, ALC - Всключенный Capslock, ALS - Зажат Shift, ALR - Русская раскладка, ALRC - Русская с Capslock, ALRS - Русская с Shiftom (Первая буква))
                "type": "keyboard"
            }]
        }]
    }]
}

PS: User-defined arrays just store information, not specific actions. But object configurations store both associative data and actions (methods). Therefore, it is normal that you will not come across the term associative array in many JS literature.

How are object configurations created?

Object configurations are mainly declared using the {} symbol. You will rarely come across a variant written with a new Object(). Object configurations usually store the configuration of a particular object according to its name, and methods for manipulating those configurations. This declaration of an object allows concepts to be viewed as objects.

You can use it as an associative array, simply storing values in object configurations. When developing large applications, it may be very necessary for the object to act as an associative array. So getting some information from the backend can be expensive for your application. At this time, we store static information in the form of an associative array. When we programmed the Emanat terminal, we saved the configuration of the providers in the form of an associative array. If the provider names on the terminal’s home page came from the backend, the terminal would be heavily loaded. But reading them from a JS associative array gives us maximum speed.

There are some rules to be aware of when creating object configurations.

Step 1

To protect the internal structure of the object from external interference, the variables of the object are usually declared with the sign _.

let person = {
    _name: "Hesen",
    _surname: "Mammadov",
    changeName: function (name) {
        //validate here
        this._name = name;
    }
}

Step 2

Following the OOP paradigm, the internal variables of an object should only be allowed to change via a method.

Step 3

It is possible to dynamically add an absent field to the object. Because object configurations can also act as just another form of arrays!

//this code however works fine...
person.age = 45;

Step 4

By configuring the object, it is possible to prohibit dynamically adding objects to its, iteration.

Step 5

The variable of the object, unlike normal variables, consists of two or more words and can be written separately. In this case, we can simply refer to it using the array syntax.

let person = {
    _name: "Hesen",
    _surname: "Mammadov",
    'full info':'Full info here',
    changeName: function (name) {
        //validate here
        this._name = name;
    }
}

Step 6

If the property names of the object follow the naming rules of variables, then we can write them in quotes or not.

let person = {
    '_name': "Hesen",
    '_surname': "Mammadov",
    'full info':'Full info here',
    changeName: function (name) {
        //validate here
        this._name = name;
    }
}

What we learned in practice

To practice what we’ve learned, we’ll try to put together a small app like the one below using javascript. We will create not only the animation on the page but all the HTML elements using Js so that we can better understand the object configurations.

In the first version of the program, the following functional codes were written to create the container and buttons of the page

function createButton(txt, classList) {
    let btn = document.createElement('button');
    btn.innerText = txt;
    for (let f of classList) {
        btn.classList.add(f);
    }
    return btn;
}

function createImage(src, width) {
    let img = document.createElement('img');
    img.src = src;
    img.style.width = width;
    return img;
}

function createContainer() {
    let dvContainer = document.createElement('div');
    dvContainer.className = 'controls';
    return dvContainer;
}

These functions are ultimately used by the **loadAllHtml() **function.

function loadAllHtml() {
    let img = createImage('img/going.gif', '20%');
    document.body.insertBefore(img, document.body.firstChild);
    let dvContainer = createContainer();
    document.body.insertBefore(dvContainer, document.getElementById('scr'));
    let btn1 = createButton('Start', ['btn', 'btn-start']);
    let btn2 = createButton('Stop', ['btn', 'btn-stop']);
    let btn3 = createButton('Reset', ['btn', 'btn-reset']);
    dvContainer.appendChild(btn1);
    dvContainer.appendChild(btn2);
    dvContainer.appendChild(btn3);
}

Our function called activeAllElements() is designed to connect to events. this function allows us to bind events to buttons on the page.

function activeAllElements() {
    let interval = null;
    let isRunning = false;
    document.getElementsByClassName('btn-start')[0].addEventListener('click', function() {
        if (!isRunning) {
            interval = setInterval(() => {
                let img = document.querySelector('img');
                let iml = parseInt(getComputedStyle(img).marginLeft);
                iml += 2;
                isRunning = true;
                img.style.marginLeft = `${iml}px`;
            }, 20);
        }
    });
    document.getElementsByClassName('btn-stop')[0].addEventListener('click', function() {
        if (interval != null) {
            clearInterval(interval);
            isRunning = false;
        }
    });
    document.getElementsByClassName('btn-reset')[0].addEventListener('click', function() {
        if (interval != null) {
            clearInterval(interval);
            isRunning = false;
            document.querySelector('img').style.marginLeft = 0;
        }
    });
}

Although the codes we wrote work, they have very serious gaps. Let’s list some of these gaps,

Functions like createButton() and createImage() are very poorly written functions. They do not allow solving all problems from a class in this example. Since we are talking about the HTML element in the end, these functions can be combined to create a final function, which we have done in the OOP version.
The registration process for events is not perfect. Because the codes we write are strongly dependent on the context, it is almost impossible to use those functions in another similar project.
The activeAllElements() function does not completely reflect reality in terms of both naming and operation. Because there are codes here just for the sake of work.

Refactoring towards object

Refactoring is the process of improving the code of a program without changing its functionality. That is, the program works correctly as it is, but the code is refined.

To do proper refactoring in the code we write, we first need to do a proper separation of responsibility.

According to the division of responsibilities, it is necessary to ensure the creation of objects on the page and the preparation of animation. We will create a separate object for each of these tasks. First, let’s look at our DOM object. This object will allow you to easily create DOM elements.

let dom = {
    createElement: function(tagName, className, cssProps) {
        //create dom element
        let element = document.createElement(tagName);
        //check if this element is valid html element
        if (element != null) {
            //if so, then validate classname existance
            if (this._isValidValue(className)) {
                element.className = className;
            }
            //enumerate all css properties given as  object
            for (let f in cssProps) {
                //check if object key is valid attribute
                if (this._isValidAttr(f)) {
                    //then use it as attribute
                    element[f] = cssProps[f];
                }
                //otherwise,it is just style.Use it as style..
                else {
                    element.style[f] = cssProps[f];
                }
            }
        }
        return element;
    },
    _isValidValue: function(val) {
        return (typeof(val) !== "undefined" && val)
    },
    _isValidAttr: function(attr) {
        return (attr == 'src' || attr == 'href' || attr == 'innerText' || attr == 'id');
    }
}

Now that we can create DOM elements, we can move on to the animation part. Object-oriented mechanism saves us from the concept of a global variable.

let animation = {
    _isRunning: false,
    _interval: null,
    _animatableElementInstance: null,
    animate: function(elementId, sec) {
        this._animatableElementInstance = document.getElementById(elementId);
        if (!this._isRunning) {
            this._interval = setInterval(function() {
                let element = document.getElementById(elementId);
                let imgML = parseInt(getComputedStyle(element).marginLeft);
                imgML += 1;
                element.style.marginLeft = `${imgML}px`;
            }, sec);
            this._isRunning = true;
        }
    },
    stopAnimate: function() {
        if (this._interval != null) {
            this._isRunning = false;
            clearInterval(this._interval);
        }
    },
    resetAnimate: function(elementId) {
        this.stopAnimate();
        let element = document.getElementById(elementId);
        element.style.marginLeft = 0;
    }
}

In the end, we just rely on the functional aspect to create the elements on the page.

function loadAllHtmlContent() {
    //create img
    let img = dom.createElement('img', 'img-basic', {
        'src': "going.gif",
        'width': "20%",
        'id': 'animetableImage'
    })
    //add it before script
    document.body.insertBefore(img, document.getElementById('scr'));
    //create container
    let divContainer = dom.createElement('div', 'controls');
    //with buttons
    let btn_start = dom.createElement('button', 'btn btn-start', {
        'innerText': 'start'
    });
    let btn_stop = dom.createElement('button', 'btn btn-stop', {
        'innerText': 'stop'
    });
    let btn_reset = dom.createElement('button', 'btn btn-reset', {
        'innerText': 'reset'
    });
    //add them to container
    divContainer.appendChild(btn_start);
    divContainer.appendChild(btn_stop);
    divContainer.appendChild(btn_reset);
    //append container after img
    document.body.after(img, divContainer);
}

After writing these codes, it is enough for us to add this code by simply opening the script tag inside the body tag on our required page.

document.addEventListener('DOMContentLoaded', function() {
    //create all html elements
    loadAllHtmlContent();
    document.querySelector('.btn-start').addEventListener('click', function() {
        animation.animate('animetableImage', 10);
    });
    document.querySelector('.btn-stop').addEventListener('click', function() {
        animation.stopAnimate();
    });
    document.querySelector('.btn-reset').addEventListener('click', function() {
        animation.resetAnimate('animetableImage');
    });
});

As you can see, each object performs only its operations.

Want to dive deeper?

Launched my Russian Version of Youtube Channel

Tural Suleymani — Fri, 02 Aug 2024 07:16:07 +0000

Dear friends,

As you already know, I have a youtube channel (TuralSuleymaniTech) where I share my Senior level expertise with the latest .NET/C# and related topics

The Russian version of my YouTube channel is now ready(TuralSuleymaniTechRU), and our first video has been released. It's not AI-generated, everything is filmed from scratch.

You can subscribe and support the channel: TuralSuleymaniTechRU

Going forward, I'll try to create both English and Russian versions of the same content.

PS: I should note that I haven't studied in a Russian-speaking environment and haven't taken any Russian language courses or training. Like with English, I'm self-taught in Russian. So, I'm quite okay with grammatical and conversational mistakes and don't feel any anxiety about it.

The first lesson video:

youtube.com

Thanks in advance,
Tural Suleymani