DEV Community: Rasul

Object Oriented Design Balance With Understanding Anti-Single Responsibility Principle

Rasul — Sat, 02 Nov 2024 19:14:49 +0000

This article will change your perspective on working with the Single Responsibility Principle (SRP). The main point is to focus on finding balance when designing object-oriented systems and applying SRP.

Let’s start by explaining the most fundamental ideas to make this easier to understand.

SRP means that each module should have one reason to change. This means that those designing a module should understand who owns it and what type of changes might be needed. And remember, there should be only one reason for change!

Everything is clear: our goal is to break down one module into sub-modules and connect them using a single method (like message passing). The main benefits are preventing spread, keeping changes localized, and reducing the risk of breakage.

Our anti-SRP example represents one of the balances in Object-Oriented Design (OOD). In fact, there are many examples.

Example of Single Responsibility Obsession (Anti-SRP)

Using any principles in an obsessive way reminds us of a silver bullet, but since there is no silver bullet, obsession takes us in the opposite direction of the principle and can have very serious consequences.

For that reason, we should not approach it in an obsessive way; an obsession with SRP turns it into anti-SRP (it means: leads to many small classes/methods and spreads the logic between them.).

Let’s assume we have a data-driven application and need to communicate with database tables. In this case, it would be a some repository. This means our context (which defines the work area and its boundaries) is a database service that fits all the functionality together and cannot be changed.

I will skip the CRC (Class-Responsibility-Collaboration) card and start by declaring functionalities to help demonstrate our context.

First of all, we will start with demonstration of anti-SRP. The idea of the anti-single responsibility principle starts with obsessive use. Let’s start with an example code.

public class GetRepo<T, Guid> where T : class
{
    private readonly DatabaseService _dbService;
    public GetRepo(DatabaseService dbService)
    {
        _dbService = dbService;
    }
    public IEnumerable<T> GetAll()
    {
        return _dbService.ToList();
    }
    public T Get(Guid id)
    {
        if (id == Guid.Empty)
            throw new Exception("Id can not be empty!");

        return _dbService.SingleOrDefault(x => x.Id == id);
    }
}

public class InsertRepo<T> where T : class 
{
    private readonly DatabaseService _dbService;
    public InsertRepo(DatabaseService dbService)
    {
        _dbService = dbService;
    }
    public bool Insert(T instance)
    {
        if (default(T) == instance)
            throw new Exception("Instance can not be empty!");

        int rowEffected = _dbService.Insert(instance);
        return rowEffected != 0;
    }
}

public class UpdateRepo<T> where T : class
{
    private readonly DatabaseService _dbService;
    public UpdateRepo(DatabaseService dbService)
    {
        _dbService = dbService;
    }
    public bool Update(T instance)
    {
        if (default(T) == instance)
            throw new Exception("Instance can not be empty!");

        int rowEffected = _dbService.Update(instance, x => x.Id == instance.Id);
        return rowEffected != 0;
    }
}

public class DeleteRepo<T, Guid> where T : class
{
    private readonly DatabaseService _dbService;
    public DeleteRepo(DatabaseService dbService)
    {
        _dbService = dbService;
    }
    public bool Delete(Guid id)
    {
        if (id == Guid.Empty)
            throw new Exception("Id can not be empty!");

        int rowEffected = _dbService.Delete(x => x.Id == instance.Id);
        return rowEffected != 0;
    }
}

As you can see, we have a DatabaseService class that helps with communication. At the repo level, we have encapsulated classes (each method belonging to the repository is encapsulated in a single class).

Keep in mind that naming conventions can be crappy, but this is just one example!

What this code shows us?

The code above shows the same repositories, divided into CRUD functionalities, using DatabaseService. This example also demonstrates a low-cohesion approach with anti-SRP because overusing SRP leads to anti-SRP and low cohesion.

What is the reason of the code for being anti-SRP? or how we understood this is anti-SRP?

Well, as I mentioned above, the context tells us that “context is a database service that fits all the functionality together and cannot be changed.”. So, we understand that our context will provide us with a single database that will not change, and all operations will belong to this database.

Hmm, we see that our code breaks the localization principle. This means that when there is a change, it should happen in only one place or need only small adjustments.

The context defines change and dependency. So, in our case, when we want to change the repo logic, the changes affect each repo class (DatabaseService implementation is change than all repository can be changed)!

Anti-SRP — overusing single responsibility brings us many problems;

Low cohesion
Violation localization principle
A side effect of changing
Complex maintainability
Increase dependencies

Fixing Anti-SRP Problem With Object-Oriented Balance

Keep in mind everything starts from struggle with complexity. The main idea in SRP and in OOD is to overcome complexity.

So, if we can collect related functionalities to the single unit. It brings us to localize functionalities, and increase cohesion. In this way we balance the complexity.

Let’s create an example.

public class Repository<T, Guid>
{
    private readonly DatabaseService _dbService;

    public Repository(DatabaseService dbService)
    {
        _dbService = dbService;
    }

    public IEnumerable<T> GetAll()
    {
        return _dbService.ToList();
    }
    public T Get(Guid id)
    {
        if (id == Guid.Empty)
            throw new Exception("Id can not be empty!");

        return _dbService.SingleOrDefault(x => x.Id == id);
    }

    public bool Insert(T instance)
    {
        if (default(T) == instance)
            throw new Exception("Instance can not be empty!");

        int rowEffected = _dbService.Insert(instance);
        return rowEffected != 0;
    }

    public bool Update(T instance)
    {
        if (default(T) == instance)
            throw new Exception("Instance can not be empty!");

        int rowEffected = _dbService.Update(instance, x => x.Id == instance.Id);
        return rowEffected != 0;
    }

    public bool Delete(Guid id)
    {
        if (id == Guid.Empty)
            throw new Exception("Id can not be empty!");

        int rowEffected = _dbService.Delete(x => x.Id == instance.Id);
        return rowEffected != 0;
    }
}

As you can see, the code above shows that all functionalities are grouped together, and it does not have any bad smell. There is only one possible reason to change (code is fit for all) this class, meaning the reason for change is consistent throughout.

The result is that when you apply SRP, do not be obsessive; be more flexible and apply it only when needed. Also, be careful with low cohesion.

Conclusion

Overusing the Single Responsibility Principle (SRP) can lead to problems in code structure. When SRP is applied too strictly, it often results in many small classes or methods, each handling only a tiny part of the functionality. This spreads the logic across multiple classes, making the code harder to understand and manage.

Use SRP wisely. Instead of splitting everything into separate classes, aim for high cohesion by grouping related functions together. This balance keeps code simple, focused, and easier to maintain.

Example of Aspect-Oriented Paradigm by Dynamically Decorating Objects with DispatchProxy Class

Rasul — Wed, 23 Oct 2024 08:34:08 +0000

In this article, we will understand how to implement cross-cutting concerns (CCC) with dynamic proxy class.

In the .NET environment, there are many types of implementations of AOP. In our case, we have to focus on dispatching any classes through the proxy object.

As you know AOP is a paradigm that provides better modularity with SoC (separation of concerns). It helps to increase loosely coupling with adding reusable code without modifying existing ones.

In our example, we will use DispatchProxy class which was created in System.Reflection namespace as a .NET Standard library.

Cross-Cutting Concerns Example In .NET Application

DispatchProxy class works based on Interfaces. It provides dynamic wrapping of any type with additional code implementations.

That part is understandable what DispatchProxy is, but we need to be careful because one of the old alternatives is the RealProxy class which provides the same thing but has more than it. RealProxy has cross-process remoting which DispatchProxy is not.

Let’s start with an example;

Assume that we have two different application services. EmailMessageSender and ServerInfoRetriever. As you can see, the names represent what they are able to do.

public interface IEmailMessageSender
{
    public bool TrySendMessage(string to, string subject, string message);
}
public class EmailMessageSender : IEmailMessageSender
{
    // Simulation
    public bool TrySendMessage(string to, string subject, string message)
    {
        Console.WriteLine($"Message sent to the {to}");
        return true; // fake info
    }
}

public record ServerInfo(string State, DateTime ActivationDate, string Version);

 public interface IServerInfoRetriever
 {
     public ServerInfo GetInfo(IPAddress address);
 }
 public class ServerInfoRetriever : IServerInfoRetriever
 {
     // Simulation
     public ServerInfo GetInfo(IPAddress address)
     {
         Console.WriteLine($"{address.ToString()} Server Info Retrieved!");
         return new ServerInfo("In process", DateTime.Now, "1.0.0"); // fake info
     }
 }

To implement our DispatchProxy we should start deriving from it and then implement/write our logic inside it.

public class LoggingDecoratorProxy<T> : DispatchProxy
{
    private readonly Logger _logger;
    public T Target { get; internal set; }

    public LoggingDecoratorProxy() : base()
    {
        _logger = new LoggerConfiguration()
            .WriteTo.Console().CreateLogger(); // used serilog
    }

    protected override object? Invoke(MethodInfo? targetMethod, object?[]? args)
    {
        try
        {
            _logger.Information("{TypeName}.{MethodName}({Arguments})", targetMethod.DeclaringType.Name, targetMethod.Name, args);

            var result = targetMethod.Invoke(Target, args);

            _logger.Information("{TypeName}.{MethodName} returned -> {ReturnValue}", targetMethod.DeclaringType.Name, targetMethod.Name, result);

            return result;
        }
        catch (TargetInvocationException exc)
        {

            _logger.Warning(exc.InnerException, "{TypeName}.{MethodName} threw exception: {Exception}", targetMethod.DeclaringType.Name, targetMethod.Name, exc.InnerException);
            throw exc.InnerException;
        }
    }
}

So, the code above shows that we have to inherit from DispatchProxy and override the Invoke method (which provides invocation dynamically through the reflection) implement additional logic then call the main object method through the reflection.

Everything is clear, we are almost ready to finish.

public static class LoggingDecoratorProxyFactory
{
    public static TInterface Create<TInterface, TConcrete>(TConcrete instance) where TConcrete : class, TInterface where TInterface : class
    {
        if (!typeof(TInterface).IsInterface)
            throw new Exception($"{typeof(TInterface).Name} must be interface!");

        LoggingDecoratorProxy<TInterface> proxy = LoggingDecoratorProxy<TInterface>.Create<TInterface, LoggingDecoratorProxy<TInterface>>()
                    as LoggingDecoratorProxy<TInterface>;

        proxy.Target = instance;
        return proxy as TInterface;
    }
}

The last code provides creating logging cross-cutting concerns through the proxy object. The input param is the instance of interface which will decorate to our proxy logging decorator. Inside the factory decorate dynamically with DispatchProxy.Create method, then wrapped interface returned to the client.

The last part is the app which is running and getting results.

IEmailMessageSender emailSender = LoggingDecoratorProxyFactory.Create<IEmailMessageSender, EmailMessageSender>(new EmailMessageSender());
emailSender.TrySendMessage("resulhsn@gmail.com", "Test", "Hi Rasul");

Console.WriteLine("------------------------------------------------------------------------------------------");

IServerInfoRetriever retriever = LoggingDecoratorProxyFactory.Create<IServerInfoRetriever, ServerInfoRetriever>(new ServerInfoRetriever());
retriever.GetInfo(IPAddress.Parse("127.0.0.1"));

As you can see in our sample we created a cross-cutting concern (logging invocation) and decorated dynamically through the DispatchProxy in runtime with any objects. This way DispatchProxy allowed us to implement AOP in any desired type.

Conclusion

The DispatchProxy class is a powerful tool in the .NET ecosystem for creating dynamic proxies, allowing developers to intercept and modify method invocations at runtime. This capability enables flexible and clean implementations of cross-cutting concerns like logging, caching, and performance monitoring without cluttering the core business logic. By leveraging DispatchProxy, developers can maintain a modular and decoupled codebase, enhancing both maintainability and extensibility.

Stay tuned!

ASP.NET Core Middleware

Rasul — Tue, 22 Oct 2024 12:56:59 +0000

ASP.NET Core middleware (sometimes called middleware, and other times referred to as intermediate software or modules) and it’s also the first post of our series. The reason I chose this topic is to explain the ease and possibilities new technologies provide, and also to dive a bit deeper into the subject.

I want to point out that during interviews, many people haven’t fully understood this topic. Unfortunately, this leads to architectural mistakes, especially in the Presentation layer related to SRP (Single Responsibility Principle).

When ASP.NET Core was built, it was designed with a modular structure. You’ll understand the modular part as you read further. ASP.NET Core uses a pipeline architecture style (incoming requests pass through specific middleware, and each middleware performs its own task. It’s not a design pattern or architectural pattern — it uses an architectural style, which I will discuss in another article). Thanks to the pipeline structure, we can create middleware, add it to the pipeline, and thus form the backbone of the application.

As a result, we can apply AOP (Aspect-Oriented Programming) features and add small modules, which helps us avoid code repetition and provides a flexible and expandable framework structure.

Of course, I know I have summarized this very briefly, but the goal is to explain the middleware structure. In another series, I will also touch on topics that intersect with middleware, like self-hosting, OWIN, and built-in features.

Let’s Start

Before diving into the details, middleware has two main functions:

It can check whether to call the next middleware in the pipeline.
It can perform actions either before or after the next middleware (Chain of Responsibility — COR).

ASP.NET Core is actually an abstract web framework. It provides us with an OWIN-based or application backbone that allows us to create a pipeline by attaching our own application as middleware.

When more than one middleware is added, each middleware in the pipeline is connected like a chain, and they can call each other. Structurally, it uses the Chain of Responsibility pattern, often shortened to COR or chain (I’ll sometimes use the word chain).

ASP.NET Core creates a response to an incoming request and sends it back to users (clients). Middleware is responsible for handling these incoming requests and responses as part of their tasks.

Frameworks are abstract by design. If we don’t add the MVC (web app) component to the pipeline, the application won’t give any result because there’s no middleware in the pipeline to process or handle it.

To work with MVC, we need to integrate the MVC middleware into the ASP.NET Core pipeline. This allows us to get results from our Controller/Action requests (the Controller is executed, returns a specific ActionResult, and this ActionResult is executed, writing the result to HttpResponse. The user then sees this). Alternatively, we can add our own custom middleware to process requests or responses and get results.

Actually, we don’t have an MVC middleware here. What we have is a Route middleware, which handles the incoming request by mapping the URI segments to RouteData. This later allows the execution of our Action using reflection.

What can we do with middleware?

In applications, certain modules are usually used, such as:

Logging
Authentication / Authorization
Routing
Static File handling
Response Caching
URL Rewriting

These are examples of middleware structures.

Before moving to examples, I want to note that we use the Chain of Responsibility (COR) pattern to create middleware!

Since middleware calls each other in a chain (I’ll use the word “chain” to make it easier for everyone to understand), they take an object of the RequestDelegate type. This object is the instance of the next or previous middleware in the chain (because of the chain structure), and it’s in your control whether to call the next one. In the following parts of my article, I’ll explain the types of middleware in more detail and how to use them in the chain.

There are two ways to create custom middleware in our applications:

Using the IMiddleware interface (strongly-typed middleware)

public class StrongyTypedMiddleware : IMiddleware
    {
        public async Task InvokeAsync(HttpContext context, RequestDelegate next)
        {
            await context.Response.WriteAsync("Hello !");
        }
    }

Using a convention-based approach

public class ConventionBasedMiddleware
    {
        private RequestDelegate _next;
        public ConventionBasedMiddleware(RequestDelegate next) { _next = next; }
        public async Task InvokeAsync(HttpContext context)
        {
            await context.Response.WriteAsync("Hello !");
        }
    }

The InvokeAsync method must always be used, and as an input parameter, it must take an object of the HttpContext type. The return type of this method should be a Task.

For the middleware to work in a Chain of Responsibility (COR) style, it needs the next middleware object. This is optional — if you don’t want to call the next middleware, you don’t have to define it. This shows how flexible the middleware architecture can be.

Now, we will continue our article with examples of convention-based middleware.

What are the types of middleware?

In general, there are 4 types of middleware. These types are just abstract definitions, meaning that when creating middleware, certain functional abstract types are formed. The way these types are used is entirely based on the Chain of Responsibility (COR) pattern (the chain structure is demonstrated here).

The functional middleware types used in ASP.NET Core applications are:

Response-editing middleware (edits the response)
Request-editing middleware (edits the request)
Short-circuiting middleware (stops the chain and doesn’t pass the request to the next middleware)
Content-generating middleware (generates content or output)

We won’t follow the order listed above, but pay attention because the order is important based on the functional types!

Content-generating middleware: This is one of the most important types of middleware. MVC is built on this category. Its purpose is to process the incoming request and produce a result for the end user.

public class ContentMiddleware
    {
        private RequestDelegate _next;

        public ContentMiddleware(RequestDelegate next) { _next = next; }

        public async Task InvokeAsync(HttpContext context)
        {
            if (context.Request.Path.ToString().ToLower().Contains("Home/Index"))
                await context.Response.WriteAsync("Simple content generator middleware!");
            else
                await _next(context);
        }
    }

Short-circuiting middleware: As an example of short-circuiting middleware, let’s use MVC again. If the view returned by your controller is already cached, there’s no need to continue through the middleware pipeline all the way to the MVC middleware. To optimize and structure the work correctly, we first go to the short-circuiting middleware, where we check if the data is already in the cache. If it is, we take the cached data and send it as the response. But if the data is not in the cache, we continue through the middleware pipeline and run the middleware that will generate the result.

// MOCK Cache sample. This class simulate caching , please use original caching middleware!
    public class ShortCircuitMiddleware
    {
        private RequestDelegate _next;

        public ShortCircuitMiddleware(RequestDelegate next) { _next = next; }

        public async Task InvokeAsync(HttpContext context, IMemoryCache cache)
        {
            string key = GetCacheKey(context);
            if(cache.TryGetValue<CacheItem>(key,out CacheItem item))
            {
                if(IsValid(context, item))
                {
                    await context.Response.WriteAsync(item.ResponseStr);
                    return;
                }
            }

            await _next(context);
        }

        // Simulate
        private string GetCacheKey(HttpContext context)
        {
            return null;
        }

        // Simulate
        private bool IsValid(HttpContext context,CacheItem item)
        {
            return true;
        }
    }

Request-editing middleware: is designed to change the structure of the incoming request before it reaches the next components (middlewares) in the chain. This is useful when there is a change in the platform’s working principles or to help the next middlewares in the chain process the request more easily. It’s like transforming the request into a different schema.

public class RequestEditingMiddleware
    {
        private RequestDelegate _next;
        public RequestEditingMiddleware(RequestDelegate next) { _next = next; }
        public async Task InvokeAsync(HttpContext context, IMemoryCache cache)
        {
            if(context.Request.Path.Value.Equals("/Home"))
            {
                context.Request.Path.Add(new PathString("Index"));
            }

            await _next(context);
        }
    }

Response-editing middleware: is a type of middleware that affects or manipulates the result produced by other middlewares in the chain that process the request and generate the response. For example, you can use this type of middleware to handle errors over HTTP. If a 404 error is encountered in the generated response, this functional middleware can be written to handle it, such as adding a custom 404 page or performing similar tasks.

// HttpErrorMiddleware Example
    public class ResponseEditingMiddleware
    {
        private RequestDelegate _next;
        private IHostingEnvironment _hosting;

        public ResponseEditingMiddleware(RequestDelegate next, IHostingEnvironment envrionment) { _next = next; _hosting = envrionment; }

        public async Task InvokeAsync(HttpContext context)
        {
            try
            {
                await _next(context);
                if (!_hosting.IsDevelopment())
                {
                    if(ExistsError(context))
                    {
                        switch (context.Response.StatusCode)
                        {
                            case 404:
                            case 403:
                            case 400:
                                await context.Response.WriteAsync($"Occur error status code {context.Response.StatusCode}!");
                                break;
                        }
                    }
                }
            }
            catch {
                // Sample
                await context.Response.WriteAsync($"Occur unknow error. Please report the site@admin.com!");
                return;
            } 
        }

        // Simulate.
        private bool ExistsError(HttpContext contex)
        {
            return true;
        }
    }

Here, the other middleware in the chain is called first because the called middleware generates a response. Afterward, we intervene with the response by adding our necessary code, which allows us to manipulate the output. This shows us the flexibility of our chain structure.

Creating and Registering Middleware In the Pipeline

We define our middleware in the Configure method inside the Startup class (there are a few ways to do this, but I will show the classic and common one). This is how our middleware runs.

In fact, in the Configure method, we create our pipeline by registering our middleware, and for this, we use the IApplicationBuilder interface.

The order of registration in the Configure method is important. Each middleware added is automatically connected to the next one in the chain (using builder and wrapper patterns behind the scenes). Based on the types of middleware we described earlier, they need to be ordered in a specific way. Let’s first look at how to register them, and then I will return to this topic later in the article.

There are 3 ways to register middleware:

With the Use() method
With the Run() method
With the UseMiddleware() method

Use() Method — The purpose of this method is to quickly register a delegate in the pipeline, and all of our abstract functional types can use this. It provides the method signature for “HttpContext” and “RequestDelegate,” meaning it takes one request container and passes the next middleware in the chain (using a Func delegate). For example:

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    app.Use(async (context, next) => {
         // Do work that doesn't write to the response.
          await next.Invoke();
         // Do logging or other work that doesn't write to the response.         
      });        
}

Run() Method — The purpose of this method is to quickly register a delegate in the pipeline, but it can only logically take on the role of content-generating middleware among the abstract functional types mentioned above. It provides the method signature for “HttpContext,” meaning it passes one request container object (with the chain delegate being RequestDelegate). For example:

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
     app.Run(async (context) => {
        await context.Response.WriteAsync("Hi , generated for clients!");     
     });       
}

Map() Method — The purpose of this method is not to register middleware in the pipeline like Use or Run. Instead, it adds a new pipeline, allowing us to create an inner or nested pipeline (it enables creating a pipeline within a pipeline).

It allows us to create branches or sub-branches within a branch. However, there is an important difference: when we register, it requires a URI segment. This means that the inner pipeline will only work if the incoming request matches this segment, so it operates under a segment condition. We use this method when the application consists of multiple modules or in specific situations. It provides the method signature for an IApplicationBuilder object (using an Action delegate). For example:

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
     app.Map("/index", (appBuilder) => {
        appBuilder.Use(async (context, next) =>
        {
             await next.Invoke();
        });        
        appBuilder.Use(async (context, next) =>
        {
            await context.Response.WriteAsync("Hi generated response for clients! (MAP) route /index");
        });

     });

     app.Run(async context => {
        await context.Response.WriteAsync("Hi generated response for clients!");
     });       
}

Best Practice — We use the UseMiddleware<>() and extension method to register middleware in a reusable, flexible, and configurable way. This method allows us to register middleware easily with its generic structure, enabling us to register it as a class.

Let’s create an example using one of the examples from above:

public static class AppBuilderErrorMiddlewareExtensions
    {
        public static IApplicationBuilder UseHttpErrorHandler(this IApplicationBuilder app)
        {
            if (app == null)
                throw new ArgumentNullException(nameof(IApplicationBuilder));

            return app.UseMiddleware<ResponseEditingMiddleware>();
        }
    }

In the Pipeline — COR and Ordering

We will create a sample pipeline with the example middleware we built.

As I mentioned, the order of the abstract functional types in the chain is very important. Why? Because the work done by the abstract function affects the chain structure. Any wrong order in the chain can be critical for security, performance, and proper functioning (if the chain structure is incorrect, it can cause the application not to work correctly and even lead to errors). Let’s explain the order one by one:

Response-editing middleware comes first. This is because it concerns the output (response) of the processed request, not the incoming request. For example, let’s use our own example, the “HttpErrorMiddleware.” When we receive a request, this middleware will be the first to run. It will call the next middleware in the chain without processing it first. After the output (response) is generated, when it returns (upward in the pipeline), the middleware will manipulate the last produced output (response) to handle it finally. After the chain runs to MVC, if MVC returns an HTTP error, the output will be sent back through the chain. Our “HttpErrorMiddleware” runs last to provide a user-friendly output (it manipulates the output). You’ll notice that exception-handling middleware is always defined first. This is because if you call the method stack first, you can handle it last, which shows the flexibility of the COR.
Request-editing middleware comes second because it may be necessary to transform the incoming request into a more flexible format for the next middleware to handle. For example, suppose we have a large shopping site that used PHP and has now transitioned to ASP.NET Core. Some URLs had to change. To prevent broken links and losing old users, we used URL rewriting. For instance, when a user used the URL www.shop.com/Home?/PAGE=1, we transformed it to www.shop.com/Home/Index/1 in MVC. How does this work? The incoming request www.shop.com/Home?/PAGE=1 will come in, and our second middleware will analyze this URL and convert it into a format that MVC can understand.
Short-circuiting middleware comes third because it produces a result quickly, stopping the chain before the middleware that generates the output. For optimization, it sends the result (response) back without going to the next middleware. An example is caching; if the previous result is cached, there is no need to perform the operation again. The result is produced and sent back without reaching the content-generating middleware. It can also be used to prevent bot attacks.
Content-generating middleware is the last functional type of middleware. Its job is straightforward: to generate content, output, or results based on the incoming request. Finally, this middleware produces the necessary output. For example, MVC falls into this category. In a brief explanation of how MVC works, the UseMvc() middleware is built on our routing middleware. The incoming request is analyzed and parsed according to its route path, creating RouteData. Then, the specified Controller and Action are executed, and when the process is finished, the generated output is added to the HttpResponse class within the HttpContext and sent back to the user.

Let’s conclude our article by registering all the middleware we have created in the ‘Configure’ method. All the added middleware is convention-based.

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    app.UseHttpErrorHandler();   // 1. Response Editing middleware
    app.UseRequestModifier();   //  2. Request Editing middleware
    app.UseCacheResponse();    //   3. Short Circuit middleware
    app.UseContentGenerator(); //   4. Content/Response generator middleware
}

I hope this has been helpful and that you have grasped the concept in a deeper way.

Stay tuned!

Validation of Complex Objects in Multi-Lingual Environment Using DynamicVNET With ASP.NET Core Example

Rasul — Mon, 14 Oct 2024 10:12:43 +0000

In this article, we will see why we created our own validation library and how it was needed.

Everything started in 2018 when we needed a flexible and easy-to-use validation tool for a commercial project. The goal was to create a library that could validate POCO (Plain Old CLR Objects) while following the Single Responsibility Principle (SRP). This is how DynamicVNET was born, a rule-based validation library that has now been downloaded over 13,000 times on NuGet.

The main aim of DynamicVNET was to give developers a solution they could easily add to their projects, so they would not have to repeat the same validation code. With a simple API, developers can create validation rules that work at runtime, making it perfect for projects that need flexible validation options.

Let’s start to demonstrate our library with asp.net core example.

ASP.NET Core Environment Preperation

In our example, we have asp.net core web api which has the phone validation controller and validate action that accepts input argument phone number object. So, in the requested action, we need to validate the input argument with rules defined by languages.

To implement this scenario we have to integrate the validation library into the request lifecycle. So, in our case, we will use an action filter to achieve this.

First of all, we need to install the DynamicVNET Nuget package. Also, keep that we are using the stable version of it.

Install-Package DynamicVNET

Let’s start using it.

To bind validation for each input parameter we have to define BindValidator attribute.

[System.AttributeUsage(AttributeTargets.Parameter, Inherited = false, AllowMultiple = false)]
public class BindValidator : Attribute { }

So simple to understand, next step is to declare action filter which provide validation.

public class LangValidatorFilter : IActionFilter
 {
     private readonly ProfileValidator _validator;

     public LangValidatorFilter(ProfileValidator validator)
     {
         this._validator = validator;
     }

     public void OnActionExecuted(ActionExecutedContext context) { }

     public void OnActionExecuting(ActionExecutingContext context)
     {
         if (context.ActionArguments != null && context.ActionArguments.Count >= 1)
         {
             var validatableArguments = context.ActionDescriptor.Parameters
                                                 .Where(ar => context.ActionArguments.ContainsKey(ar.Name))
                                                 .Select(s => s as ControllerParameterDescriptor)
                                                 .Where(a => a != null && a.ParameterInfo.CustomAttributes.Any(at => at.AttributeType == typeof(BindValidator)));

             // That part can be replaced by any globalization technique.
             string langName = context.HttpContext.Request.Query["culture"].ToString() ?? "en";

             foreach (var argument in validatableArguments)
             {
                 try
                 {
                     var validationResults = this._validator.Validate(langName, context.ActionArguments[argument.Name]);

                     if (validationResults != null && validationResults.Any())
                     {
                         foreach (var validationResultItem in validationResults)
                         {
                             context.ModelState.AddModelError(validationResultItem.MemberName,
                                                                 validationResultItem.ErrorMessage);
                         }
                     }
                 }
                 catch (InvalidProfileNameException)
                 {
                     context.Result = new StatusCodeResult(404);
                     break;
                 }
             }
         }
     }
 }

As you can see code above, in the executing step we are getting arguments marked by BindValidator attributes. After that, we got the language key and escape the globalization stuff. Instead, we used query param for language detection (just for simulation). In the last step we looped all arguments with ProfileValidator (I should add ProfileValidator is simple, it provides validation rules by string name) and validated them and got the results. We then filled in the errors using modelState.

After all this steps we should implement factory for action filter.

[System.AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, Inherited = false, AllowMultiple = false)]
public class VNETFilterFactory : Attribute, IFilterFactory
{
    public bool IsReusable => false;

    public IFilterMetadata CreateInstance(IServiceProvider serviceProvider)
    {
        DynamicVNET.Lib.ProfileValidator instance = serviceProvider.GetService(typeof(DynamicVNET.Lib.ProfileValidator)) as DynamicVNET.Lib.ProfileValidator;
        return new LangValidatorFilter(instance);
    }
}

Last part is defined configuration from ConfigureServices. To apply best practice we are using extension method.

public static class DIExtensions
{
    public static IServiceCollection AddProfileValidation(this IServiceCollection services,
                                                                    Action<ProfileValidator> setup)
    {
        if (services == null)
            throw new ArgumentNullException(nameof(services));

        if (setup == null)
            throw new ArgumentNullException(nameof(ProfileValidator));

        ProfileValidator validator = new ProfileValidator();
        setup(validator);

        services.AddSingleton(validator);

        return services;
    }
}

Everything is ready to go.

ASP.NET Core Web Api Example

As we explain above we are planning to validate phone number through API controller.

The first of all we need to declare request and response models.

public class PhoneNumberRequest
{
    public byte AttempsCount { get; set; }
    public string Name { get; set; }
    public string NumberPrefix { get; set; }
    public string Number { get; set; }
}

public sealed class PhoneNumberValidationException(string member, string message)
{
    public string Member { get; } = member;
    public string Message { get; } = message;
}

public class PhoneNumberResponse
{
    public bool IsSuccess => this.Errors == null || !this.Errors.Any();
    public IEnumerable<PhoneNumberValidationException> Errors { get; }

    public PhoneNumberResponse(IEnumerable<PhoneNumberValidationException> errors)
    {
        this.Errors = errors;
    }

    public PhoneNumberResponse() : this(null) { }
}

Everything is clear, so next step is declare api controller.

[ApiController]
[Route("[controller]")]
public class PhoneNumberValidatorController : ControllerBase
{
    [HttpGet]
    [VNETFilterFactory]
    public IActionResult Validate([BindValidator][FromBody] PhoneNumberRequest model)
    {
        if (!ModelState.IsValid)
        {
            var errors = ModelState.Select(x => new PhoneNumberValidationException(x.Key, x.Value.Errors.FirstOrDefault()?.ErrorMessage));
            return Ok(new PhoneNumberResponse(errors));
        }

        return Ok(new PhoneNumberResponse());                
    }
}

The controller contains the Validate action method which provides validation of all types of numbers. So, as you can see BindValidator declared with argument. In case of an invalid modelState the action returns errors otherwise returns an empty response with a success message.

Our sample is almost ready, only validation methods are missing. Let’s continue with defining validation through DynamicVNET. Also according to readability, we will use method-based validation instead of strongly typed one.

builder.Services.AddProfileValidation(validator =>
{
    validator.AddProfile<PhoneNumberRequest>("az", marker =>
    {
        marker
        .Required(x => x.Name)
        .Required(x => x.Number)
        .Range(x => x.AttempsCount, 1, 10)
        .StringLen(x => x.Number, 12)
        .Required(x => x.NumberPrefix)
        .Predicate(x => x.NumberPrefix.StartsWith("+994"), "Invalid prefix, it should start with +994!");
    });

    validator.AddProfile<PhoneNumberRequest>("en", marker =>
    {
        marker
        .Required(x => x.Name)
        .Required(x => x.Number)
        .Range(x => x.AttempsCount, 1, 5)
        .StringLen(x => x.Number, 10);
    });

    validator.AddProfile<PhoneNumberRequest>("fr", marker =>
    {
        marker
        .Required(x => x.Name)
        .Required(x => x.Number)
        .Range(x => x.AttempsCount, 1, 3)
        .StringLen(x => x.Number, 9)
        .Required(x => x.NumberPrefix)
        .Predicate(x => x.NumberPrefix.StartsWith("+33"), "Invalid prefix, it should start with +33!");
    });
});

As you can see in ConfigureServices we just declared AddProfileValidation and setup needed validations by culture. But the main point is that DynamicVNET provides a declarative approach for achieving flexible validations.

Let’s see the result through the postman.

When inputs are not valid we are getting error which in our case is invalid number prefix. Also we can apply to another language.

As you can see how it works in a very simple and straightforward way.

As a result of our example easily shows how we can work flexibly and validate very complex structures.

Conclusion

DynamicVNET is a lightweight, flexible, and easy-to-use validation library designed for .NET applications. It provides a dynamic, rule-based validation system that can be integrated seamlessly into your projects, offering robust validation logic without the need for hard-coded rules. It also includes various methods and extension points, making it adaptable to all types of validations.

One of the strengths of DynamicVNET is how it integrates effortlessly with ASP.NET Core. By using DynamicVNET in your ASP.NET Core applications, you can easily validate incoming request models (e.g., in API controllers) without tightly coupling your validation logic to specific models or methods. This ensures clean separation of concerns, allowing you to maintain a well-structured, SRP-compliant application. Its flexibility enables developers to adapt validation rules dynamically, which is especially useful in applications with frequently changing business requirements.

Whether you are building an API, a web app or a service-based architecture, DynamicVNET can help you keep your validation logic clean, reusable, and maintainable. Also you can check it out here.

Understanding Builder Design Pattern

Rasul — Fri, 02 Aug 2024 10:50:05 +0000

The Builder is a pattern that belongs to the creational design patterns family. What makes it special is its ability to provide abstraction to the client by encapsulating different types of the same object with various combinations during the creation process. This creates reusable and safe access points. Another key feature of this design pattern is its capability to handle the creation of objects that are not within the responsibility of the object itself, thereby adhering to the Single Responsibility Principle (SRP). This feature supports modularity and code maintainability.

Problem

The complexity that arises in the creation of instances or initialization of instances of classes. In other words, the complexity due to combinations and dependencies during the creation of a family of objects leads to complex constructors and tightly coupled code.

Context

The goal of the Builder design pattern is not to manage the increasing number of constructors due to additional features. The main idea is to facilitate the dynamic creation of different types/representations/combinations of objects of a class.

By different types, we mean enabling the client to manage the creation process step by step. We provide the client with the flexibility to create the desired combination of the object.

Another main goal of this pattern is to address the complexity of creating objects with different combinations using a single solution. For example, if an object can be created in 3 or 4 different ways, the aim is to ensure that any new combinations in the future can be handled by adding properties rather than changing the creation mechanism, thus maintaining reusability and maintainability.

In other words, it abstracts the complexity of creating object families by encapsulating the step-by-step creation process in a single place.

When to Use It?

When creating an object involves many complex steps.
When there are different combinations for creating an object.
When the object needs to be created according to specific access types.
When future architectural changes might introduce new object combinations.

In these situations, the Builder design pattern can be used to solve the problem.

However, one of the biggest mistakes with design patterns is treating them as DOGMAS. This is a flawed approach because it can lead us towards anti-patterns.

The DOGMA approach can lead to Curved Architecture in design patterns!
The word COMBINATION (representation) implies that each object is created in different ways and represented in different types!
CLIENT: The side which using builder!

GoF Definition

“Separate the construction of a complex object from its representation so that the same construction process can create different representations.“

By abstracting the complexity of nested class constructors and delegating responsibilities to different objects, the Builder pattern allows the same creation algorithm to be applied to every member of an object family.

For those not very familiar with OOP/OOAD and paradigm techniques, this might seem complex. However, it’s not very different from what we expressed in the first part of the article — just a bit more technical.

The Builder design pattern is based on algorithmic dependencies. It should allow for changes in the algorithms associated with any object family. Therefore, changing algorithmic structures should create a separate class hierarchy (potentially changing algorithms should be isolated).

Responsibilities

Builder: An abstract interface that defines the process for creating parts of a product.
ConcreteBuilder: A concrete implementation of the Builder class that constructs the parts of the product.
Director: A class that uses the Builder interface to create complex and varied types of objects according to specific rules. It manages the order of method calls and which methods are called, encapsulating the complexity of creating the object with certain rules.
Product: The object that is being created.

Real Case Example

The Builder design pattern has different implementations depending on the programming language. In our example, we will implement a more complex structure: the SQLite Query Generator. We will use a combination of the Builder pattern and the State pattern.

Before we dive into the example, let’s take a look at our class diagram.

The role of the Director is played by DeclarativeSelectQueryGenerator (we will not delve into this part to keep the article concise; for details, click here). Interfaces are used for contract encapsulation to manage compile-time states, and the builder classes responsible for generating queries are present in our diagram.

Let’s continue with the example from a TDD perspective, noting that we will not examine all the classes in detail.

Our main goal is to set up a declarative SQLite query generation structure. To do this, let’s create our example usage scenario (test-first approach).

[TestMethod]
public void Build_WhenCalledSelectAndWhereCondition_ShouldReturnCorrectQuery()
{
    const string expectedQuery = @"SELECT * FROM Users WHERE Users.Id > 5 OR Users.Name = 'Rasul' ";
    string generatedQuery = new SQLiteSelectQueryBuilder()
                                .Select()
                                .From("Users")
                                .Where("Id")
                                .GreaterThan(5)
                                .Or("Name")
                                .EqualTo("Rasul")
                                .Build()
    Assert.AreEqual(expectedQuery, generatedQuery);
}

Our example class shows that within SQLite, query structures are provided through methods with different stages (fluent/method chaining). To design this, we plan to apply the fluent API concept. However, an important detail is that after the From method is called, it should no longer appear in IntelliSense, meaning a state change is involved.

To design this, we need to manage the state using the State pattern. However, in our case, we need to solve this in compile time. Therefore, we can achieve this using the well-known interface segregation and information-hiding principles.

First, let’s take a look at the Select example.

public class SQLiteSelectQueryBuilder : ChainQueryBuilder,
                                            ISelectQueryBuilder
{
    const string SELECT = "SELECT";
    const string ALL = "*";
    private IEnumerable<string> _columns;
    private bool HasColumns => _columns != null && _columns.Any();
    public SQLiteSelectQueryBuilder(params string[] columns) : base(null) => _columns = columns;
    public ISelectQueryBuilder Select(params string[] columns)
    {
        _columns = columns;
        return this;
    }
    public ISelectQueryBuilder Select(IEnumerable<string> columns)
    {
        _columns = columns;
        return this;
    }
    public IFromQueryBuilder From(string table)
        => new SQLiteFromQueryBuilder(table, this);
    protected override string Build(QueryBuilderContext? context = null)
    {
        if (HasColumns)
        {
            string aggregatedColumn = string.Empty;
            if (context.HasValue)
            {
                IEnumerable<string> columns = _columns.Select(x => $"{context.Value.Alias}.{x}");
                aggregatedColumn = string.Join(",", columns);
            }
            else
            {
                aggregatedColumn = string.Join(",", _columns);
            }
            return $"{SELECT} {aggregatedColumn} ";
        }

        return $"{SELECT} {ALL} ";
    }
    public override string Build() => Build(null);
}

As seen in the code structure, each method for the Select query returns its own state object. Examining the builder and state approaches used in this class, we see that because there are more than two management options for constructing a query, it supports a step-by-step approach (which is where the Builder pattern comes into play). Additionally, to enable step-by-step construction of other queries (building), transitions between interfaces are used (e.g., the From example).

Let’s continue with the From example.

public class SQLiteFromQueryBuilder : ChainQueryBuilder,
                                        IFromQueryBuilder
{
    const string FROM = "FROM";
    const string AS = "AS";
    private readonly string _tableName;
    private string _alias;
    private bool HasAlias => !string.IsNullOrWhiteSpace(_alias);
    private string DefaultAlias => HasAlias ? _alias : _tableName;
    public SQLiteFromQueryBuilder(string tableName, ChainQueryBuilder innerQueryBuilder) : base(innerQueryBuilder)
        => _tableName = tableName ?? throw new ArgumentNullException(nameof(tableName));
    public IFromQueryBuilder Alias(string alias)
    {
        _alias = alias;
        return this;
    }
    public IWhereQueryBuilder Where(string column)
        => new SQLiteWhereQueryBuilder(DefaultAlias, column, this);
    public IOrderByQueryBuilder OrderBy()
        => new SQLiteOrderByQueryBuilder(this);
    protected override string Build(QueryBuilderContext? context = null)
    {
        StringBuilder builder = new StringBuilder();
        context = new QueryBuilderContext(DefaultAlias);
        BuildChain(builder, context);
        if (HasAlias)
        {
            builder.Append($"{FROM} {_tableName} {AS} {_alias} ");
        }
        else
        {
            builder.Append($"{FROM} {_tableName} ");
        }
        return builder.ToString();
    }
    public override string Build() => Build(null);
}

The same approach is used here as well. Within the class, the From structure can be set up using two different types of methods. This flexibility is provided by the Builder design pattern.

Let’s take a look at the Where example.

public class SQLiteWhereQueryBuilder : ChainQueryBuilder,
                                        IWhereQueryBuilder,
                                        IConditionQueryBuilder<IWhereQueryBuilder>
{
    const string WHERE = "WHERE";
    readonly Dictionary<string, string> Symbols = new Dictionary<string, string>()
    {
        {nameof(IWhereQueryBuilder.EqualTo), "="},
        {nameof(IWhereQueryBuilder.NotEqualTo), "!="},
        {nameof(IWhereQueryBuilder.GreaterThan), ">"},
        {nameof(IWhereQueryBuilder.LessThan), "<"},
        {nameof(IWhereQueryBuilder.In), "IN"},
        {nameof(IConditionQueryBuilder<IWhereQueryBuilder>.And), "AND"},
        {nameof(IConditionQueryBuilder<IWhereQueryBuilder>.Or), "OR"},
    };
    private readonly string _firstColumn;
    private readonly string _alias;
    private readonly List<string> _conditions;
    public SQLiteWhereQueryBuilder(string alias, string firstColumn, ChainQueryBuilder innerQueryBuilder) : base(innerQueryBuilder)
    {
        _firstColumn = firstColumn ?? throw new ArgumentNullException(nameof(firstColumn));
        _alias = alias ?? throw new ArgumentNullException(nameof(alias));

        _conditions = new List<string>
        {
            $"{WHERE} {GetColumnName(_firstColumn)} "
        };
    }
    public IConditionQueryBuilder<IWhereQueryBuilder> EqualTo(string value)
    {
        _conditions.Add($"{Symbols[nameof(IWhereQueryBuilder.EqualTo)]} '{value}' ");
        return this;
    }
    public IConditionQueryBuilder<IWhereQueryBuilder> EqualTo(double value)
    {
        _conditions.Add($"{Symbols[nameof(IWhereQueryBuilder.EqualTo)]} {value} ");
        return this;
    }
    public IConditionQueryBuilder<IWhereQueryBuilder> EqualTo(int value)
    {
        _conditions.Add($"{Symbols[nameof(IWhereQueryBuilder.EqualTo)]} {value} ");
        return this;
    }
    public IConditionQueryBuilder<IWhereQueryBuilder> NotEqualTo(string value)
    {
        _conditions.Add($"{Symbols[nameof(IWhereQueryBuilder.NotEqualTo)]} '{value}' ");
        return this;
    }
    public IConditionQueryBuilder<IWhereQueryBuilder> NotEqualTo(double value)
    {
        _conditions.Add($"{Symbols[nameof(IWhereQueryBuilder.NotEqualTo)]} {value} ");
        return this;
    }
    public IConditionQueryBuilder<IWhereQueryBuilder> NotEqualTo(int value)
    {
        _conditions.Add($"{Symbols[nameof(IWhereQueryBuilder.NotEqualTo)]} {value} ");
        return this;
    }
    public IConditionQueryBuilder<IWhereQueryBuilder> In(params string[] values)
    {
        IEnumerable<string> convertedStrings = values.Select(x => $"'{x}'");
        string joinedValues = string.Join(",", convertedStrings);

        _conditions.Add($"{Symbols[nameof(IWhereQueryBuilder.In)]} ({joinedValues}) ");
        return this;
    }
    public IConditionQueryBuilder<IWhereQueryBuilder> In(params double[] values)
    {
        string joinedValues = string.Join(",", values);

        _conditions.Add($"{Symbols[nameof(IWhereQueryBuilder.In)]} ({joinedValues}) ");
        return this;
    }
    public IConditionQueryBuilder<IWhereQueryBuilder> In(params int[] values)
    {
        string joinedValues = string.Join(",", values);

        _conditions.Add($"{Symbols[nameof(IWhereQueryBuilder.In)]} ({joinedValues}) ");
        return this;
    }
    public IConditionQueryBuilder<IWhereQueryBuilder> GreaterThan(int value)
    {
        _conditions.Add($"{Symbols[nameof(IWhereQueryBuilder.GreaterThan)]} {value} ");
        return this;
    }
    public IConditionQueryBuilder<IWhereQueryBuilder> GreaterThan(double value)
    {
        _conditions.Add($"{Symbols[nameof(IWhereQueryBuilder.GreaterThan)]} {value} ");
        return this;
    }
    public IConditionQueryBuilder<IWhereQueryBuilder> LessThan(int value)
    {
        _conditions.Add($"{Symbols[nameof(IWhereQueryBuilder.LessThan)]} {value} ");
        return this;
    }
    public IConditionQueryBuilder<IWhereQueryBuilder> LessThan(decimal value)
    {
        _conditions.Add($"{Symbols[nameof(IWhereQueryBuilder.LessThan)]} {value} ");
        return this;
    }
    public IOrderByQueryBuilder OrderBy()
      => new SQLiteOrderByQueryBuilder(this);

    public IWhereQueryBuilder And(string column)
    {
        _conditions.Add($"{Symbols[nameof(IConditionQueryBuilder<IWhereQueryBuilder>.And)]} {GetColumnName(column)} ");
        return this;
    }
    public IWhereQueryBuilder Or(string column)
    {
        _conditions.Add($"{Symbols[nameof(IConditionQueryBuilder<IWhereQueryBuilder>.Or)]} {GetColumnName(column)} ");
        return this;
    }
    protected override string Build(QueryBuilderContext? context = null)
    {
        StringBuilder builder = new StringBuilder();
        BuildChain(builder, context);
        if (_conditions.Any())
        {
            foreach (string condition in _conditions)
            {
                builder.Append(condition);
            }
        }
        return builder.ToString();
    }
    public override string Build() => Build(null);
    private string GetColumnName(string column)
        => $"{_alias}.{column}";
}

Although it might seem a bit complex, the Where clause structure allows us to create logical blocks in many different types and should be flexible. Therefore, it includes all the functions within itself. In the shown code, all conditions are collected and then combined into a single query using the builder and chaining methods. This approach is applied similarly to all subqueries using the Builder pattern.

The code is lengthy, so I won’t dwell on it too much, but for those interested, you can view the complete code here.

Conclusion

The Builder design pattern is an important pattern that provides flexibility and control in software development. This pattern simplifies the creation of complex objects and enables writing more readable and maintainable code.

It is particularly useful in systems that need to handle multiple requests simultaneously. Most importantly, this pattern allows detailed and step-by-step control of the object creation process. As a result, client code becomes independent of how the object is created, enhancing code reusability. Additionally, it makes it easy to create objects in different configurations.

Stay Tuned!

Understanding LINQ While Writing Your Own

Rasul — Wed, 03 Jul 2024 09:10:31 +0000

In this article, we will learn what LINQ is and how it works behind the scenes while writing your own LINQ methods.

LINQ stands for Language-Integrated Query. It is one of the most powerful tools in the .NET platform that provides an abstract way of querying data for various data sources. In other words, it abstracts away us from concrete data source dependencies. Every data source has its language and format. But with LINQ in your arsenal, you don’t need to talk to the data source using its language. You use a central language (LINQ in this case) and it acts like an API Gateway to different data sources. LINQ supports common query methods that can be used with various data sources like runtime objects, relational databases, and XML.

A small note before getting started: We will focus on the LINQ to Object, and will talk about other LINQ forms in our next articles.

Language-Integrated Query

LINQ lives in System.Linq.dll and is a query language (library + syntax). The main purpose of it is to provide a consistent, declarative, and type-safe way to query and manipulate data across various sources.

The image above shows how the LINQ works within the .NET frameworks with various data sources.

LINQ to Objects lets you query collections like arrays and lists in memory using LINQ syntax. It offers powerful filtering, ordering, and grouping capabilities directly on these collections.
LINQ to Entities is used to query databases through the Entity Framework, an ORM (Object-Relational Mapper). It allows you to write queries in C# instead of SQL, translating them into SQL for database execution.
LINQ to SQL (old) is a component that provides a runtime infrastructure for managing relational data as objects. It translates LINQ queries into SQL queries to interact directly with the SQL Server database.
LINQ to DataSet enables querying and manipulating data stored in ADO.NET DataSets. It is useful for working with disconnected data that is retrieved from databases and stored in memory.
LINQ to XML provides the ability to query and manipulate XML data using LINQ. It simplifies working with XML documents by offering a more readable and concise way to handle XML data.

In built-in LINQ there are most common methods for querying from data sources.

Where: Filters elements based on a predicate.
Select: Projects each element into a new form.
OrderBy: Sorts elements in ascending/descending order.
GroupBy: Groups elements that share a common attribute.
Join: Joins two sequences based on a key.

To view the list, please click.

So, LINQ is easier than you think because LINQ has two ways of querying data sources.

Query syntax or language level syntax — this often called “SQL-like syntax”, and is a declarative way to write LINQ queries.

List<int> numbers = new List<int> { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };

var evenNumbers = from num in numbers
                  where num % 2 == 0
                  select num;

foreach (var num in evenNumbers)
{
    Console.WriteLine(num);
}

Method syntax — also known as “fluent syntax” or “method chaining”. This syntax is often more concise and can be more powerful when dealing with complex queries.

List<int> numbers = new List<int> { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };

var evenNumbers = numbers.Where(num => num % 2 == 0);

foreach (var num in evenNumbers)
{
    Console.WriteLine(num);
}

The design of the LINQ library stands on iteration elements and execution of methods at different times.

Everything starts with IEnumerable and IEnumerator interfaces;

public interface IEnumerable
{
    IEnumerator GetEnumerator();
}

public interface IEnumerator
{
    object Current { get; }
    bool MoveNext();
    void Reset();
}

These interfaces lead to an iterator pattern which helps to encapsulate the iteration process for all data sources, specifically in LINQ to objects as known collections implement these interfaces. They also provide the contract for the LINQ base.

The LINQ methods are implemented as extension methods. Also, the trick is they are method-based extensible, and not coupled with each other. But everything starts with method syntax and their execution.

Let’s start with one of the popular methods of LINQ which is Where.

public static IEnumerable<TSource> Where<TSource>(this IEnumerable<TSource> source, Func<TSource, bool> predicate)
{
     if (source == null)
     {
        ThrowHelper.ThrowArgumentNullException(ExceptionArgument.source);
     }

     if (predicate == null)
     {
        ThrowHelper.ThrowArgumentNullException(ExceptionArgument.predicate);
     }

     if (source is Iterator<TSource> iterator)
     {
        return iterator.Where(predicate);
     }

     if (source is TSource[] array)
     {
        return array.Length == 0 ?
              Empty<TSource>() :
              new WhereArrayIterator<TSource>(array, predicate);
     }

     if (source is List<TSource> list)
     {
        return new WhereListIterator<TSource>(list, predicate);
     }

     return new WhereEnumerableIterator<TSource>(source, predicate);
}

As you can see the source code above shows the Where extension method with encapsulating Source and Predicate. As defined before, the method helps to filter with custom conditions. So, the method participates as a high-order function and has accepted the delegate for the hiding custom part.

We are closing the critical part, why does the method use the WhereEnumerableIteration class?

Because of the lazy loading, when executing the method chaining (shown below), it helps to avoid immediate loading for optimizing memory and performance. It’s executed in case of need. On the other hand, there are also some methods that execute immediately like ToList, ToArray, Single, etc.

Sample of lazy loading with fluent API.

List<string> strList = new List<string>()
{
    "Rasul",
    "Huseynov",
    "LINQ",
    "Q123"
};

strList.Where(x => x.Contains("Q"))
        .Select(x => x.Substring(0, 2))
        .OrderBy(x => x);

Note: Other LINQ providers work through the extended part of IQueryable also with different interfaces, which we will cover in another article.

Writing OWN Methods

In this section, we will extend the LINQ with our own method.

As I mentioned before, everything starts with IEnumerable, so in our case, we want to implement a general filter by choosing the first two elements by condition. It is a little bit easy but the idea is to understand how to extend LINQ by yourself. In our case, we will use method chaining and to avoid loading issues we need to implement our method using lazy loading.

Let’s get started!

Our custom extension is similar to “Where” but it has an item count rule. That means when iterating over data we will take the first two items that match the given condition.

public static class CustomEnumerableExtensions
{
    public static IEnumerable<TSource> FirstTwoItems<TSource>(this IEnumerable<TSource> source, Func<TSource, bool> predicate)
    {
        if (source == null)
            throw new ArgumentNullException(nameof(source));
        if (predicate == null)
            throw new ArgumentNullException(nameof(predicate));

        return new FirstTwoItemsEnumerable<TSource>(source, predicate);
    }

    private sealed class FirstTwoItemsEnumerable<TSource> : IEnumerable<TSource>, IEnumerator<TSource>
    {
        private int _itemCount;
        private int _state;
        private TSource _current;

        private readonly IEnumerable<TSource> _source;
        private readonly Func<TSource, bool> _predicate;
        private IEnumerator<TSource>? _enumerator;

        public FirstTwoItemsEnumerable(IEnumerable<TSource> source,
                                        Func<TSource, bool> predicate)
        {
            _source = source;
            _predicate = predicate;
        }

        public TSource Current
        {
            get { return _current; }
        }

        object IEnumerator.Current
        {
            get { return _current; }
        }

        public void Dispose()
        {
            if (_enumerator != null)
            {
                _enumerator.Dispose();
                _enumerator = null;
            }

            _current = default;
            _state = -1;
            _itemCount = 0;
        }

        public IEnumerator<TSource> GetEnumerator()
        {
            var instance = new FirstTwoItemsEnumerable<TSource>(this._source, this._predicate);
            instance._state = 1;
            return instance;
        }

        IEnumerator IEnumerable.GetEnumerator()
        {
            var instance = new FirstTwoItemsEnumerable<TSource>(this._source, this._predicate);
            instance._state = 1;
            return instance;
        }

        public bool MoveNext()
        {
            switch (_state)
            {
                case 1:
                    _enumerator = _source.GetEnumerator();
                    _state = 2;
                    goto case 2;
                case 2:
                    while (_enumerator.MoveNext())
                    {
                        if (_itemCount == 2)
                            break;

                        TSource item = _enumerator.Current;
                        if (_predicate(item))
                        {
                            _itemCount++;
                            _current = item;
                            return true;
                        }
                    }

                    Dispose();
                    break;
            }

            return false;
        }

        public void Reset()
        {
            if (_enumerator != null)
            {
                _enumerator.Dispose();
                _enumerator = null;
            }

            _current = default;
            _state = 1;
            _itemCount = 0;
        }

    }
}

So, for it to be lazy loading, there is a FirstTwoItemsEnumerable wrapper class for managing states.

The implementation of code stands on the default iteration pattern and behaves like other extensions. With this concept, you can write your own methods and use them in case of LINQ to objects is needed.

Sample of usage;

List<string> strList = new List<string>()
{
    "rasul",
    "huseynov",
    "turkey",
    "usa"
};

var result = strList.FirstTwoItems(x => x.Contains("u")).Select(x => x.ToUpper());

foreach (var item in result)
{
    Console.WriteLine(item);
}

Conclusion

LINQ (Language-Integrated Query) is a powerful library of the .NET framework that enables developers to query data in a more intuitive and readable way. By understanding how LINQ works under the hood and learning to write your own LINQ methods, you can harness its full potential to make your code cleaner, more efficient, and easier to maintain.

Through this explanation, we have seen how LINQ makes working with data easier and helps us write clear, simple code. By making your own LINQ methods, you can learn how LINQ works and can create solutions that fit your needs.

Stay tuned!

Behind the Scenes of OWIN (Open Web Interface for .NET)

Rasul — Wed, 26 Jun 2024 09:32:54 +0000

In this article, we will understand what OWIN is and its history of how it was created. The article will helps to who are wondering behind it.

And we will understand how a group of people brought valuable ideas to the .NET community. Also important is the fact that Microsoft has embraced OWIN and ASP.NET Core is basically built on this idea.

Let’s get started!

In simple term, OWIN is the acronym of Open Web Interface for .NET and it is a specification that provides to decouple web servers with web applications. This specification is an also open standard for all .NET ecosystems.

The officially OWIN is — “defines a standard interface between .NET web servers and web applications. The goal of the OWIN interface is to decouple server and application, encourage the development of simple modules for .NET web development, and, by being an open standard, stimulate the open source ecosystem of .NET web development tools.”

The word “specification” can confuse you but it means standardized contracts/interfaces which provide how communication between web servers and applications should be. So, these things are not concrete implementations rather it is telling developers how to communicate between web servers and web applications.

The Story

The story of OWIN began (2010) with a group of people who were inspired by other programming language libraries and they were trying to create a library that provides HTTP abstraction between web servers and web applications/frameworks.

One day a group of people who were working on their own framework/library, Ryan Riley (the creator of Kayak) sent an email to others who were working on its framework/library about sharing knowledge and working together (because they were working on the same and different but complementary things).

September 7 (2010), The first email sent by Ryan Riley is shown below;

Then the group started discussing it together because the message made sense. And they started to collaborate through the google group.

First of all, the group was meet for find out solution for the problems. The main problems were:

ASP.NET was coupled to IIS (System.Web.* packages)
ASP.NET was too heavy
IIS was to slow (old versions)
Hard to do REST (like minimal API, Sinatra -> DSL things)

In addition, the .NET community needed decoupled/independent, lightweight and efficient web frameworks that worked with different lightweight web servers.

So, the group people started to solve it with inspiration from Rack (Ruby) and WSGI (Python). In simple term, Rack and WSGI define a common interface between a web server and a web application.

September 27 (2010), the first draft came from Benjamin van der Veen;

As you can see the idea stands on Responder, Request and Response interfaces. (It provides certain abstraction from the server side and provides abstraction by Request and Response passed to the application side. The idea provides standardization with interfaces)

November 29 (2010), one of the people (Scott Koon) from the group created a working group under the name “.NET HTTP Abstractions” (check it out google group).

The group of people has come to a consensus about web server wrappers. At the same time, the Open Web Interface for .NET (OWIN) nomenclature was decided upon and the specification started to be written.

Also, at the beginning of the specification, the group of people declared goals;

No dependencies
Flexibility of style
Server independence

Well, during the writing of the specification, some problems were of course encountered (I will skip this part, if you want more details you can check out the google group and website links).

After some brainstorming and experimentation, the group of people found the following solutions, which shown below;

They were concentrated on “Delegate Of Doom” (informal term) which restricted pattern or technique that provides minimal dependency and contract (standardization) for communication between web server and application. The idea was to create a communication contract + communication pipeline.

And they had decided to create gate reference (entry point) which represent helper library for who wants to start using OWIN standard. And the library is open source in Github.

On July 25, 2011, the OWIN project was publicly announced, marking the beginning of a collaborative effort to create the specification.

And, finally, in December 2012 the OWIN project had been released its first official version 1.0.

OWIN

In this section, we will cover the general idea. The specification details will not be considered, click if you are interested in the more details.

The image shown above gives a bird’s eye view of OWIN. As I said before, OWIN is just provides decoupling communication between web server and web application. But take into account that OWIN is not framework, library or implementation. If you look at the OWIN specification you will get details of rules or samples of code on how web servers and applications communicate with each other.

To get into the details, it is necessary to start with the actors, which helps to understand the idea of OWIN.

Web Framework — A self-contained component on top of OWIN exposing its own object model or API that applications may use to facilitate request processing. Web Frameworks may require an adapter layer that converts from OWIN semantics.
Web Application — A specific application, possibly built on top of a Web Framework, which is run using OWIN compatible Servers.
Middleware — Pass through components that form a pipeline between a server and application to inspect, route, or modify request and response messages for a specific purpose.
Server — The HTTP server that directly communicates with the client and then uses OWIN semantics to process requests. Servers may require an adapter layer that converts to OWIN semantics.
Host — The process an application and server execute inside of, primarily responsible for application startup. Some Servers are also Hosts.

Mainly, the actors must-have for any web applications. So, OWIN dictate to all of them to achieve goals.

Note that for understandable explanation I will describe all processes with runtime things. Some parts of the explanation may be outside the scope of OWIN.

The communication between a web server and a web application (top of the web framework). For that reason, these actors also have a huge computational logic. The idea brought the pipeline for handling web requests and responses lifecycle. This is also relies on Pipeline architecture which provides to achieve cross-cutting concerns between web servers and web applications.

Everything starts with a host which creates the needed environment and configuration things. In runtime should consist of a built-in web server and application which is top of a framework. So, basically web server listens to HTTP requests populate things some data structure and send to the communication pipeline.

The idea of building a pipeline starts with middleware, which is a chain or filter for all its parts. The middleware provides some cross-cutting concerns and helps to handle requests and responses lifecycle. When requests are coming from the server send them to the first chain/filter (middleware) and then requests will continue one by one with each other. And end of the pipeline requests are handled with the application and generate the response, the same things happen with response in a back way.

So far we have explained the general idea, you can look at the specification for more details.

Solution of OWIN start with;

using Environment = IDictionary<string, object>
using AppFunc = Func<Environment, Task>

Hmm, it’s a little easier than you think!

Environment is the key-value pair that contains all needed data including request and response objects. So, it makes sense when the server and application communicate with each other!
AppFunc is the delegate that provides handling requests.

Basically, how AppFunc handle requests on an Application side? Code is shown below;

using System.IO;
using System.Text;
using Headers = IDictionary<string, string[]>;

var app = AppFunc(env => {
    var bytes = Encoding.UTF8.GetBytes("Hello, OWIN!");
    var length = bytes.Length.ToString();

    var headers = env.["owin.ResponseHeaders"];
    headers.Add("Content-Type", new[] { "text/plain" });
    headers.Add("Content-Length", new[] { length });

    var stream = (Stream)env.["owin.ResponseBody"];
    return stream.WriteAsync(bytes, 0, bytes.Length);
});

For building the pipeline we need to create the idea of Middlewares. It can be achieved with the Chain Of Responsibility pattern.

The code above AppFunc is middleware and it can encapsulate the next element of the chain (which is middleware). And it calls the next one, repeating in the same way as the others, like a nested chain.

So, it can be implemented as Class and Function. But I will just show the Class sample;

public class LoggingMiddleware{
    private AppFunc _next;

    public LoggingMiddleware(AppFunc next){
        this._next = next; 
    }

    public Task LogBefore(Environment env) { /**/ }
    public Task LogAfter(Environment env) { /**/ }
    public async Task Invoke(Environment env) {
        LogBefore(env);
        await _next(env)
        LogAfter(env);
    }
}

That was the middleware side.

Also has IAppBuilder interface for the pipeline side. Its main purpose is to provide the contract for configuring the middleware pipeline that handles HTTP requests and responses.

Originally intended to provide the delegate signatures
IAppBuilder was later added and left as the only interface

public interface IAppBuilder
{
    IDictionary<string, object> Properties { get; }
    IAppBuilder Use(object middleware, params object[] args);
    object Build(Type returnType);
    IAppBuilder New();
}

As you can see, the OWIN solution has minimal dependency by simply relying on the FCL. These basic things that help OWIN to achieve its goal.

P.S. Microsoft implemented OWIN idea, its Katana, but this is another topic.

Conclusion

OWIN is a specification that provides the rules for decoupling between web servers and web applications. Also, it helps to achieve modular architecture between web servers and applications. The idea behind it is used in many other web technology environments. OWIN is a powerful idea that helped to .NET community by providing an alternative to traditional IIS and ASP.NET (old) hosting models, addressing their limitations.

Stay tuned!

Understanding of Load Balancers

Rasul — Wed, 19 Jun 2024 11:54:11 +0000

Modern systems are becoming increasingly complex, but this complexity ensures that a huge amount of network traffic and requests are handled. As a result of this Load balancers bring complexity but solve the important problems.

So, in simple terms, the main idea behind Load Balancers, as the name suggests, is providing direct client requests across servers. In other words, Load Balancers are systems or devices that distribute network or application traffic across multiple servers.

There are two fundamental strategies used to handle increased load and improve performance in computing environments, particularly when it comes to servers and applications.

Scaling-out also known as horizontal scaling
Scaling-up also known as vertical scaling

Modern distributed systems have a scaling-out strategy that dictates to increase nodes/units. This means you need to create or copy the entire node; not part of the node/unit. For example, in our case, we have 1 server and it handles 10000 requests per minute but if we want to increase requests count? what should we do? In simple terms, we will increase part of the server (RAM, CPU, etc..). So, this means scaling-up! But the scaling-out means we will not increase part of the server. Instead, we will add new server (or node/unit).

So, in case of scaling-out, the load balancers distribute traffic among the same servers using specified algorithms/methods.

Basically How do Load Balancers Work?

Let’s assume we have three copies of servers that perform the same tasks. When a client makes a request to the server (such as sending data over the network), the load balancer is the first point of contact. The load balancer sits in front of the servers and handles the request. It then chooses one of the servers based on their current state and forwards the request to that server.

When a client sends a request to access a service, like a webpage or an application, this request is directed to the load balancer, not directly to any of the servers. The load balancer, positioned in front of the servers, intercepts the incoming request and acts as the traffic manager, determining the best server to handle the request.

The load balancer checks the current state of each server, including the number of active connections on each server, the current load or traffic each server is handling, and the overall performance and health status of each server. Based on this evaluation, the load balancer selects the most suitable server. For instance, if Server A has fewer active connections than Servers B and C, the load balancer will choose Server A.

The load balancer then forwards the client’s request to the chosen server. The server processes the request and prepares the response. The server sends the response back to the load balancer, which then forwards this response to the client. The client perceives the interaction as if it is directly communicating with the server.

By distributing client requests among multiple servers, the load balancer ensures that no single server becomes overwhelmed. This helps in maintaining high performance, availability, and reliability of the service.

What Do They Provide?

High Availability: Load balancers make sure that if one server goes down, another server can take over. This means the service stays up and running, even if there are problems with one of the servers.
Scaling: When more people use the service, load balancers help by spreading the traffic across multiple servers. This way, the service can handle more users without slowing down or crashing.
Reliability: Load balancers check the health of servers and only send requests to servers that are working well. This ensures that users get a smooth and dependable experience every time they use the service.
Security: Load balancers can help protect against attacks by distributing traffic and hiding the actual servers from direct access. This makes it harder for attackers to target a specific server.

Types of Load Balancers

There are different types of load balancers that help manage traffic to servers.

Hardware Load Balancers: These are physical devices. They sit in your server room and handle the traffic. They are fast and powerful but can be expensive.
Software Load Balancers: These are programs that run on regular servers. They do the same job as hardware load balancers but are usually cheaper and easier to update.
Layer 4 Load Balancers (Transport Layer): These load balancers work at the transport layer of the network. They make decisions based on data like IP addresses and port numbers. They are fast and simple.
Layer 7 Load Balancers (Application Layer): These load balancers work at the application layer. They make decisions based on the content of the request, like URLs or cookies. They are more flexible and can handle more complex tasks.

In simple terms, hardware load balancers are physical devices, while software load balancers are programs. Layer 4 load balancers work with basic data like IP addresses, and Layer 7 load balancers work with more detailed data like URLs.

Types of Load Balancing Algorithms/Methods

Load balancers use different algorithms/methods to share traffic among servers. Here are some common ones.

Round Robin: This method sends traffic to servers one by one in a line. Each server gets a turn, so the workload is evenly spread.
Least Connections: With this method, traffic goes to the server that has the fewest people using it at the moment. It helps balance the load by sending more traffic to less busy servers.
IP Hash: Traffic is sent to servers based on the client’s IP address. This way, the same client always goes to the same server, which can be helpful for some applications.
Others: There are also algorithms/methods like weighted round robin, where some servers get more traffic based on their strength. Another method is random with two choices, which randomly picks between two servers to balance the load.

Load balancers use these algorithms/methods to make sure servers stay balanced and everyone gets a good experience using the service.

Stay tuned!

A Different Perspective of Stack Memory in .NET

Rasul — Sun, 19 May 2024 11:44:08 +0000

In this article, we will talk about the Stack memory in .NET.

The Stack is the part of the memory in .NET that belongs to a thread. In a simple term, each thread has its own memory that acts as a stack (last in first out).

Note: The article does not cover Heap memory, some parts are explained, but this is because there is a connection between Stack memory.

Stack Memory

The concept of a memory stack originates from computer memory architecture. And its nature touches the many programming languages. After all, a programming language is some kind of abstraction (or way of communication) on any computer machine. For this reason, many programming languages must use it.

So, even the CPU has push and pop instructions for providing thread (stack) execution and other things.

Well, without further confusion, let’s continue our look at .NET and their use of Stack memory.

Almost every developer knows that the name “stack” refers to a data structure. In general, the stack data structure has two techniques, one is LIFO and the other is FIFO. Our stack memory works using the LIFO (last in first out) technique. Also, do not confuse the stack data structure and stack memory. Names are the same because stack memory uses the stack data structure technique.

In .NET, there are two types of memory, one of them is a Heap and another one is a Stack and they are both located in RAM. The stack keeps track of method calls, return addresses, arguments/parameters and local variables. But Heap stores reference-type objects (this is a topic for another article).

In simple terms, each thread has its own storage in memory. This is the stack, which is the local memory for each thread. So, when threads execute methods, they declare some local variable, get the result and call another method, etc. It is for these actions that stack memory is needed.

Also, note that Stack memory is used only by one thread of execution. On the other hand, Heap is shared memory between threads (the scope of which is declared by the OS process).

Stack Allocation

In simple term, Stack allocation refers to the method of memory allocation that uses a region of memory known as the stack. As I mentioned before, the stack operates on a LIFO (last in first out) technique, meaning the most recently allocated memory is the first to be deallocated.

In .NET has two types. One is a value type and the other is a reference type. So, when allocating a value type in a stack, it saves the value of the type directly. But in the case of reference type which is located in heap, it saves the reference address (or pointer to the object) of the heap. In general, it means stack holds just values (reference address is some kind of value, but when needs to reach to it needs to indirect access, this is a topic for another article).

As you can see in the image above, the thread stack has 2 declared variables in a different stack frame: one is a pInst, instance of Person class and the other is a flag (int32, value type). We will not discuss the heap, but the image shows that the stack stores the reference address of pInst and the value of the flag (which is 1). In the case of the reference type, the address is the value for the stack that references the heap object (object with data and methods located in the heap).

In .NET, Stack memory has fixed size which a 1-MB stack is allocated when a thread is created. This size is predefined once the program starts, providing a consistent amount of stack memory available for function calls and local variable storage. However, the fixed nature of the stack means that it can only hold a limited amount of data. In addition, the speed of stack memory operations is a key advantage based on the simplicity of the stack’s push and pop mechanisms.

So, almost everyone wonders why the Stack memory is fast?

Firstly, stack memory uses static memory allocation, meaning that memory is given and taken away in a predictable way. This removes the need for complicated memory management, making operations quick and efficient.
Secondly, as I said above the stack has a fixed size, which avoids the overhead of finding available memory space, unlike heap memory. The simple push and pop operations also make it faster. These operations just adjust the stack pointer, which is very fast compared to the more complex operations needed for heap memory.
Thirdly, modern CPUs have special instructions for stack operations, like pushing and popping data, which are made to be very fast. The CPU also uses a dedicated register, the stack pointer, to efficiently manage the stack’s top position. In additional, stack memory often gets loaded into the CPU cache. This happens because stack data is frequently accessed in a straight line, making it more friendly to the cache. In contrast, accessing heap memory is indirect and can involve more cache misses, making it slower. This direct hardware support and cache efficiency ensure that stack operations have very little overhead.

Stack Frame/Call Stack

A stack frame, also known as an activation record, is a section of the stack memory allocated for a single function/method call. When a function/method is called, a new stack frame is allocated, and when the function/method returns, its stack frame is deallocated.

Basically, a stack frame contains:

Function/Method Arguments: The values passed to the function/method.
Local Variables: Variables that are declared inside the function/method.
Return Address: The location in the code to return to after the function/method completes.

Let’s start the explanation with a code example:

public void Calculate()
{
    int a = 1;
    int b = 10;

    int c = a + b;

    Write(c);
    Console.WriteLine("Finished");
}

public void Write(int number)
{
    Console.WriteLine(number);
}

As you can see, there is a method named Calculate that declared 3 variables and also called the Write method in it.

Assume that, a thread calls the Calculate method, how would this be visualized?

As you can see from the image above, the first stack frame belongs to the Calculate method which stores 3 variables, one of which is the result of the sum of two variables. The second stack frame belongs to the Write method which stores the argument and return address (line 9 just for understanding, in fact, it contains different value). In addition, as you can see every call creates a new stack frame and each stack frame provides isolation. Therefore, it achieves local and temporary memory for each function/method.

After the Write method completes the execution, the second stack frame is deallocated and the thread returns to the first stack frame that belongs to the Calculate method, through the return address it will execute from that address line and complete the execution of the Calculate method. In this way, the call stack loop that was called in this thread will continue to execute (in our case, execution is completed).

StackOverflowException

As I mentioned before, the stack size is limited and for this reason, when any function/method is called in an unlimited and nested way, a “StackOverflowException” occurs.

Assume that we have some function/method that is using recursion but there is not any limitation. It means the method has some calculation in which there is no return state. So, in this case, we will get “StackOverFlowException”.

Let’s take a look at a code example:

public class Program
{
    public static void Main()
    {
        Calculate(2, 3);
    }

    public static void Calculate(int a, int b)
    {
        int c = a + b;
        Calculate(c, c * c);
    }
}

When you execute this code, you will get “StackOverFlowException” because recursion provides the call itself and there is no return state or limit in this function/method. With this technique, every call to the function/method creates new stack frames and when the stack limit is exceeded, we get a “StackOverFlowException”.

If you would like to learn more, stay tuned!

Under The Hood Of Thread Synchronization With LOCK

Rasul — Wed, 15 May 2024 10:21:21 +0000

In the previous article, we talked about “Introduction to Monitor Class In C#”. In this article, we will discuss the LOCK statement.

The LOCK statement is the most popular mutual-exclusive thread synchronization construct. You will be surprised when we deep dive into details.

LOCK Statement

Basically, LOCK is a reserved keyword that is recognized by the C# compiler. It provides thread synchronization for a shared resource in case of execution in the critical section area and helps to avoid problems related to race conditions.

In order to use LOCK, you need to define a critical section area and wrap a LOCK around it. This will prevent threads from accessing shared resources at the same time.

How to use the LOCK statement is shown in the code below.

public class Program
{
    private static readonly object _LOCKREF = new object();

    // A basic field for the simulation of shared data
    static int _sharedField = 0;

    public static void Main()
    {
        for (int i = 0; i < 10; i++) {
            ThreadPool.QueueUserWorkItem((s) => IncrementTheValue());
        }

        Console.Read();
    }

    static void IncrementTheValue()
    {
        // Critical section start point
        lock (_LOCKREF)
        {
            if (_sharedField < 5)
            {
                Thread.Sleep(55); // just for simulation execution time!
                _sharedField += 1;
                Console.WriteLine($"Field: {_sharedField} | Thread: {Thread.CurrentThread.ManagedThreadId}");
            } 
            else
            {
                Console.WriteLine($"Field count is complited {_sharedField} | Thread Id is {Thread.CurrentThread.ManagedThreadId}");
            }
        }
       // Critical section end point
    }
}

As you can see, the above code prevents the race condition problem by wrapping the LOCK statement around the critical section area. Also, it is best to use a private reference type to provide a lock state (_LOCKREF). So do not use public reference instance, the ‘this’ keyword or interned string type; They may cause a deadlock.

Let’s understand how it works basically; Threads enter one by one into the code block, i.e. it accepts the thread into the critical section and executes the block with a single thread, then releases the thread. This means that all the other threads must wait and halt the execution until the locked section is released. And this cycle will continue in the same way.

Under The Hood

In fact, the LOCK statement is syntactic sugar. This means that when the C# compiler compiles code into the IL language, some keywords are converted into more complex code. In this way, it provides efficiency (providing ease of use).

In the code below you can see our IL (Microsoft Intermediate Language) code. It is very interesting that LOCK is converted to a Monitor class!

.method private hidebysig static void  IncrementTheValue() cil managed
{
  // Code size       206 (0xce)
  .maxstack  2
  .locals init (object V_0,
           bool V_1,
           valuetype [System.Runtime]System.Runtime.CompilerServices.DefaultInterpolatedStringHandler V_2)
  IL_0000:  ldsfld     object Program::_LOCKREF
  IL_0005:  stloc.0
  IL_0006:  ldc.i4.0
  IL_0007:  stloc.1
  .try
  {
    IL_0008:  ldloc.0
    IL_0009:  ldloca.s   V_1
    IL_000b:  call       void [System.Threading]System.Threading.Monitor::Enter(object,
                                                                                bool&)
    IL_0010:  ldsfld     int32 Program::_sharedField
    IL_0015:  ldc.i4.5
    IL_0016:  bge.s      IL_0077
    IL_0018:  ldc.i4.s   55
    IL_001a:  call       void [System.Threading.Thread]System.Threading.Thread::Sleep(int32)
    IL_001f:  ldsfld     int32 Program::_sharedField
    IL_0024:  ldc.i4.1
    IL_0025:  add
    IL_0026:  stsfld     int32 Program::_sharedField
    IL_002b:  ldc.i4.s   18
    IL_002d:  ldc.i4.2
    IL_002e:  newobj     instance void [System.Runtime]System.Runtime.CompilerServices.DefaultInterpolatedStringHandler::.ctor(int32,
                                                                                                                               int32)
    IL_0033:  stloc.2
    IL_0034:  ldloca.s   V_2
    IL_0036:  ldstr      "Field: "
    IL_003b:  call       instance void [System.Runtime]System.Runtime.CompilerServices.DefaultInterpolatedStringHandler::AppendLiteral(string)
    IL_0040:  ldloca.s   V_2
    IL_0042:  ldsfld     int32 Program::_sharedField
    IL_0047:  call       instance void [System.Runtime]System.Runtime.CompilerServices.DefaultInterpolatedStringHandler::AppendFormatted<int32>(!!0)
    IL_004c:  ldloca.s   V_2
    IL_004e:  ldstr      " | Thread: "
    IL_0053:  call       instance void [System.Runtime]System.Runtime.CompilerServices.DefaultInterpolatedStringHandler::AppendLiteral(string)
    IL_0058:  ldloca.s   V_2
    IL_005a:  call       class [System.Threading.Thread]System.Threading.Thread [System.Threading.Thread]System.Threading.Thread::get_CurrentThread()
    IL_005f:  callvirt   instance int32 [System.Threading.Thread]System.Threading.Thread::get_ManagedThreadId()
    IL_0064:  call       instance void [System.Runtime]System.Runtime.CompilerServices.DefaultInterpolatedStringHandler::AppendFormatted<int32>(!!0)
    IL_0069:  ldloca.s   V_2
    IL_006b:  call       instance string [System.Runtime]System.Runtime.CompilerServices.DefaultInterpolatedStringHandler::ToStringAndClear()
    IL_0070:  call       void [System.Console]System.Console::WriteLine(string)
    IL_0075:  leave.s    IL_00cd
    IL_0077:  ldc.i4.s   41
    IL_0079:  ldc.i4.2
    IL_007a:  newobj     instance void [System.Runtime]System.Runtime.CompilerServices.DefaultInterpolatedStringHandler::.ctor(int32,
                                                                                                                               int32)
    IL_007f:  stloc.2
    IL_0080:  ldloca.s   V_2
    IL_0082:  ldstr      "Field count is complited "
    IL_0087:  call       instance void [System.Runtime]System.Runtime.CompilerServices.DefaultInterpolatedStringHandler::AppendLiteral(string)
    IL_008c:  ldloca.s   V_2
    IL_008e:  ldsfld     int32 Program::_sharedField
    IL_0093:  call       instance void [System.Runtime]System.Runtime.CompilerServices.DefaultInterpolatedStringHandler::AppendFormatted<int32>(!!0)
    IL_0098:  ldloca.s   V_2
    IL_009a:  ldstr      " | Thread Id is "
    IL_009f:  call       instance void [System.Runtime]System.Runtime.CompilerServices.DefaultInterpolatedStringHandler::AppendLiteral(string)
    IL_00a4:  ldloca.s   V_2
    IL_00a6:  call       class [System.Threading.Thread]System.Threading.Thread [System.Threading.Thread]System.Threading.Thread::get_CurrentThread()
    IL_00ab:  callvirt   instance int32 [System.Threading.Thread]System.Threading.Thread::get_ManagedThreadId()
    IL_00b0:  call       instance void [System.Runtime]System.Runtime.CompilerServices.DefaultInterpolatedStringHandler::AppendFormatted<int32>(!!0)
    IL_00b5:  ldloca.s   V_2
    IL_00b7:  call       instance string [System.Runtime]System.Runtime.CompilerServices.DefaultInterpolatedStringHandler::ToStringAndClear()
    IL_00bc:  call       void [System.Console]System.Console::WriteLine(string)
    IL_00c1:  leave.s    IL_00cd
  }  // end .try
  finally
  {
    IL_00c3:  ldloc.1
    IL_00c4:  brfalse.s  IL_00cc
    IL_00c6:  ldloc.0
    IL_00c7:  call       void [System.Threading]System.Threading.Monitor::Exit(object)
    IL_00cc:  endfinally
  }  // end handler
  IL_00cd:  ret
} // end of method Program::IncrementTheValue

If the above IL code is not clear, you can check line 16, which declares System.Threading.Monitor::Enter(object, bool&). As you can see the LOCK statement is converting to Monitor class. Also the line 77 declares System.Threading.Monitor::Exit(object) which provides release lock state.

So, let’s look at the above IL code as C# code. The code declared below is similar to the code above (our previous method is wrapped by the Monitor class):

static void IncrementTheValue()
{
    bool lockTaken = false;

    try
    {
        Monitor.Enter(_LOCKREF, ref lockTaken);

        // Critical section start point
        if (_sharedField < 5)
        {
            Thread.Sleep(55); // just for simulation execution time!
            _sharedField += 1;
            Console.WriteLine($"Field: {_sharedField} | Thread: {Thread.CurrentThread.ManagedThreadId}");
        } 
        else
        {
            Console.WriteLine($"Field count is complited {_sharedField} | Thread Id is {Thread.CurrentThread.ManagedThreadId}");
        }// Critical section end point
    }
    finally
    {
        if (lockTaken)
        {
            Monitor.Exit(_LOCKREF);
        }       
    }     
}

As you can see, try + finally blocks and Monitor class are used. Also, the LockTaken pattern is used to release only the locked ones (for more details).

Conclusion

The LOCK statement is a mutual-exclusive thread synchronization construct (because it uses the Monitor class). Also, this is syntactic sugar in C# that improves usability.

If you want to learn more about Monitor class, you can check it out.

Stay Tuned!

Deep Dive Into Monitor Class as Hybrid Synchronization Construct in .NET

Rasul — Tue, 14 May 2024 07:13:03 +0000

In the previous article, we talked about “Introduction to Monitor Class In C#”. So, in this article, we will take a detailed look at the Monitor class.

The Monitor class is a hybrid thread synchronization construct. It provides a mutual exclusive lock with supporting spinning, thread ownership, and recursion.

Hybrid Synchronization Constructs

Hybrid Synchronization Constructs are the combining the user-mode synchronization constructs and kernel-mode thread synchronization constructs.

The user-mode synchronization constructs are application level and faster than the kernel-mode synchronization constructs. Because it uses special hardware-level instructions; which it makes faster.

The user-mode synchronization constructs are:

Volatile
Interlocked

Using user-mode constructs has some disadvantages:

The thread wants to execute some code block, but another thread is currently running, then it is put on hold, but this hold is simulated by spins on the CPU, which wastes CPU time.
The threads are blocked through spins. For that reason, the Windows operating system never detects in case of it. So, in the case of a livelock (the thread is running on a CPU forever) situation will waste both CPU time and memory (the worst one).

The kernel-mode synchronization constructs are provided by the Windows operating system. The threads call functionalities are implemented at the level of the operating system kernel. But the advantage of this mode is when a thread wants to acquire a resource that another thread has, Windows blocks the thread, and when a resource becomes available, resumes the thread again, so the Windows throughs that achieve not wasting the CPU time.

Also, in case a thread holding the construct and another one is waiting for the construct, it might be blocked forever. Well, it is called deadlock, which wastes only memory (better than livelock because livelock wastes both CPU and memory).

The kernel-mode synchronization constructs are:

AutoResetEvent
ManualResetEvent
Semaphore
Mutex

As a result of these, preferable to use fast and non-blocking constructs. This is where Hybrid Synchronization Constructs come in!

User-mode synchronization constructs are fast and non-blocking. It is very effective, but in the case of racing/competition (needs blocking + efficiency) would be better to use Kernel-mode synchronization constructs. Also, note that the Kernel mode can block (stop) the threads, but the User-mode can only spin in case of the blocking (for that reason, they are non-blocking, on the other hand they help us to reuse threads).

Hybrid synchronization constructs combine the user-mode and kernel-mode constructs and provide more effective constructs.

How Monitor Class Works Internally?

As we see from the image above, the image contains the Managed heap and CLR’s Array of Sync Blocks. Well, when CLR allocates the object instance in a heap, it has some fields. One of them is type object pointer and another one is sync block index. The sync block index belongs to thread synchronization, and it contains an integer index in the array of sync blocks. So, CLR also, for efficiency, creates the array of sync blocks (in a native heap) that provides association with type through the index number.

Let’s think that we constructed an object (initialized). So, when the Object is constructed its sync block index assigns -1 by default. Which means that there is no reference to any sync block. Well, let’s assume that we want to use that object as an input parameter in Monitor.Enter method. So, in this case, CLR will find the free index in CLR’s array of sync blocks, occupy it, and then set the index into the object sync block (it would be on the fly reference).

In the case of the Monitor.Exit method, the used object sync block index will assign -1. And CLR’s array of sync blocks will be free. As we see the sync block index looks like a conditional flag (indicates the locked or not locked).

There are Some Problems With the Monitor Class

When using a monitor, there are some significant problems. So, be careful when using the Monitor class.

If Monitor.Enter input parameter is derived from the System.MarshalByRefObject class, it means there is a proxy object between the client (who working with Monitor class) and actual object. So, locking will be applied to the proxy one.
Assume that we have 2 strings with the same value (as you know strings can be interned) and belong to separate pieces of code. So, when we pass a one of string as an input parameter to Monitor.Enter method both strings will synchronize their execution with each other unknowingly (the same situation will be a problem for the “this” keyword). In the case of both using the thread synchronization construct, a deadlock may occur.
If we pass a value type as an input parameter to Monitor.Enter method, causes the value type to get boxed, resulting in the thread taking a lock on the boxed object. Each time Monitor.Enter is called, a lock is taken on a completely different object and you get no thread synchronization at all.

If you want to learn more, you can check out the book "CLR via C# by Jeffrey Richter".

Stay tuned!

Introduction to Monitor Class in C#

Rasul — Mon, 13 May 2024 10:15:07 +0000

In the previous article, we have talked about race condition problem. So, in this article we will focus on one of the solutions (using critical section), the Monitor class, which is a mutual-exclusive thread synchronization construct.

The Monitor class is built on dotNET’s FCL (Framework Class Library) infrastructure. In general, it provides to achieve thread safety.

Thread Synchronization In Critical Section

Thread synchronization is used to prevent corruption when multiple threads access shared data at the same time; Because thread synchronization is all about timing.

The System.Threading.Monitor class basically helps us to avoid the race condition problem by protecting the critical section.

Let’s think that we have a code block that has shared resource (variable or any state) and this resource is using in executed context by multithread. So, in this case, we need to understand that the potential race condition problem exists for us. And also that code block is a critical section for us.

The Monitor class provides threads to enter one by one into the code block, i.e. it accepts the thread into the critical section and executes the block with a single thread, then releases the thread. This means that all the other threads must wait and halt the execution until the locked section is released. And this cycle will continue in the same way.

In other words, it helps us to synchronize the execution of threads between each other.

How to Use The Monitor Class?

Let’s simulate the race condition problem and then prevent to it with Monitor Class.

public class Program
{
    // A basic field for the simulation of shared data
    static int _sharedField = 0;

    public static void Main()
    {
        for (int i = 0; i < 10; i++)
        {
            ThreadPool.QueueUserWorkItem((s) => IncrementTheValue());
        }

        Console.WriteLine("Main method is finished!");
        Console.Read();
    }

    // Method variable will give inconsistent result
    static void IncrementTheValue()
    {
        if (_sharedField < 5)
        {
            Thread.Sleep(55); // just for simulation execution time!
            _sharedField += 1;
            Console.WriteLine($"Field: {_sharedField} | Thread: {Thread.CurrentThread.ManagedThreadId}");
        }
        else
        {
            Console.WriteLine($"Field count is complited {_sharedField} | Thread Id is {Thread.CurrentThread.ManagedThreadId}");
        }
    }
}

The code above shows that we have a main method that creates 10 threads and they independently executes the IncrementTheValue method. And the method steps are:

if _sharedField is less than 5 then enter the code block
Execution time simulation with Thread.Sleep(55) (fake operation)
Increment the value of _sharedField
Write into the console _sharedField value and current thread id

Well, if you run the code, you will see that the result of the _sharedField variable is above 5. So, we are caught in a race condition problem.

Hmm, let’s avoid this problem. First of all, we need to identify our critical section area, in our case, it is the IncrementTheValue method completely. Because the method uses IF and ELSE blocks. So, we need to guarantee that our IF block will not participate in the case of racing threads. Otherwise, the race condition problem will continue.

public class Program
{
    // Should be reference type
    // Should be private
    static readonly object _LOCKREF = new object();

    // A basic field for the simulation of shared data
    static int _sharedField = 0; 

    public static void Main()
    {
        for (int i = 0; i < 10; i++)
        {
            ThreadPool.QueueUserWorkItem((s) => IncrementTheValue());
        }

        Console.WriteLine("Main method is finished!");
        Console.Read();
    }

    static void IncrementTheValue()
    {
        try
        {
            Monitor.Enter(_LOCKREF);

            // Critical section start point
            if (_sharedField < 5)
            {
                Thread.Sleep(55); // just for simulation execution time!
                _sharedField += 1;
                Console.WriteLine($"Field: {_sharedField} | Thread: {Thread.CurrentThread.ManagedThreadId}");
            }// Critical section end point
            else
            {
                Console.WriteLine($"Field count is complited {_sharedField} | Thread Id is {Thread.CurrentThread.ManagedThreadId}");
            }
        }
        finally
        {
            // Ensure that the lock is released
            Monitor.Exit(_LOCKREF);
        }
    }
}

In the code above, we see that Monitor.Enter method is declared and this method provides a wait-based (the waiting is implemented effectively, because avoids the CPU wasting) synchronization mechanism that allows only one thread to access the critical section code.

For the case of exceptions, we are using try + finally blocks. Because the Monitor class must release the lock at the end of any situation. So, we see that in the finally block has been declared Monitor.Exit method which helps us to release the lock from critical section.

As a result, with the code above, we have solved our race condition problem through the Monitor class. And now our code is thread-safe!

Input Parameter of Monitor.Enter Method

In the code above, where we avoid the race condition problem, there is an instance of the Object class its _LOCKREF which taking input into the Monitor.Enter method.

In our example code, the _LOCKREF instance helps us lock the thread. That is, it provides a reference to distinguish the context. So, you need to be careful when using Monitor.Enter method, that the reference to be used as an input parameter is closed outside the class.

So, we can understand that the input parameter of Monitor.Enter is some flag for understanding the locking state (in a simple term).

Note that if the _LOCKREF instance is manipulated or opened outside, it may cause a deadlock. Also, do not use the “this” keyword instead of _LOCKREF as it may cause deadlock, too.

LockTaken Pattern

As we can see in our code example, there is a try + finally block. So, in a multithreaded scenario, what happens if one of the threads enters the code block and an exception is thrown (or the thread is aborted) in the first step where it enters ‘try’ before reaching the Monitor.Enter method?

Hmm, I think almost everyone realized that the finally block will work in any case. So, if the thread without acquiring the lock tries to release the lock; it will receive System.Threading.SynchronizationLockException exception. Well, to avoid any case of this situation we need to use the LockTaken Pattern.

static void IncrementTheValue()
{
    bool lockTaken = false;

    try
    {
        Monitor.Enter(_LOCKREF, ref lockTaken);

        // Critical section start point
        if (_sharedField < 5)
        {
            Thread.Sleep(55); // just for simulation execution time!
            _sharedField += 1;
            Console.WriteLine($"Field: {_sharedField} | Thread: {Thread.CurrentThread.ManagedThreadId}");
        }
        else
        {
            Console.WriteLine($"Field count is complited {_sharedField} | Thread Id is {Thread.CurrentThread.ManagedThreadId}");
        }
        // Critical section end point
    }
    finally
    {         
        if (lockTaken)
        {
            // Ensure that the lock is released
            Monitor.Exit(_LOCKREF);
            Console.WriteLine($"Thread Id is {Thread.CurrentThread.ManagedThreadId} released from the lock!");
        }
    }
}

To implement this pattern, the Monitor.Enter method has an overload, it accepts 2 parameters. One of them is the lock reference instance, the other accepts a boolean variable.

As we see that, in the first line of the code above, there is a declared boolean variable lockTaken.

Monitor.Enter accepts the lockTaken variable with ref keyword. This means that if the method is true, the lock is taken and if false, the lock is not taken. So, finally block handles by checking lockTaken variable and in case of true if block executes otherwise it does not.

The lockTaken pattern provides safety in case any thread is aborted without acquiring a lock.

Methods of The Monitor Class

Only the most commonly used methods are described. For more information Microsoft Documentation.

Enter(Object): Acquires an exclusive lock on the specified object.
Enter(Object, Boolean): Acquires an exclusive lock on the specified object, and atomically sets a value that indicates whether the lock was taken.
Exit(Object): Releases an exclusive lock on the specified object.
IsEntered(Object): Determines whether the current thread holds the lock on the specified object.
TryEnter(Object): Attempts to acquire an exclusive lock on the specified object.

In the next article, I will explain the Monitor class in detail. Stay tuned!