The Request Pipeline in Asp.Net Explained: From a request, to the response

Hello there!

Quick question:

🤔 Do you know how an Asp.Net web app processes requests?

Well, that's what we'll be talking about today, so get yourself a drink or something, because we're diving into an exciting and a seemingly complicated topic in the world of web development using C# and Asp.Net, that is Middleware Components that make up the so called Request Pipeline, but I promise you, I'll make it very easy for you to understand, so stick around till the end, and do the practice tasks at the end of the post for hands-on training on creating custom middleware components.

Introduction 🧠

In a web app, we have an essential concept and that is HTTP requests, they run the web after all, and if you're building a web app, then you need to know how to process those HTTP requests, well, you're not going to process them by hand, but you'll have the Request Pipeline handle that for you, so what's the request pipeline after all?

This thing, is the orchestra that governs and controls, how incoming HTTP requests are handled in a proper way, so that you get a correct output and that is the response, like if you're getting an incorrect request, let's assume there's something wrong with it, we don't want that request to proceed any further, just get rid of it, now the pipeline isn't only responsible for keeping broken requests away, it's also controlling the modification of the request so that it generates a response, that suits your application's needs.

Now this all is simplified and just theory, but understanding the idea behind it is significantly important especially for those who are new to the framework. So for that, we'll use a simple analogy, please stick around till the end

The Workers On The Order Line 📦

The name of the analogy is weird, I just made it up, but believe me, it works.

Say we have a belt, with workers standing one after the other in order, a box gets loaded onto the line, and each worker now has to do their part, until the box reaches the end of the belt, and it's ready to be shipped, now imagine this:

What if the box which is made out of cardboard, has a tear, or it's overall broken and unusable?

The guy standing first will look at that box and be like, well, the box is broken, we can't use it, so instead of handing the box to the next worker in line, they just throw the box away, and grab a new one. Now if the box is fine, the first guy will pass it to the next person, they'll check/add or remove something from the box, pass it to the next one, and keep doing that until the box has passed all the checks and modifications and is ready to go.

The previous image, is a screenshot from an out of the box Model-View-Controller Web App Template, this code comes by default when you create a new MVC app since .NET 6, prior to it, there was a Program.cs class and a Startup.cs class, the request pipeline was then configured inside Startup.cs, but since .NET 6, the whole thing happens inside Program.cs only.

The Actual Request Pipeline 🗂️

In the previous analogy, we had a box, a belt on which the box moves, and the workers who would do some work on the box. Now let's replace the analogous objects, with the real concepts.

• The Belt = The Request Pipeline

• The Box = Incoming HTTP Request

• The Worker = A Middleware Component

When an HTTP request is made in the app, it goes through the request pipeline, the pipeline defines a certain order for the middleware components (The ordered workers in the analogy), and the request goes over from one middleware to the other, and similar to the analogy, if a middleware finds something wrong with the request, it could simply terminate the process and not allow the request to move further down the pipeline, and for that reason, the order in which the middleware components appear in is crucial.

What Do All These Middleware Components Do?

It all starts inside if(!app.Environment.IsDevelopment()), this check sees if we're running the app in an environment outside of the development environment, if that's true, we want to use a more user friendly page to display the errors to the user, so less technical exceptions are shown, because it's tough to assume that users will like to see this:

💀💀

In the following code block, we're registering two middleware components, only if we're not in development environment:

if (!app.Environment.IsDevelopment())
{
    app.UseExceptionHandler("/Home/Error");
    app.UseHsts();
}

The first taking care of routing the users to a more user-friendly page to tell them that something has gone wrong.

The second, app.UseHsts() ensures that the HTTP Strict Transport Security (HSTS) header is added which instructs the browser to use HTTPS for subsequent requests so that the requests are secured in an attempt to prevent cyber threats, you can read more about this here:
https://learn.microsoft.com/en-us/aspnet/core/security/enforcing-ssl?view=aspnetcore-7.0&tabs=visual-studio%2Clinux-ubuntu#http-strict-transport-security-protocol-hsts

Now I'll quickly describe what each middleware that appeared in the picture does, and after that, we'll see how we can create our own custom middleware, yup, that's right, that's going to happen here shortly 🥸

• app.UseHttpsRedirection(): This middleware automatically takes care of taking an incoming HTTP request, and redirecting it to an HTTPS request, so that the ongoing communication between the client and the server is encrypted and secure.

💡 Tip You can see this middleware in action by creating an MVC project, running it, and grabbing the HTTP port from the properties.json file, pasting it in the URL bar, if you do that, you'll notice that the app has flipped back to HTTPS, that's because the previous middleware took care of the redirection.

• app.UseStaticFiles(): This bad boy is responsible for serving the static files to your project, like the HTML, Css, and JavaScript files inside of wwwroot, the folder in which resides static files such as the files mentioned above, images, sample data files etc...

• app.UseRouting(): This dude here, takes care of setting up the routing for your application, and routing if you don't know, is the process of matching incoming requests to the appropriate action methods or (endpoints if it's an API), in the controllers.

• app.UseAuthorization(): Simply, it checks if the user is in fact authorized and authenticated properly, if so, it passes the request to the rest of the components, otherwise, it returns an HTTP 401 Unauthorized Status Code which indicates that the user is, basically not authorized to access a particular resource.

• Lastly, this guy right here:
  

app.MapControllerRoute(
    name: "default",
    pattern: "{controller=Home}/{action=Index}/{id?}");

This is a bit project specific, in MVC projects, a default pattern is defined for a request that goes like this:
https://localhost:port-number/controllerName/action

In MVC, controllers use patterns like the previous one, in web API controllers, attribute routing is used, like the following endpoint example:

[HttpGet("/books/all")]
public IActionResult GetProducts() {
    // code goes here
}

The Importance of The Middleware Order ⚠️

I've said in the analogy that the workers standing on the belt, are standing in order, same in the pipeline, each middleware should be at the right position. For example, the
UseEndpoints() middleware in the API project, we should add routing, before adding the endpoints, because if we have the endpoints, but we're not routing them, how will the request access them? So pay attention to the order, and make use of this article on Microsoft learn to better understand the ordering of middleware components:
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/middleware/?view=aspnetcore-7.0#middleware-order

Ok enough talk.

Let's Build a Custom Middleware 🛠️

We'll create a very basic middleware, that every time a request is made, it prints a message to the console. The upcoming set of instructions are capable to make you able to write some really handy middleware components, register them, and add them to your application's request pipeline, so follow along, and check out the tasks I've provided at the end of the post to get some hands on practice. With that said, let's start!
To build our simple middleware, follow the next steps:

1: Create a new class and call it PrintHelloMiddleware.cs
2: Make that class implement the IMiddleware interface
3: Implement the InvokeAsync method that IMiddleware defines.

Following the previous 3 steps, you can create any custom middleware you like, let's see the content of the method in our case:

public async Task InvokeAsync(HttpContext context, RequestDelegate next) {
    Console.WriteLine("Hello From PrintHello Middleware on the way to the end!");

    await next(context);

    Console.WriteLine("Hello From PrintHello Middleware on the way back!");
}

Ok what's going on?
The method takes two parameters, an HttpContext object called context, that'll represent an incoming request, and a RequestDelegate called next, first we print a message to the console, then we have this line:
await next(context);
next is a delegate, it takes an HttpContext object as a parameter and returns a Task. When a middleware calls the next delegate, it forwards the context object to the next middleware in the pipeline, it does some work on the context, passes it to the next middleware and so on. But notice we also have another print statement after calling the next delegate.
When the last middleware is done with the request, it sends back a response, and it gets passed up to every middleware component, in case they want to modify it, they can, if not, just forward it up in the pipeline.

In Program.cs, register the middleware as a transient service, like this:

builder.Services.AddTransient<PrintHelloMiddleware>();

Then in the beginning of the pipeline, register the middleware by using the UseMiddleware extension method, just like the following:

app.UseMiddleware<PrintHelloMiddleware>();
// Rest of the middleware components

ℹ️ Make sure to add the required using statement to access your newly created class PrintHelloMiddleware.

Run the app, and in the application's output window, every time you request a new page, refresh the page or something, you should see the messages printed like the following:

The first message is printed when the request is made, and when the request is processed, and the response was on its way back up the middleware chain, we printed another message indicating that it was printed when the request was already processed and the response was generated, isn't it simple?

Practice Tasks 📝

Now I have left out some pieces without coverage, you can tell me in the comments if you want to me to cover them!
Those are short-circuiting middleware components, and inline-middleware components, but I believe you're good on your own to figure those out, they're pretty simple as long as you understood the ideas we discussed earlier.

✅ Task 1

Build a logging middleware, our basic middleware used write line statements to print basic messages to the console, so why not create a middleware that gets called at the beginning of each request, and logs information about the request, like the request method type (GET, POST, etc...), and the response, if it's a successful request or not, you can easily do that following the framework we have used for building our simple middleware!

💡 The benefit of that is you'll have a centralized place for logging information about the requests and responses instead of having to do that in the action methods for instance!

✅ Task 2

Do some research, and see if the URL contains a particular number/value/character for example, redirect the user to an error page, saying the access is restricted, and make it that if the user is redirected, the middleware should short-circuit the request and not pass it down to the rest of the middleware components.

💡 Use tools like ChatGPT to help you out in completing the second task, as it includes things we haven't discussed in the post, like how to short-circuit the request pipeline

Conclusion

We have looked in this post at the request pipeline in Asp.Net projects from a very simplified perspective, to ensure that you understand and realize the benefit of it. Then we dissected the registered middleware components that come included by default in the Asp.Net MVC project template, learned the importance of the relative order of each middleware component in the pipeline, and also learned about how we can build custom middleware components that we can take advantage of by adding custom behavior to the request processing line, I hope you enjoyed that and learned something from this quite lengthy post! 🌟

Bye for now 👋🏻