DEV Community: Scott Hannen

Reflections and Corrections

Scott Hannen — Wed, 14 Aug 2024 04:00:00 +0000

If you’re reading this, perhaps you’ve found something on this blog interesting or helpful. Odds are that you’ve never read the About Me page. And why would you?

This post is about my other blog posts. Like the About Me page, I’m writing it only because to some extent my blog represents me. Over time, however, the individual posts may not. I feel compelled to address this somehow, even though it’s unlikely that anyone cares except me.

Some of this reflecton is a little bit funny. This blog contains bits and pieces of my learning journey, but it’s weird to read myself describing unit tests like I’ve just discovered fire. In other cases the information is obsolete. And sometimes I no longer feel the same way about what I wrote or I was just wrong.

With that in mind, here are some notes on a few older posts:

Abstract Factory Pattern

Alternatives To Abstract Factories Part I - Lifetime Management

I used abstract factories a lot in 2016. Since then it’s completely fallen off my radar. I didn’t use it in any domain modeling sense because I had no idea what that was.

In the example I created an IAddressValidatorFactory. Code that needed an IAddressValidator would pass it some argument (like an address). The implementation would decide what sort of IAddressValidator to create, create it, and return it.

In retrospect the way I used these factories made little sense. Even if it’s abstract, why would a consumer need to know that there’s a factory creating implementations of IAddressValidator?

There may be different address validation strategies. For example, the way we validate postal codes or provinces could vary depending on the country. But I’d encapsulate all of that within an implementation of IAddressValidator. Pass the address to it, and internally it decides which validation strategies to implement. Within that encapsulation there could still be a factory that returns a different IAddressValidator. But I wouldn’t expose any of that complexity to any consumer that just wants to validate an address.

Dependency (Constructor) Injection With Web API Action Filters

Once upon a time we needed this. Now we don’t. There are attributes like ServiceFilterAttribute and TypeFilterAttribute that simplify injecting dependencies into filters.

Define Types Instead of Using Primitives

This post could be replaced with

Google “value objects.” They’re great. Use them.

If we’re using C# records then we don’t need to implement equality checks.

# .NET Event Bus - Dependency Injection Friendly and Agnostic

This was fun to play with back in 2016. I think I used it once. In real life I think I used it once. I was picking up little pieces of domain modelling concepts but never got to use them. Eight years later I rarely get the opportunity to do anything that resembles domain modelling. Unfortunately the pull of anemic, database-centric code is very strong in many .NET environments.

Given the opportunity to use domain events I don’t know whether I’d bother with writing custom code when options like NServiceBus are available.

The Dependency Inversion Principle For Beginners

Dependency inversion is a valuable concept. These days it’s so baked into .NET applications that I’m not sure if the explanation helps. I don’t think about it when writing code. It’s just what we do. At the same time I didn’t understand the concept as well and my explanation was probably all over the place.

Depending on Functions Instead of Interfaces - Why and How

Depending on Functions Instead of Interfaces - The Navigation Problem

I still like the idea of it and I use it once in a while, but not as much as I hoped. It’s one more slightly different thing introduced into a code base and never catches on.

String Interpolation Functions vs. string.Format Constants

This is a solution without a problem.

The Interface Segregation Principle Applied in C#/.NET

I mostly stand by this. I argued that the principle is about dependencies and coupling, not about avoiding large interfaces. I suppose it’s about both, although we generally overlook the coupling issues and focus on the size of the interface. I’ll re-iterate that articles focusing on not throwing NotImplementedException completely miss the point. We shouldn’t throw that, but that’s not what the principle is about.

No, MediatR Didn’t Run Over My Dog

This is the one that haunts me. I was in an odd purist phase in which I ranted about MediatR left and right. Yes, it was misused in places. No, I still don’t see how it’s connected to the mediator design pattern. And yes, I still cringe when I see “CQRS with MediatR” articles because the two have nothing to do with each other.

That has nothing to do with the tool. I’ve used it many times since then. It’s familiar. It does something we need. My reasons why we shouldn’t use it didn’t make sense. I recant. (In this case I felt compelled to add comments to that effect all over the original post.)

An Experiment With Making Integration Tests Easier to Write - Part One

An Experiment With Making Integration Tests Easier to Write - Part Two

It was an experiment. It led me in the right direction but I wouldn’t do what I described here. It’s overcomplicated. Over time I’ve settled on something more like what I described in this post.

Cleanly separate dependency injection code so that I can re-use the production code in a test.
Create a TestServer or IHost and register dependencies with it using the same code as I use in production, including code that reads from configuration settings.
If neeeded, override some dependencies with mocks.
Either send HTTP requests using the HttpClient provided by TestServer, or resolve dependencies from the ServiceProvider and interact with them.

That’s it. I’m the only one who cares, but now I feel better.

Hexagonal/Ports and Adapters Architecture Part II

Scott Hannen — Tue, 06 Aug 2024 04:00:00 +0000

Summary of Part I

The previous post introduced some basics of Ports and Adapters architecture.

There’s an inside which contains application logic independent of technology like UI frameworks, databases, external HTTP endpoints, or messaging systems. The inside contains no references of any kind to any of those details, which are on the outside. We call the inside the application.
The inside exposes some sort of methods or interfaces used to interact with it. Those are called provided interfaces. (Interface does not mean the interface keyword in many languages, although we might use that.)
The inside also exposes interfaces that describe what it requires in order to function, like repositories. These are abstract and don’t describe any particular technology. They are called required interfaces.
Together, we call these external-facing interfaces ports. More specifically, a port is a group of related interfaces. That distinction matters, but if it’s unclear then set it aside for now and come back to it.
The ports used to make application do things are driving ports. The ports the application requires in order to function are driven ports.

I prefer not to overemphasize the terminology. We all learn differently. It helped me to understand concepts first. Then putting names with them “anchors” them.

How Do We Talk to the Application?

Everything on the inside of the application is abstract. There’s no UI and no database. It wouldn’t be useful or usable if we left it that way. How do we add those technology-specific details so we can actually use the application?

The missing pieces are adapters. These include things that interact with the application’s ports, telling it to do things. For example:

A UI where a user submits input or presses buttons
An HTTP endpoint that receives requests
A listener that receives messages
A timer that acts at intervals or scheduled times

Each of these starts with something that the inner application doesn’t know about - button presses, HTTP request models, events, and so on. Then they adapt those by calling methods exposed through the application’s ports. That might mean calling a method on an interface, sending a command to a command handler, or raising an event.

As mentioned in the previous post, the outside knows about the inside, because it must adapt the input it receives to something that the inner application can understand. The inside remains unaware of the outside.

Here’s an example in C#. Our application - the “inner” application - exposes an interface that accepts certain commands. That interface is a port. An HTTP endpoint accepts a request and adapts it to that port.

[HttpPost]
public async Task<IActionResult> CancelOrder(CancelOrderRequest request)
{
    // The inner application doesn't know what CancelOrderRequest is.
    // Map it to a command defined within the inner application.
    CancelOrderCommand command = request.ToCommand();

    // The application exposes some command handler interface.
    await _commandHandler.Handle(command);
    return Ok(); 
}

This bears repeating because it’s where the name of the architecture - “Ports and Adapters” comes in.

The application exposes a port, which is some technology-agnostic method consumers can call.
In the real world we need technology-specific ways to call those methods. Adapters are technology-specific. They provide a way to adapt an external input to a port.

That’s how we tell the application what to do. But what about the things the application needs in order do it, like a database, or somewhere to send notifications?

How Does the Application Talk to Its Database and Other External Resources

The answer is that we use adapters for that too. Remember, the application exposes required interfaces, which describe external things it expects to interact with. It knows nothing about the implementation. We meet that requirement by writing code which implements those interfaces, adapting them to some concrete implementation.

As an example, suppose our blog application requires a repository interface. It exposes this:

public interface IBlogRepository
{
    BlogPost[] GetAllBlogPosts();
    void SaveBlogPost(BlogPost blogPost);
}

We might look at that and assume that it represents a database, but notice that nothing in the interface specifies that. The implementation could also be something like this:

public class FileSystemBlogRepository : IBlogRepository
{
    public BlogPost[] GetAllBlogPosts()
    {
       // Code that reads files and maps their contents
       // to a collection of BlogPost objects       
    }

    public void SaveBlogPost(BlogPost blogPost)
    {
        // Code that converts the BlogPost to some file format
        // and saves it
    }
}

Remember the HTTP endpoint? It accepts some request object that the application doesn’t know about, and maps it to a command the application understands. The application doesn’t know about those details.

The same is true here. The application only knows about the IBlogRepository interface and its own objects like BlogPost. If that must be mapped to and from something else like a relational SQL tables or files, that happens in the adapter. The outside knows about the inside. The inside doesn’t know about the implementation details on the outside.

That keeps a lot of clutter out of the inside. Have you worked in an application that has numerous models for the same thing, like BlogPost, BlogPostModel, BlogPostDto, BlogPostData? At first you can’t tell what they’re used for, and eventually you memorize what each one means in the context of this one application. (The naming conventions are different in other applications because they’re meaningless.) That happens because either there is no inside and outside, or we bring all the classes from the outside to the inside, where they all mix together. Hexagonal architecture prevents that by keeping models used by the adapters on the outside.

Abstractions, Not Interfaces

A common mistake is for the inside to define interfaces that aren’t abstract. For example, if we need data that comes from an external Web API, we might define an interface like

public interface IAuthorApi
{
    AuthorResponse GetAuthor(Guid authorId);
}

Can you see how this goes against the flow of the architecture?

The interface isn’t abstract. It describes a specific implementation.
It returns the model used by that Web API.
If it fails we’re likely to get an exception or response with an HTTP status code. If the inner application must check for that, it’s no longer technology-agnostic.

Instead, we should define something truly abstract, like

public interface IAuthorRepository
// or IAuthorData - just not IAuthorApi
{
    Author GetAuthor(Guid authorId);
}

This assumes that we’re using an interface with a class implementation. That’s typical, but it’s a feature of certain languages. It’s not a requirement of Hexagonal architecture. The point is that we should avoid accidentally bringing technology-specific details into the inner application. The required interfaces on the inside should be abstract. Technology-specific details belong in the adapters.

Some More Terminology

Like before, I’m saving terminology for last. Why? Because understanding the concepts is more important than knowing names for them. If we can apply the concepts then it doesn’t matter as much whether we remember the terms. The terms are helpful, though, because they cement the concepts in our minds and enable us to communicate. There’s plenty of content surrounding this architecture, and it’s easier to understand if we speak the language.

With that, here are some more terms. Not all are specific to Ports and Adapters architecture.

A primary actor is whatever initiates action. That could be the UI, message listener, timer, etc. The adapters they interact with are called driving adapters. They drive. They’re active. They make things happen.
A secondary actor is something that acts because the application tells it to. That could be the database or outgoing message system. The adapters for those are called driven adapters. They don’t initiate anything. The inner application tells them what to do - give me this data, save this data, send this message.

That’s It?

At this point you might wonder if that’s all there is to it. After all that explanation, the underlying concepts are rather simple. If so, great. Why so many words? Because while it’s easy to understand, it’s oddly challenging to communicate. Try it and you’ll find out.

It’s not difficult to explain because people are incapable of understanding it. It’s because we all have our own mental models, and we’re thinking in terms of the code we’re used to seeing. Once we understand the concepts, it might take more effort to map them to our preferred technology stack and whatever type of software we write.

If you’ve followed along and got the concepts and even a few of the terms, now is a good time to Google and find other material. My hope is that if none of it quite made sense to you before, maybe it will now. Or if you understand it and you’ve struggled to explain it to someone else, maybe this post will help. We all learn differently, so there’s no telling which explanation will help any given person the most.

Also, Google “hexagonal architecture example in (whatever language you prefer.)” There are numerous concrete examples on GitHub. As you look at them, see if you can recognize how they implement the architecture. They will all be very different. Look for the patterns. See if you can spot things that don’t make sense, and think about what you might do differently and why. Remember that implementing the architecture does not mean that you must write code as it appears in any example.

What’s Next?

One aspect of the architecture I haven’t touched on is: How do we combine the inner application with the adapters to create fully functional software with a UI, database, etc. that we can deploy? I haven’t gone into that because the answer is specific to the tech stack we use, and even then there are numerous ways to go about it. Some of the existing code samples on GitHub will help. I will eventually follow up with a .NET-specific example.

The architecture and its terminology describe how it fits together. That part isn’t left out. I find it confusing to describe in abstract terms. If you’re interested, check out the section entitled The Configurable Dependency Pattern in this article by the late Juan Manuel Garrido de Paz who sadly passed away shortly after co-authoring the book Hexagonal Architecture Explained with Alistair Cockburn. Actually, read the whole article. Read the book.

Also, when should we use this architecture? When should we not?

I welcome any feedback on what helped or what didn’t. scotthannen@gmail.com.

Hexagonal/Ports and Adapters Architecture Part I

Scott Hannen — Mon, 29 Jul 2024 04:00:00 +0000

Hexagonal architecture is not a complex concept to understand, but that’s only true once we understand it. I find it difficult to explain. You can draw a diagram, but what does the code look like in a language you’re familiar with? You can illustrate it using code, but the architecture isn’t about how we write the code. Someone might look at the code and think that the architecture means writing code that looks like the example.

It also has terminology which is very useful when discussing pieces within the architecture, but might not initially help us understand the concepts. Someone might see a bunch of words used in unfamiliar words and say, “That’s it, I’m out.”

This post is a stab at making it easier to understand. We all learn differently. Maybe this will help someone who didn’t get it before.

I’m going to leave out the diagram. I’ll also try to lay off the terminology. After explaining concepts I’ll introduce the names we call them.

Brief Overview

Hexagonal architecture was described by Alistair Cockburn in 2005. Later he used the name “Ports and Adapters” because it’s more descriptive. There is no hexagon. There’s not six of anything. It was the shape of the diagram. “Hexagonal architecture” is still a valid name, and I’ll use that throughout because it’s shorter.

Some benefits:

We can design and test an application’s core behavior without having to go through the UI, setting up data in a database, or connecting one service to other services. Tests that do those things have value, but we shouldn’t need them just to test logic.
We can replace one technology with another more easily. Some might ask whether that actually happens. It certainly does. During initial development we might use a document database because it’s fast and easy to change, and later we might see the need for SQL. Today we do something when we receive an HTTP request. Tomorrow we might want to do the same thing in response to a message. That’s easier if the behavior isn’t tightly coupled to an HTTP endpoint.
The architecture helps us to avoid database-centric design and thinking. It shifts focus to the application’s uses and behaviors. Storing data in a database may be a necessity, but it’s not the reason why the application exists. The database is just something the application needs.

Inside and Outside

We can think of Hexagonal architecture as defining two areas - the inside and the outside. Picture a hexagon, a circle, whatever works for you.

Inside is code that describes what the application does and how it behaves. Inside is logic and behavior. There is nothing technology-specific on the inside.

No databases
No messages
No inputs and outputs defined by other systems, like the DTOs they use for HTTP requests and responses
No UI

None of those things are mentioned on the inside. All of those technology-specific details are on the outside.

An important principle is that the parts on the outside know a little bit about the inside. They exist to make the parts on the inside work. But the inside knows nothing about the outside.

How the Inside and Outside Work Together

The inside might not know about the outside, but in order for the application to do anything, all the pieces need to work together.

To make that possible, the inside contains interfaces that describe its interaction with the outside world. There are two types of interfaces:

Things that you can tell the application to do. Think of these as inputs.
Things that the application needs. It might need a repository that provides data it needs, or where it can store data. It might need to send out notifications when things happen. In both cases it knows that those things exist, but nothing about how they are implemented. Those interfaces are abstract.

Don’t be thrown off by the word “interface.” It does not refer to the literal interface keyword in some languages, although it might include such interfaces. They are ultimately methods, or things with methods.

Things on the outside call methods to convey input to the application, telling it to do something. Often an application has a set of commands, and we provide input by passing a command to a method that handles it.
Things on the inside call interface methods to interact with the outside.
- Give me this data
- Save this data
- Send a notification

At this point a concrete example might help. Suppose your application is a blog website, and one of the operations is to add a blog post. When a blog post is added, you want to

Verify that there’s not already a blog post with the same title
Save the new blog post
Raise a notification or event indicating that a new post has been added.

What might that look like?

First, there’s going to be some sort of interface that allows callers to tell the application. “I want to add a blog post.”

That could look like

public interface IBlogService
{
    void AddBlogPost(BlogPost post);
}

…or

public class AddBlogPostCommand : ICommand
{
    public string Title { get; set; }
    public string Content { get; set; }
    public Guid AuthorId { get; set; }
    public string[] Tags { get; set; }
}

…and something that accepts the command, like

public interface ICommandHandler
{
    void Handle(ICommand command);
}

Remember, don’t take these as concrete examples. Hexagonal architecture does not tell us how to write this code. The point is that the application provides some programmatic way to tell it to do something.

Then, in order to do its work, the application needs more interfaces describing the things it must interact with. For example:

public interface IBlogRepository
{
    BlogPost[] GetAllBlogs();
    void SaveBlogPost(BlogPost blogPost);
}

public interface INotificationServive
{
    void SendNotification(INotification notification);
}

Again, these are examples, not specifications for how we should write the code.

This allows us to write something like:

public void Handle(AddBlogPost command)
{
    var blogsWithSameTitle =
         _blogRepository.GetBlogs().Where(blog => blog.Title == command.Title);

    if(blogsWithSameTitle.Length > 0)
    {
        throw new Exception("There is already a blog with that title!");
    }
    var newPost = new BlogPost(
        Command.Title, command.Content, command.AuthorId, command.Tags);
    _blogRepository.SaveBlogPost(new BlogPost);
    var notification = new BlogPostAddedNotification(newPost.Id);
    _notificationService.SendNotification(notification);
    }

The behavior of the application is described apart from any technology.

Where does the command come from? A UI? An HTTP request? It’s not specified. This code doesn’t know and doesn’t care.
Where is the data? In a database? A file? Something else? Not specified.
Where does the notification go? An email? A message? Both? Not specified.

It’s easy to write tests for this because we don’t have to send HTTP requests or set up a database with test data. We don’t have to check a message queue to make sure a notification was sent.

What’s more, suppose we’re working on the logic of the application but we haven’t even decided what sort of database we want to use? That’s okay. We don’t need to answer that right now. It’s helpful to be able to delay those decisions or to change our minds as we learn more about the needs of the software.

If that makes sense then you understand most of Hexagonal/Ports and Adapters architecture.

This is a good time to introduce some terminology.

Application

The application is everything on the inside, everything that isn’t technology-specific. That’s confusing because really the application is everything from the UI to the database. But it helps to know what someone means when they use the term in the context of the architecture.

Ports

Ports are the interfaces that the application exposes to the outside world. Ports are part of the inside of the application.

There are two kinds of ports:

Driving ports are the interfaces used to tell the application what to do. They are also called provided interfaces. The application provides them to consumers.
Driven ports are the interfaces the application uses to do its job. They are also called required interfaces. The application requires them to do its job.

Now that I’ve explained ports in a simple way, I’m going to change that definition to be more accurate.

Ports are not the individual methods. Driving ports, in particular, are groups of methods provided for related use cases. For example, one port might be the methods used by people who create and modify blogs. Another port might be the methods used by automated services to perform some sort of maintenance.

That’s not too confusing, but maybe it’s one piece of information too many right now. It’s okay to back up and leave that for later. The distinction matters, but if everything else is mostly clear then our understanding is good enough for now.

One detail I want to call attention to: When we look at an application’s ports, we see all of the operations it is possible for someone to perform with that application. That’s helpful for two reasons:

If we want to understand or change what happens when we add a blog post, we know where to start looking.
This helps us to encapsulate our application’s data. The only way to interact with the application is through its ports. Unless we deliberately expose a port that says, “Update my data however you want,” then we have control over how our data is modified.

So far we’ve talked about the inside of the application. What comes next? The adapters - the parts on the outside which enable our application to connect to the outside world - to receive requests from actual users, save data, etc. That’s what the next post will cover.

For this to really make sense you might need to see a code example. If you want to see that right now, Google “hexagonal architecture examples in code” and specify your preferred language. Remember to look for the parts that correspond to what you’ve learned so far, but don’t get hung up on the specifics. That’s where confusion can set in. You can’t show someone code without writing code, and that inevitably means the code will be written in some specific manner. There’s no getting around that. But remember that those details are examples and do not define the architecture.

Eventually I’ll write a code sample myself in C#.

Don't Use Random Numbers in Unit Tests

Scott Hannen — Mon, 20 May 2024 07:00:00 +0000

Have you ever seen unit tests that generate random input values to ensure that the code works with a wide range of input values? This article explains why that makes no sense and we shouldn’t do it.

Here is a simple, contrived example. This function takes an int argument and returns true if the number is in the range from 5,000 to 10,000.

public static bool IsNumberBetween5000And10000(int number)
{
    if (number > 4999 && number <= 10001) return true;
    return false;
}

Did you spot the bug? The function will return true if the number we’re checking is 10,001. But that’s okay - I’m going to write unit tests, so I’ll find my mistake. (You’ve also caught me not doing TDD.)

But I’m worried that if I hard-code numbers in the tests then the test will only prove that the code works with those exact numbers. So instead of testing with hard-coded numbers I’ll use random numbers. That way I’m not just testing with one number. I’ll know that it passes for all the numbers in that range. (Yes, that’s exactly the reasoning.)

Here’s a test to verify that if the number is greater than 10,000 the method returns false.

[TestMethod]
public void RangeCheckerReturnsFalseIfNumberIsGreaterThan10000()
{
    Random random = new Random();
    int input = 10000 + random.Next(10000, int.MaxValue);
    int result = NumberChecker.IsNumberBetween5000And10000(input);
    Assert.IsFalse(result);
}

Great, the test passes! Do you see the problem? That’s a trick question. There are three problems.

First, the test shouldn’t pass because the code is wrong. I thought I was being clever by testing with every possible number greater than 10,000, so why doesn’t my test catch the bug? Because each time the test runs I’m not testing with all the numbers, I’m testing with one number. The test will catch the bug but only if the randomly-generated input is exactly 10,001. I might have to run it billions of times before it fails. Not a great strategy, is it? Hopefully this alone demonstrates that using random numbers to test a “range” makes no sense. The only real way to test a range is to run the test for every number in that range. (No, I am not recommending that.)

Next, my random number generation is wrong. I’m generating a random number that could be as large as int.MaxValue - 1 and then I’m adding 10,000 to it. In C# that could add up to a number greater than int.MaxValue, and the result is a negative number. In this case my mistake means that the test will pass, but it’s not testing what it should. In other cases this sort of mistake causes a test to sporadically fail even though the code it tests is correct.

These are the two biggest problems with using random inputs in tests.

Even if the test can catch a bug, you might have to run it hundreds or thousands of times before that happens. The bug the test is supposed to prevent might happen in production first. What was the point of the test?
Tests with random inputs often fail not because the code is wrong, but because we didn’t get the random value generation right.

What happens in real life when a test full of random inputs fails? The output doesn’t tell us what value was used. We don’t spot the bug. We run the test again and it passes. Great, never mind. If our test beats the odds and catches a bug, we will probably ignore it. If we actually try to figure out why the test failed that one time we will almost certainly spend more time trying to understand the test than trying to figure out if anything is really wrong with the code.

That brings us to to a third problem: If we do this in real-world scenarios (not like my contrived example) we’re likely to have tests with lots of random values. It’s not just numbers - we might use random numbers to create random dates. (Lots of things go wrong with random dates.) This results in tests that are very hard to understand. We have to carefully read the range of every random value to understand what the inputs are and then remember them. That’s hard. If we don’t know what the expected inputs are we can’t tell what the test is testing or why it should pass or fail. It’s exasperating to look at a failing test, find that it contains a dozen random numbers or dates, perhaps some of them are added to each other, and have to figure out what values they’re supposed to contain so we can understand why the test failed.

Please don’t do it. It’s all risk and waste with no reward. Instead, know what the edge cases are and test them.

In this case:

[TestMethod]
public void RangeCheckerReturnsFalseIfNumberIsGreaterThan10000()
{
    var result = NumberChecker.IsNumberBetween5000And10000(10001);
    Assert.IsFalse(result);
}

// or a parameterized test

[DataTestMethod]
[DataRow(4999, false)]
[DataRow(5000, true)]
[DataRow(10000, true)]
[DataRow(10001, false)]
public void RangeCheckerReturnsCorrectResult(int input, bool expected)
{
    var result = NumberChecker.IsNumberBetween5000And10000(input);
    Assert.AreEqual(expected, result);
}

Both of these tests are easy to read and catch the bug.

I do sometimes use random values when they aren’t relevant to the outcome, like Guid.NewGuid().ToString(). Perhaps I shouldn’t, but it’s arguably a different issue.

The Multidimensional Developer

Scott Hannen — Tue, 02 Jan 2024 05:00:00 +0000

Hopefully we’re past talking about the 10x developer. No one knows what it means except that if there’s a 10x there must be a 1x. It’s about comparisons between developers. I’m not interested in that.

Instead I’ll describe five dimensions along which developers can grow. Why five? Because that’s how many I think of right now. If I’d written this post ten years ago it would be called The 2D Developer. My purpose is not to compare anyone to anyone. (Such a comparison wouldn’t be stacked in my favor.) Nor is it to say that everyone should care about all of these things right now or how far anyone should pursue them. And although some things might naturally come before others, these are not sequential.

1. Languages, Frameworks, and Tools

This is the only dimension that is even remotely quantifiable, but it’s still vague. It describes the number of things with which we have worked or can work. It’s also where we start. We go from working with zero programming languages to one. Within that programming language we use additional tools or different frameworks. Being able to use one or two enables us to create software. The ability to use more than one increases our options.

We shouldn’t over- or under-emphasize this. Instead we should put it in perspective. Knowing how to use more things is good. It looks better on our resume. It keeps us from getting bored. It keeps our skills from becoming outdated. We have more ways to solve problems. We prove to ourselves and others that we can learn and adapt.

If we focus all of our energy on this there’s a risk that we’ll ignore other areas where we can grow, and the returns diminish. We might know how to write code in ten languages and use a dozen JavaScript frameworks but how much does that help us when we’re working on single project?

When I talk about this dimension I’m referring to the ability to make something work. It might not be pretty, but we can get something to render on a screen, move data in and out of a database, or send a message from A to B using some particular technology. If someone else doesn’t know how to make it work and we do, we can help them. We can answer questions on Stack Overflow. It’s valuable all by itself.

2. Code Quality

At some point we’ll likely care about more than getting stuff to work. We’ll want to write code that’s easier to read, change, and test. It’s not the same as knowing how to write code in a given language. We can be fluent in English but have room to improve our communication skills.

There are plenty of divergent opinions on what “code quality” means. What matters is that there is such a thing, it matters, and we can improve. It might include language-agnostic details like using descriptive names. If we work in OOP languages we might learn principles such as those included in the SOLID acronym. (I hesitate to mention SOLID because we often place great emphasis on those five principles and leave out many others. Or we might not agree with all of those principles. But do we know what they are and why we agree or disagree with them?)

In my opinion code quality includes automated testing. There are differing opinions on what sort of tests we should write, how many, and what they are called. (You may have encountered testsplaining. “Actually, that’s not a unit test. It’s an integration test.”) But most agree that we should learn to write some sort of automated tests so that we don’t depend on manually testing everything to know whether it works, which eventually borders on impossible. We can and should form our own opinions. Many developers swear by Test-Driven-Development. You may agree. but realize that writing tests and TDD are not the same.

As with learning new technologies, we should put code quality in perspective. If we’re struggling to learn how to make Angular display a component within a component then we probably have to focus on that and not get hung up on other details. There are subjective opinions about the importance of code quality. I will concede that it isn’t everything (only because nothing is everything.) At some point we must decide what’s good enough and deliver code. Its subjective quality only matters to the extent that it helps us to deliver valuable software.

Having described two of these dimensions, it’s also easier to see how they complement each other even if we grow more along one and not the other. Consider two developers within a team. One knows more about what makes code maintainable, but they’re learning React and they can’t figure out why their component is stuck in a loop or how to manage state. The other developer hasn’t thought much about code quality but they understand React hooks. What each knows is valuable despite what they don’t know. It’s pointless to argue over which is more important.

3. Business Value and Process

We can make a computer do what we want using various languages and frameworks. We can make it easy for developers to read and maintain. Those qualities matter. But regardless of that, is the software valuable? Does it software do anything that people need or want? To the extent possible, does it work the way they want it to? Does our end-to-end process streamline the creation and delivery of value or hinder them?

At some point we’ll likely experience frustration because although we can competently build what someone tells us to build, we find that it doesn’t get used. Maybe no one wanted it, or when they got what they wanted they didn’t like it. It might get thrown away, or we might spend time re-working it and wonder if there could have been a way to prevent so much waste.

Another dimension along which we can grow is learning how to find out what is valuable and deliver it with less waste.

A number of principles to help us accomplish this are in the Agile Manifesto. Unfortunately this has also become controversial. The word “Agile” has been abused over the years and when people say it they often mean Scrum, backlogs, story points, standups, and more that has nothing to do with Agile. It can help but more often it adds bureaucracy and distracts from Agile. Others may disagree with the Agile Manifesto itself.

I’ll sidestep most of that and cite the first value:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Hopefully we can all agree on that. If we don’t deliver software then what are we doing? If it’s not valuable then why are we doing it? Read the other values and the twelve principles. They represent lifetimes of experience and are in my opinion a solid foundation, but my intent is not to dogmatize them. My point is not that we should grow by adhering to Agile but that we should learn to pay attention to the value of software and reduce waste in the process of delivering it.

This area of growth can be harder because unless you work alone you can’t improve alone. For example, producing valuable software with less waste requires increased collaboration with users and business experts. It could be that your organization doesn’t work that way, doesn’t realize how much better the results would be, and doesn’t want to change. They might be stuck in a version of “Agile” which is actually a rigid process that hinders collaboration and innovation.

If that happens you’ll have to decide whether to limit your own growth or to move on to another organization or even become self-employed. I’m not recommending any over the others. Sometimes it makes more sense to just get paid. I hope that your organization wants you to grow even if it must grow with you. A developer who collaborates and solve problems is far more valuable than one who simply follows instructions, but your organization must recognize, allow, and encourage it.

Like everything else I’m writing about here, this is not all-or-nothing. We can start small by questioning requirements and suggesting alternatives. One wasteful thing we don’t do can save many hours and lots of money while enabling us to do something more valuable instead.

4. Architecture

In some respects this is just an extension of code quality. Just as we decide how to make code readable and maintainable when writing classes or methods, we do so when deciding how to arrange and structure code. Good architecture is always deliberate. If we don’t decide how to architect our code then we’ll make each change according to whatever seems easiest to the person making it at that time. We will not accidentally achieve a maintainable design. We will create a big ball of mud over and over.

Architecture involves more than just our own code. Eventually we’ll work on larger systems that are composed of smaller applications. (Microservics are common. Not necessarily better, but common.) If a mass of spaghetti code can happen with one application, it can be so much worse with numerous such applications talking to each other. It can easily reach an unrecoverable state in which the larger system no longer works and it is impossible for anyone to understand why or fix it.

As applications work together we must decide how they should interact and why. Should they send messages to each other, make HTTP requests, both, or something else? What happens if one application becomes unavailable? Can we stop the entire system from failing?

Many companies have separate teams to handle architecture across systems. Whether you make decisions, they do, or you collaborate will vary. That may depend on how much you know. It helps if we’re in a better position to make decisions instead of depending one someone else.

5. User Experience (UX)

This is closely related to business value but it’s worth calling out separately.

UX involves seeing software from the perspective of its users. This overlaps with something I mentioned earlier, which is creating software that users want instead of assuming that they’ll use whatever we give them. It also includes making software less confusing and easier to understand.

Software developers have a mixed reputation in this area. Some say that developers are incapable of making good UX decisions. This has led to the unfortunate proliferation of design-first development processes in which a UX designer decides what is best for users and then hands their design off to developers to implement.

We can prove the naysayers wrong by learning for ourselves what enhances or detracts from UX. This may include more communication with our users so that we understand their perspective more. A lot of UX can be generalized. For example, if we realize that a button’s text doesn’t clearly describe what it does, it’s better to change the text than to expect users to learn what it means.

There are degrees of UX expertise. A skilled UX designer will likely have insights that developers won’t. But I’ve never once seen a case where a designer made all the right choices. We arrive at the best design by collaborating, and learning about UX allows us to contribute meaningfully to that collaboration. Regardless, we should speak up when something in a design seems unclear or counterintuitive and listen to anyone else who does so.

The Circles of Zorro

I hope no one reads this and thinks that they or someone else must learn about Agile or architecture in order to contribute value to a software development team. That is absolutely not the case. What I’m describing are opportunities for growth. If we’re fortunate enough to work within a collaborative team and organization then we can complement each others’ skill and knowledge.

If I omitted somethng it’s not because I don’t think it’s important. Communication is important. It just isn’t specific to software development.

This brings me to the Circles of Zorro, which is a made-up term for the way old Zorro (Anthony Hopkins) trains new Zorro (Antonio Banderas) in the film The Mask of Zorro.

Old Zorro instructs new Zorro to stand inside a small circle and tells him that his entire world is within that circle. He cannot think in terms of larger strategies until he learns to fight directly against opponents within that circle. He must start with a narrow focus and then expand his reach.

I like this illustration not because it describes how software developers should learn (it doesn’t) but rather how our sphere of decision-making and influence grows. For better or for worse, our early work experience will place us in a small circle. We’ll be told to make something work and we won’t think about more than how to do that. That’s okay. At that point we’re under a heavy cognitive load trying to figure out one thing and we’re almost certainly not thinking about architecture.

But that does not mean that we must be like Zorro-in-training and master one circle before moving on to larger outer circles. The areas of growth I’ve described are not circles with in circles. They are different ways to grow. As we get out of our initial survival mode we can explore any or all of them in any order. (This isn’t exhaustive. These are the ones that come to mind now categorized according to how I think of them.)

The point is that we should not measure our growth by the number of languages or frameworks we learn, or limit our growth to that. There’s more to learn and more ways to grow.

Simplify .NET Development With Fewer Projects and Solutions

Scott Hannen — Tue, 26 Dec 2023 05:00:00 +0000

I’m trying to write shorter blog posts so that perhaps I’ll produce more.

Two ways in which we unnecessarily complicate .NET applications are by

creating more projects than we need within a solution
creating more solutions than we need within an application

Too Many Projects

Projects are a unit of modularization. In other words, they are a way to keep some parts of our code separate from other parts. They aren’t the only way to modularize. We also do this by putting code in different folders, which usually means putting it in different namespaces.

Why Separate Code Into Projects at All?

If we can modularize using namespaces, what is the benefit of projects? A big one is that projects allow us more control over how classes are referenced by and visible to other classes.

If we’re employing something like Clean Architecture or Hexagonal Ports-and-Adapters architecture (or we’re just trying to stay well-organized) then we might want to keep higher-level logic like business rules separate from implementation code like database repositories.

An example is that our logic might refer to an ISomethingRepository interface. That’s all we want it to know about. We don’t want details about SQL or some other database leaking into our logic. We can enforce this by placing our logic in one project which contains the interface. Then another project contains implementation details. It references the project containing the logic and it implements the interface. Two projects cannot reference each other, so this relationship enforces our wish that the logic will not reference implementation details.

A second, more obvious reason to have separate projects is that they contain applications which can or must be deployed separately. For example you might have a Web API project and a message listener that processes dequeued messages. These are separate components and we wouldn’t normally try to put them both in one project.

When Not to Separate Projects

Sometimes we have SQL code, ServiceBus code, HTTP client code, and some other implementation code, and for some reason we feel that because they are different they belong in separate projects. Why? I don’t know. Unless we’re aware of a reason why they must be separate, it probably doesn’t do anything for us except increase the number of projects which makes the solution harder to navigate.

If we put every different type of code in a different project, we’re likely to also create separate unit test projects for each of them, doubling the number of projects.

Finally, we’ll inevitably discover something that all of those projects need to share, so we’ll create yet another project to put that code in. At this point our dependencies will be unnecessarily confusing. I prefer to avoid the overused “new developers will be confused” argument, but in this case it might actually apply. It can be hard to look at all of these projects and keep track of what references what and why.

Too Many Solutions

Excess solutions can be even more problematic than excess projects. An example is when we have a single domain or bounded context. That might require a Web API, a message listener, a scheduled service, and perhaps other related components. Our reflex may be to put each of them in a separate solution. Perhaps we reason that if we put them in one, the solution will have too many projects.

(When I refer to multiple solutions, I don’t mean those that exist within the same directory and perhaps share projects, but rather having multiple directories each containing one solution and projects which belong to that solution.)

There’s obviously no rule that says we can’t put components in separate solutions that way. There may be valid reasons. But we shouldn’t do this by default because it creates complications that can increase the complexity of development.

We Lose the Benefits of Project References

This is a concern if we’re trying to keep our application logic separate from our implementation details. If we have a web API and a message listener service in two solutions, where is the higher-level application logic?

There are a few ways this can work. And they do all work, but they’re all sub-optimal.

One possibility is that we put some application logic in the Web API solution and some in the message listener solution depending on where we need it right now. We might put it in a separate project within that solution so we can pretend that we’re keeping our logic and implementation details separate, That’s self-deception. Wherever we put it that logic, that’s what it’s coupled to. What happens when there’s some operation performed in the web API but now we want to do it in response to a message? We’re either going to have to duplicate that logic or we’re going to have to find some other way to share it between solutions.

That need to share code between solutions is almost inevitable. It frequently leads us to place shared code in yet another solution and build a NuGet package which is consumed by the others. This slows development because we must first change code in one place, build a package, and then reference it in another place. But there are other projects that don’t need the latest version of our NuGet package. It’s easy to introduce a breaking change and not notice until later when we update those other projects to a newer version.

These problems go away when projects are located in the same solution. As we make changes to shared code we know right away if we’ve introduced breaking changes. Either the solution won’t build or unit tests fail. The feedback is immediate. And we don’t spend time building a package from one solution and updating references in another.

A Gift That Keeps on Giving

The problems created by sharing code between solutions tend to self-perpetuate.

We might create a package for sharing code between solutions, but it’s for a specific purpose. When we need to share more code that does something else, we create more packages.
We might not move shared logic into shared code until we need to share it. This leads us to having application logic scattered in various places which makes it harder to understand. This often results from a misunderstanding of DRY and a lack of architecture. We may have learned (correctly) that we shouldn’t over-aggressively remove code duplication. Only de-duplicate when similar or identical code exists in perhaps three or more places. But when it comes to business logic we don’t want to scatter it about and then gather it again as needed to avoid duplication. Our architecture should keep business logic in one place right from the start. This avoids the costly and risky work of pulling it out of an application (to which it has likely become coupled) and moving it into some shared repository.

Code Duplication

If we don’t have lots of packages then we’re likely to have lots of duplicate code. Our different application components might use the same database (which is okay if they’re part of the same domain or context) and end up duplicating code to interact with it. Or perhaps they communicate with the same external Web APIs and we duplicate HTTP client code.

Everything Goes Through the Web API

This is quite possibly the most common solution, but also one of the worst. Our application likely has not only business logic, but code to ensure that when data is modified, a message is sent out on a message bus, or some other behavior occurs. To ensure that this always takes place, we put all of that behavior in a Web API application, and other components in the same domain interact with domain entities and data by sending HTTP requests to that web API.

This adds a level of unnecessary complication. If these projects were in the same solution they could all reference the same project containing business logic, command and query handlers (often called the “application layer.”) Instead each operation becomes distributed. One component handles part of an operation and then makes an HTTP request to a web API to complete another part.

This makes understanding the application at runtime more difficult. An operation might begin within one component but fail because of an exception thrown in another component, and now we must correlate the two to understand the problem.

Unless we’ve been careful with our architecture there’s a good chance that the operation consists of business logic that exists at both ends of that HTTP request. That also makes it harder to understand.

It also makes testing more difficult. We can write a test for the message listener that receives a message, performs some logic, and sends off an HTTP request. We can write another test for the part that’s handled by the web API. But it’s much more difficult to write a test for the entire end-to-end operation.

This is inevitable if our application makes a necessary HTTP request to a web API which is part of another application. It is needless complication if our application could invoke code via a project reference but instead makes an HTTP request to another part of itself.

Ironically, while we split code into solutions to avoid having solutions with more projects, we’ll likely find that when we keep it together we’ll have fewer total projects and less code because of reduced duplication and infrastructure.

Conclusion

The complexity we add by unnecessary splitting code into projects and solutions doesn’t stop us in our tracks. Otherwise we wouldn’t do it. Rather it slows us down, compounding the effects of other unrelated decisions that slow us down. No number of straws break the camel’s back. Each added straw just makes the camel’s job harder.

We can also compare excess splitting to when we were children and we spread the food we didn’t want to eat around on the plate to make it look like there was less of it. But there wasn’t really less of it. In this case by spreading it around we might even make more of it, and someone has to eat it.

I’m not proposing a rule. Rules rarely work. But if our rule has been that different types of code go in different projects and different components go in different solutions, we don’t have to keep following that rule either. It probably hasn’t helped us.

SOLID Principles Expressed as Rules

Scott Hannen — Fri, 19 Nov 2021 05:20:58 +0000

Rules are specific applications of principles. We can and must apply principles without rules, but any application of a principle is an action or inaction. Rules make it easier to remember which actions or inactions apply a principle.

For example, “Be safe” is a principle to follow when driving. But what if every time we drove we had to consider every action and decide what was or wasn’t safe? That’s a lot to think about. Rules help us with that. Drive on the right side of the road. Stay within the speed limit. Stop at red lights. We still have to make decisions, but many decisions are already made because they uphold the principle, “Be safe.”

Rules aren’t perfect. They can’t cover everything. We still need to use judgment. Sometimes they’re arbitrary. Why is 70 miles per hour the limit instead of 75? Someone picked it. Do we have to drive on the right side of the road to be safe? No, people in other parts of the world drive on the wrong side of the road and they’re okay.

We can make up our own rules. I don’t make left turns across busy roads unless there’s no traffic. I can if I want to but I don’t.

I’m saying all of this to put rules in perspective. Principles are more valuable than rules, but rules help us to follow principles.

With that in mind, I’ve put together some rules that I use to help me follow SOLID principles. They convert valuable but sometimes vague principles into specific behaviors. I do not assert that everyone else should follow them any more than I would tell people in the UK to drive on the right side of the road. They are for me. And I suggest them when I’m trying to help others apply the same principles.

I don’t regard SOLID principles themselves as if they were carved in stone and brought down from the mountain. I’ve just found that they help me to write better code. Others found that out first which is why they described them and made a nifty acronym out of them.

So here are my rules. Maybe you’ll find them helpful. Feedback is welcome because I like to know if this was helpful, but also because you might have an perspective I haven’t considered.

Single Responsibility

The name of a class should be specific enough to both indicate what it does and exclude what it does not do. If a class is named OrderService there is no limit to what we might add to it. If a class is named PostalCodeValidator then its responsibility is obvious and anything we add to it beyond that responsibility will stick out.
A class should have no more than n injected dependencies. My magic number is five. New classes rarely have more than three, but if I’m about to exceed I refactor.

Open/Closed

If we have a class that does X but we also need Y, don’t modify the existing class so that it does both. Just make a new class.

If we have a method that does X but we also need Y, don’t add optional parameters or figure out how to make one method do different things. We don’t get extra points for making one thing do two things. Make another method.

To be honest I have the hardest time with this principle. I almost never consciously apply it. Maybe I should. But principles exist to benefit us, not the other way around. It doesn’t need me to apply it. So I’ll leave this as a question mark. Maybe I’ll understand better one day, or I won’t.

Liskov Substitution

Do not throw NotImplementedException. The exception is if we’re forced to implement an interface we didn’t create. It should be rare. It might never happen.
If a method has a variable or argument of type X, it should not need to know about anything else about the type of that object. It should not need to use reflection to see if the object is of some other type so that it knows which methods it can call or what to do with it.

This second rule is riddled with exceptions. We may deal with framework types or libraries that give us no other choice. Sometimes infrastructure code uses reflection to inspect types and create generic types.

If we must have code that inspects the types of objects, we should ask, is it deliberate, or is it a one-off because of poor interface or class design? If it’s the latter then we should either try to fix it or encapsulate it inside some class that “hides” that weird, one-off difference from the rest of our code.

Interface Segregation

This one is confusing because .NET has an interface keyword. The Interface Segregation applies to those interfaces, but it’s not limited to them. The “interface” is any contract that a type exposes. For simplicity I’ll refer to interface interfaces, but we can apply it to classes as well.

Define interfaces and abstractions from the viewpoint of the class that depends on them. In other words, if you realize that a class which submits orders needs to validate orders, create an IOrderValidator interface with only the methods the consumer needs. Don’t find or create a general-purpose IOrderAllTheStuff interface, add another method there, and depend on that.
Just like with the Single Responsibility Principle, give interfaces names that describe what they do and exclude the rest. Single-responsibility interfaces tend to have single-responsibility implementations.
If the functionality we need to depend on is in a giant class or interface, consider defining the smaller interface you need and adapting the existing type to your narrower interface. In other words, the implementation of your small interface “wraps” or hides the larger type.
Avoid mutually exclusive use of interface members. That is, one class uses some members and another class uses other members. In that case why are those members in the same interface instead of two or more different interfaces?

There are exceptions. IRepository classes with all sorts of create, update, get, and delete methods are common. It’s familiar to developers across codebases, so I question how helpful it is to insist on interface segregation. I like defining a separate IReadRepository interface with read methods and then having the write methods in an IRepository interface which inherits from IReadRepository. That way if a class should only do queries it can depend on the smaller interface, even if the implementation is all one class.

I’m not rigid with interface segregation. Maybe one consumer depends on all the members of an interface but another depends on only one. That’s okay. What matters is that we aim for that separation instead of throwing everything into a few big interfaces willy-nilly.

Dependency Inversion

Create a “high-level” project which contains important application logic. Don’t put any implementation details in that project - no API clients, no database code, no messaging, and no controllers. Define interfaces to represent whatever that high-level code needs to do that might involve those implementation details.

(I work mostly in .NET which is why I refer to “projects.” It’s a way that we organize parts of our code that stay together because they’re related but stay separate so that they aren’t coupled or we can decide how they are coupled.)

Those interfaces should describe uses or behaviors. They should not have names like IFooApiClient or IFooSqlRepository that describe implementations. And those interfaces should be in the same project as the code that depends on them.

The implementation details are in in other projects. Those implementation details are “low-level” code. They depend on (reference) the high-level project and implement its interfaces.

Whatever models or entities that high-level code uses should also be defined within the high-level project. If it’s convenient low-level code can re-use them. But high-level code should never know about contracts or models used by low-level code.

For example, our high-level code might have an Order class. If we get order data from an API in low-level code, whatever model the API returns should be mapped to the high-level Order class within the low-level code.

Code that allows users or external actors to interact with our high-level code is also an implementation detail. It is low-level code. A Web API project or Azure Function can re-use models from our high-level code, but models from low-level code should not appear in our high-level code.

We should not create “shared” projects with models used by low-level and high-level code. Once that happens the sharing can go two ways when it should only go one way.

An exception is when the whole application is low-level code. Maybe it’s infrastructure and only exists to interact with databases or messaging. In that case the difference between high-level and low-level doesn’t exist.

But when in doubt it’s easy to start with separate projects. We can always combine them later.

Conclusion

If all of this sounds a bit opinionated it’s because interjecting “maybe” and “sometimes” is a poor writing practice that I give into a lot anyway.

These are my rules (in the sense that I follow them, not that they all originate with me.) I chose them, which means I’m also free to revise or break them as I see fit. They serve me by acting as shortcuts that help me apply what I’ve already learned. Good things happen when I follow them.

I am not asserting that my rules should be your rules. Use them, some or all, or something like them, if it helps.

ASP.NET Core TestServer - How Did I Not Know About This?

Scott Hannen — Thu, 18 Nov 2021 05:20:58 +0000

I’ve been looking for ways to write better integration tests. My previous set of experiments proved useful in practice, but the approach could use maybe one level of de-complication.

In the meantime I discovered something out-of-the-box tool that provides a simple way to test ASP.NET Core applications: ASP.NET Core TestServer. It doesn’t replace what I worked on previously. It’s something separate. I use both approaches. But this is easier and doesn’t require changing the way an application is configured.

In this post I’ll start with a simple scenario in case that’s all you need and then get into some of the more fun stuff we can do with TestServer. Once you see the potential you’ll likely get more ideas about how to write the sort of tests you want to write.

The Shortest, Simplest Version

I’m starting with Visual Studio’s included Web API application which includes a WeatherForecastController which returns a random weather forecast from this GET endpoint:

[HttpGet]
public IEnumerable<WeatherForecast> Get()
{
    var rng = new Random();
    return Enumerable.Range(1, 5).Select(index => new WeatherForecast
    {
        Date = DateTime.Now.AddDays(index),
        TemperatureC = rng.Next(-20, 55),
        Summary = Summaries[rng.Next(Summaries.Length)]
    })
    .ToArray();
}

Next I’m adding a test project using the “NUnit Test Project (.NET Core)” template. You can use MsTest. It doesn’t matter. The test project references the Web API project. And I’m adding a few NuGet packages:

Microsoft.AspNetCore.TestHost v3.1.21
Newtonsoft.Json (latest)
Moq (latest)

Here’s the test file in its entirety. I’ll explain what each part does, but it’s probably obvious just from looking it.

using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.TestHost;
using Newtonsoft.Json;
using NUnit.Framework;
using System.Net.Http;
using System.Threading.Tasks;

namespace TestServerBlogPost.Tests
{
    [TestFixture]
    public class WeatherForecastIntegrationTests
    {
        private TestServer _testServer;
        private HttpClient _testClient;

        [SetUp]
        public void Setup()
        {
            _testServer = new TestServer(
                new WebHostBuilder()
                    .UseStartup<Startup>());
            _testClient = _testServer.CreateClient();
        }

        [TearDown]
        public void TearDown()
        {
            _testClient?.Dispose();
            _testServer?.Dispose();
        }

        [Test]
        public async Task Controller_Returns_Forecast()
        {
            var request = new HttpRequestMessage(HttpMethod.Get, "/weatherforecast");
            HttpResponseMessage response = await _testClient.SendAsync(request);
            string responseContent = await response.Content.ReadAsStringAsync();
            WeatherForecast[] forecasts = JsonConvert.DeserializeObject<WeatherForecast[]>(responseContent);
            Assert.AreEqual(5, forecasts.Length);
        }
    }
}

Here’s what’s happening:

We’re creating a TestServer using a WebHostBuilder that in turn uses the Startup from the Web API application.
TestServer exposes an HttpClient which allows us to send HTTP requests to the application. Both are created before each test runs and disposed after each test runs.
The test sends an HTTP request to the “weatherforecast” endpoint, gets a response, and asserts something about that response.

We can already see the possibilities. We can send different types of requests. We can add headers to requests to verify that our authorization works. We can inspect response headers if that’s what we want to verify. Our tests include middleware.

Some other benefits:

Unlike tests that require starting the whole application, these are fast. The first test might take 300ms, but after that they’re less than 50ms.
We don’t have to duplicate models used within the application. The test references the application and uses its models.
We’re not duplicating what’s in Startup so we can run tests. The more we do that, the less we’re testing our actual runtime code, defeating the purpose of the test. We’re doing the opposite - we’re testing Startup. If there’s some dependency we haven’t configured in Startup we’ll catch that.

I wouldn’t write a lot of these. We can cover most application logic with unit tests. These integration tests are only to cover what unit tests don’t cover: Making sure that the pieces are all configured correctly and they all come together as a whole.

Having covered the happy path, let’s look at some other scenarios.

Configuration

What if our Startup depends on some configuration value? I’m going to change startup so that it depends on a configuration value by adding this to ConfigureServices:

var awesomeness = Configuration.GetValue<int>("awesomeness");
if (awesomeness < 1) throw new Exception("Not awesome enough!");

Now the test that used to pass fails instead because that setting is missing. So how do we supply it?

Here’s one way. It feels like cheating but I can live with it. In the web application’s Program class I have this:

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .ConfigureHostConfiguration(configurationBuilder =>
        {
            configurationBuilder.AddJsonFile("appsetting.json");
        })
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.UseStartup<Startup>();
        });

It loads a JSON settings file, and I’ve put the setting in appsetting.json so the app can run:

"awesomeness": "1"

Now how do we supply this setting to our test also? One solution is just to copy the settings file into the test project. Maybe add a comment noting that it’s a copy of the “real” file. Make sure the file’s Properties indicate that it’s copied to the output directory.

Then, in the test, make sure that settings file is included in the configuration:

[SetUp]
public void Setup()
{
    _testServer = new TestServer(
        new WebHostBuilder()
            .ConfigureAppConfiguration(configurationBuilder =>
            {
                configurationBuilder.AddJsonFile("appsetting.json");
            })
            .UseStartup<Startup>());
    _testClient = _testServer.CreateClient();
}

(Note that in Program it’s ConfigureHostConfiguration but here it’sConfigureAppConfiguration.)

Now the test passes. That might be good enough. But what if you want your tests to use different values? We don’t want to have to create separate versions of our settings files. In that case we can supply or override those settings in our test using AddInMemoryCollection like this:

[SetUp]
public void Setup()
{
    _testServer = new TestServer(
        new WebHostBuilder()
            .ConfigureAppConfiguration(configurationBuilder =>
            {
                configurationBuilder.AddJsonFile("appsetting.json");
                configurationBuilder.AddInMemoryCollection(CreateInMemoryConfiguration());
            })
            .UseStartup<Startup>());
    _testClient = _testServer.CreateClient();
}

private IDictionary<string, string> CreateInMemoryConfiguration()
{
    var configuration = new Dictionary<string, string> { {"awesomeness", "2"} };
    return configuration;
}

Because we call AddInMemoryCollection after AddJsonFile the new value we supply overrides what’s in the file. We could decide that we don’t want the file at all or we could keep it for convenience and override whatever we want to change. It might make sense to have a second file that overrides settings in the first file. Our options are open.

Adding Missing Dependencies

Sometimes the application depends on components that are registered in Program instead of in Startup, and those can cause our tests to fail. If we need to we can add them to our TestServer using ConfigureServices like this:

[SetUp]
public void Setup()
{
    _testServer = new TestServer(
        new WebHostBuilder()
            .ConfigureAppConfiguration(configurationBuilder =>
            {
                configurationBuilder.AddJsonFile("appsetting.json");
                configurationBuilder.AddInMemoryCollection(CreateInMemoryConfiguration());
            })
            .ConfigureServices(services =>
            {
                services.AddTransient(typeof(ILogger<>), typeof(NullLogger<>));
            })
            .UseStartup<Startup>());
    _testClient = _testServer.CreateClient();
}

Replacing Services With Fakes or Mocks

This is where TestServer provides some great flexibility. Since this is an integration test we might want it to interact with a database instance. In that case the test will get the connection string from the settings file or we can supply a different one somehow.

What if for whatever reason we don’t want our test communicating with a real database? What if want to replace it with some in-memory database?

Let’s say our application depends on this interface, and the implementation interacts with the database:

public interface IFooRepository
{
    Task<Foo> GetFoo(Guid id);
    Task SaveFoo(Foo foo);
}

Startup may register a “real” implementation, but TestServer allows us to override it. Moq isn’t my first choice, but it’s brief and familiar, so let’s replace the runtime implementation with a mock.

We’ll add a field for the mock, and then Setup will instantiate it and use the ConfigureTestServices method to override whatever implementation Startup specifies for IFooRepository.

private Mock<IFooRepository> _fooRepositoryMock;

[SetUp]
public void Setup()
{
    _fooRepositoryMock = new Mock<IFooRepository>();
    _testServer = new TestServer(
        new WebHostBuilder()
            .ConfigureAppConfiguration(configurationBuilder =>
            {
                configurationBuilder.AddJsonFile("appsetting.json");
                configurationBuilder.AddInMemoryCollection(CreateInMemoryConfiguration());
            })
            .ConfigureServices(services =>
            {
                services.AddTransient(typeof(ILogger<>), typeof(NullLogger<>));
            })
            .ConfigureTestServices(services =>
            {
                services.AddSingleton<IFooRepository>(_fooRepositoryMock.Object);
            })
            .UseStartup<Startup>());
    _testClient = _testServer.CreateClient();
}

Now we can write tests that set up the mock to return certain values or we can use it to verify that the application saved the expected Foo.

Resolving Services

In the above example we’re using a mock instead of a real database. What if we wanted to use a real database but we needed to set up test data (inserting a Foo) or see what was inserted into it (reading a Foo)?

TestServer exposes the application’s IServiceProvider via its Services property. We can use that to resolve IFooRepository and interact with it. Instead of writing more database code to insert and inspect records, we can use the same classes that we use in our application.

In our test method we can set up test data like this:

var fooRepository = this._testServer.Services.GetService<IFooRepository>();
await fooRepository.SaveFoo(new Foo {Id = Guid.NewGuid(), Name = "Foo!"});

A Base Class to Make Writing Tests Easier

I rarely write base classes for tests but in this case it helps by reducing duplicate code. This base class

encapsulates creation and disposal of TestServer
exposes the HttpClient needed to send requests and the IServiceProvider for resolving services
calls virtual methods for adding configuration settings or replacing services with fakes, mocks, or something else

[TestFixture]
public class TestServerTestBase // worst name ever
{
    private TestServer _testServer;
    protected HttpClient TestClient { get; private set; }
    protected IServiceProvider Services => _testServer.Services;

    public virtual void Setup()
    {
        _testServer = new TestServer(
            new WebHostBuilder()
                .ConfigureAppConfiguration(configurationBuilder =>
                {
                    configurationBuilder.AddJsonFile("appsetting.json");
                    configurationBuilder.AddInMemoryCollection(CreateInMemoryConfiguration());
                })
                .ConfigureServices(services =>
                {
                    services.AddTransient(typeof(ILogger<>), typeof(NullLogger<>));
                })
                .ConfigureTestServices(OverrideServices)
                .UseStartup<Startup>());
        TestClient = _testServer.CreateClient();
    }

    [TearDown]
    public void TearDown()
    {
        TestClient?.Dispose();
        _testServer?.Dispose();
    }

    protected virtual void OverrideServices(IServiceCollection services)
    {
    }

    protected virtual Dictionary<string, string> CreateInMemoryConfiguration()
    {
        return new Dictionary<string, string>();
    }
}

How is it possible that I wrote slower, more complicated, more brittle tests for years and didn’t know about this? It’s a mystery. I hope you find it useful.

Don't POOP - The Partial/Optional Object Population Anti-Pattern

Scott Hannen — Mon, 19 Apr 2021 09:20:58 +0000

The Partial/Optional Object Population (POOP) anti-pattern occurs when have a class with multiple properties, we re-use it in various parts of our application, and in different places we populate some properties and ignore others, leaving some with default values. We do this because it seems easier than creating a new class with only the properties we need. It appears to save a few seconds but leads to maintainability issues and defects.

Suppose we have these classes containing information about a product:

public class Product
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public string Description { get; set; }
    public ProductPrice Price { get; set; } 
}

public class ProductPrice
{
    public decimal Amount { get; set; }
    public string Currency { get; set; }
}

(You may see issues with this code that have nothing to do with whether objects are partially populated, but that’s realistic. Wherever we find one anti-pattern we’re likely to find others.)

We start with methods that return a Product or a collection:

Task<Product> FindProduct(Guid id);
Task<IEnumerable<Product>> FindProducts(SearchCriteria criteria);

Each Product returned from these methods has an ID, name, description, and a ProductPrice object representing the price of the product. So far, so good.

Then we encounter another scenario: Another part of our application needs the ID, name, and description, but not the price. Perhaps the price lookup consumes more resources and that method doesn’t use it, so why retrieve it?

We create a new method:

Task<Product> GetProductDisplayDetails(Guid productId);

This method returns a Product with a null Price property. What’s the harm? The method that callsGetProductDisplayDetail doesn’t use the Price property, and We know that Price isn’t populated when we get a Product from this method.

Then we have a requirement to provide promotional discounts for some products, so we modify our ProductPrice with some new properties:

public class ProductPrice
{
    public decimal Amount { get; set; }
    public string Currency { get; set; }
    public decimal DiscountPrice { get; set; }
    public string DiscountCode { get; set; }
}

…and we create a new method that returns discounted products for specific customers:

Task<Product> GetDiscountedProduct(Guid productId);

When we call this method we get a Product with a ProductPrice, and the DiscountPrice and DiscountCode properties are populated.

Now we’ve got three ways to get a Product, and which properties are or aren’t populated depends on which method we got it from.

The Problem

When we start out we know that some Product or ProductPrice properties are populated depending on which method returned them. We know that because we wrote the code five minutes ago.

Over time as our code becomes more complex we may pass these Product objects to more methods, and as we do so it becomes harder to keep track of where they came from. Developers working in other parts of the code may be surprised to find that when they perform an operation on one Product it works, but in another case they get a NullReferenceException because they expected a Price and there was none. Or there was a price and they expected a DiscountCode but there was none. Or, worse, they expected the DiscountPrice to be populated and introduced another defect because sometimes the discounted price is $0.

Chances are that if we “solve” a few problems by adding a few properties to these classes we won’t stop. The more properties the class has the more likely it that some new consumer will have a use for it, and re-using it will seem more expedient than creating a new one. That consumer may have a need to add its own properties, and the problem grows. We may end up with dozens of properties, combinations of which are populated by some code paths and not others. I’ve seen this become so confusing that developers begin to duplicate properties. Clusters of properties appear on a class and on an object contained within that class. Imagine seeing multiple DiscountPrice properties when reading code and having to figure out which one has a value, or realizing that the answer is “It depends.”

POOP also violates the Interface Segregation Principle. The varied, unrelated consumers of the class become effectively coupled to each other because a change to the class made to meet the need of one consumer potentially impacts the other consumers. That wouldn’t be a concern if they all used different classes instead of using the same one for unrelated purposes.

This is the sort of problem we learn to work around, but doing so has consequences. Developers lose the ability to look at code in a smaller context. If a method does something with a Product, we can’t understand that method in isolation. We find ourselves tracing all the code paths leading to that method to figure out where that Product came from. We can figure it out if we’re careful, but having to be careful slows us down. When we carefully add new code we add to the complexity for the next developer. They must be as careful as we were and understand the new code we’ve added as well.

This cycle may be sustainable, but it costs us. Changes take longer and longer and the chances of introducing defects increases. Instead of modifying the code to add useful functionality we’re doing so to fix defects. Fixing them takes longer and is more likely to introduce even more defects. This churn can go on for months or years

Good thing we saved a few seconds by re-using existing classes instead of creating new ones!

Solution: Invariants

An invariant is a condition that it always true. The desired invariant in this case is that the properties that should be populated must be populated. We can defineProduct and ProductPrice so that they cannot be instantiated without all of their properties and they are immutable. Then the problem of partially populated objects becomes impossible. We don’t solve it - we prevent it.

It might look like this:

public class Product
{
    public Product(Guid id, string name, string description, ProductPrice price)
    {
        if (id == Guid.Empty) 
            throw new ArgumentException($"'{nameof(id)}' cannot be an empty Guid.");
        if (string.IsNullOrEmpty(name)) 
            throw new ArgumentException($"'{nameof(name)}' cannot be null or empty.");
        if (string.IsNullOrEmpty(description)) 
            throw new ArgumentException($"'{nameof(description)}' cannot be null or empty.");
        Id = id;
        Name = name;
        Description = description;
        Price = price ?? throw new ArgumentNullException(nameof(price));
    }
    public Guid Id { get; }
    public string Name { get; }
    public string Description { get; }
    public ProductPrice Price { get; }
}

The object can’t be created without supplying all of its properties, and once it’s created those properties can’t be changed. The properties aren’t optional. I had to work that extra “O” in there to make the “POOP” acronym.

It’s more code and takes longer to write, but that effort may pay for itself many times over. Why? Because whenever a future developer encounters Product anywhere they don’t need to consider where it came from. The class definition itself tells them everything they need to know. What they think they know about it will never be wrong. The extra minute or two it took to create the type may in the long run save dozens of hours or more that might have been spent reading code over and over or fixing defects.

If we have lots of properties then applying the builder pattern may be easier than having a constructor with many arguments. C# 9 also introduces record types which make defining immutable classes easier.

Solution: Group Sets of Properties Into Classes

One of the reasons why this problem might occur in the first place is that Product has dozens of properties. If we need a class that has many of those properties and a few new ones, that’s when it seems easier to add to existing classes instead of creating new ones.

We can mitigate this by grouping properties into classes as we did with the ProductPrice class. This makes it easier to create new classes with combinations of the properties we need. Whenever we find ourselves duplicating sets of related properties across multiple classes we should consider encapsulating them within a single class.

Solution: Inheritance

I’m just kidding. Inheritance might seem like an easy solution. If we need a class with all the properties of Product and a few more, we can inherit from Product and the new class has more properties. But unless we’re careful it leads us back to the same place. We’ll need a class with some of the properties from Product, some of the properties from the new inherited class, and a few new ones. The we end up with partially populated objects plus the confusion of inheritance. The previous solution is better. Prefer composition over inheritance.

Conclusion

We’ve all found ourselves working in code that’s difficult to understand and safely modify. When that happens, can we identify individual decisions that led to that difficulty? If we can identify them then we can avoid repeating them and even begin to fix them.

Big complexity is usually the accumulation of small decisions, such as

We decided to create a class with multiple properties but didn’t think making it immutable was worth the effort.
We needed a new property some of the time, and adding it to an existing class and populating it only when needed would take less time than creating a new class.
Repeat step 2. We just need one more property.

We make these choices because individually they are small, having no apparent consequences. The effects come later. This, in my opinion, is one of the greatest challenges of software development. There are causes and effects, but we don’t connect them because the effects are deferred, and by the time we encounter them there are lots of causes mixed together. This is why our code gets out of control, we hate it, and yet given the chance to start over we make the same decisions with the same eventual results.

Once we begin to connect cause and effect we see that few decisions are truly insignificant. Realizing this could overwhelm us. How do we look into the future and see the outcome of each choice? We can’t. We can guide our decisions by applying principles and avoiding anti-patterns so that we make fewer decisions we regret and the rest are easier to change as the need arises.

Splitting Large Interfaces Into Smaller Ones Is Easy

Scott Hannen — Mon, 26 Oct 2020 04:00:00 +0000

There are so many problems with big interfaces, that is, interfaces with lots of methods.

A big interface has a big implementation, a giant class that does lots of stuff.
The Interface Segregation Principle tells us that if we have a big interface and different consumers all use different methods, using the same interface needlessly couples those consumers together. That effect isn’t obvious, but it’s real.
When we inject a big interface into another class, it’s not always obvious what we’re using it for. Which of its twenty methods do we call? It’s not hard to find out, but not having to read more code to figure that out is better.
When we have a big interface there’s often a big class that depends on it, calling lots of its methods. That class is also doing too many things, and now we’ve got two big classes (the one that depends on the big interface and the implementation of the big interface) that are tightly coupled.

Those giant interfaces tend to have names that end in “Service” or “Manager.” Sometimes it’s “Repo,” as if that’s a special case where a single huge interface makes more sense.

Cleaning that up is challenging but so rewarding. It’s technical debt that slows down anyone who has to deal with it. It also breeds more technical debt because once it’s out of control it becomes easier to pile onto it than sort it out. It feels great to grapple with that, own it, and come out on the other side with something better.

Fortunately the first step is the easiest, and that’s just splitting up the interfaces. Let’s start with a simple example.

How to Painlessly Split Up an Interface

Suppose we’ve got a big repo interface:

public interface IFooRepo  
{  
    Foo GetFoo(Guid fooId);  
    IEnumerable<Foo> GetAllFoos();  
    void SaveFoo(Foo foo);  
    void DeleteFoo(Foo foo);  
    void UpdateFoo(Foo foo);  
    IEnumerable<Foo> GetFooByName(string name);  
    IEnumerable<Foo> GetFoosByPartialNameMatch(string name);  
    IEnumerable<Foo> GetFoosByDateRange(DateTime startDate, DateTime endDate);  
    void RenameFoo(Guid fooId, string name);  
    bool IsFooPurple(Guid fooId, int howPurpleIsIt);  
}

If you’ve never seen an interface like this, good for you. It happens all the time, though. We inject IFooRepo into any class that needs to touch Foo-related data. When we have some new interaction with Foo data we add it to the interface, and eventually it gets to be like this.

In addition to all the other problems with big interfaces, another reason to split this up is CQRS (Command/Query Responsibility Segregation) which recommends that an operation should either modify data (command) or read data (query) but not both. If a class depends on IFooRepo we can’t immediately tell whether it executes commands, queries, or both. Even if we’re enforcing CQRS now, it’s easy to violate if we’re injecting a dependency that allows executing commands into a class that should only be executing queries.

How do we split this up? It’s easy. First we create two new interfaces, IFooReadRepo and IFooWriteRepo. Then we look at each method in IFooRepo and copy it into either one or the other depending on whether it reads data or modifies data. We end up with this:

public interface IFooReadRepo  
{  
    Foo GetFoo(Guid fooId);  
    IEnumerable<Foo> GetAllFoos();  
    IEnumerable<Foo> GetFooByName(string name);  
    IEnumerable<Foo> GetFoosByPartialNameMatch(string name);  
    IEnumerable<Foo> GetFoosByDateRange(DateTime startDate, DateTime endDate);  
    bool IsFooPurple(Guid fooId, int howPurpleIsIt);  
}  

public interface IFooWriteRepo  
{  
    void SaveFoo(Foo foo);  
    void DeleteFoo(Foo foo);  
    void UpdateFoo(Foo foo);  
    void RenameFoo(Guid fooId, string name);  
}

Then we delete all of the methods from IFooRepo and change it to inherit from the read and write interfaces:

public interface IFooRepo : IFooReadRepo, IFooWriteRepo  
{  
}

We can do this safely without breaking anything. Our concrete repo class still implements the same interface, and that interface still has all the same methods. It just inherits them from the two new interfaces. Our code still compiles and our tests still pass.

Next we look for any classes that depends on IFooRepo. We’ll likely find classes that depend on only read methods or only write methods. When we do, we modify those classes to depend on the smaller interface. That is, we inject IFooReadRepo instead of IFooRepo.

We’ll need to update our dependency injection. If we’re using Microsoft.Extensions.DependencyInjection we would look for

serviceCollection.AddSingleton<IFooRepo, SqlFooRepo>();

…and add, as needed:

serviceCollection.AddSingleton<IFooReadRepo, SqlFooRepo>();  
serviceCollection.AddSingleton<IFooWriteRepo, SqlFooRepo>();

We haven’t split up the implementation, just the interfaces, so we can still use the same class as the implementation for both interfaces.

If we end up without any classes that depend on IFooRepo we can delete that service registration. Why stop there? We can delete the interface, and change SqlFooRepo so that it implements the read and write interfaces instead of IFooRepo. And then we can delete IFooRepo.

These are all low-risk refactors. The compiler is checking most of it for us. If it’s beneficial we can keep going and split the repo class itself into two classes, or we can leave it the way it is. Our options are open. We could decide to repeat the process and split some of the interfaces even further. (What are the odds that one class needs to both rename a Foo and delete a Foo? Do they need to depend on the same interface?)

I Can See Clearly Now

Splitting up an interface might be the end of our refactoring, or it may identify more opportunities for improvement that were previously too hard to see. What if a class depends on both the read and write methods of our repo? We might see something interesting if we replace the dependency on the big interface with dependencies the separate, smaller interfaces.

Quite likely if a big class depended on that big repo interface and now it depends on both the read and write interfaces, we’ve identified the lines where we can break up that big class too. We’ll likely find that in that big class, some methods depend on one interface and others depend on the other interface. If that’s the case, do they really need to be in one big class? Maybe we can split that class up too. What if that class implements an interface that other classes depend on? We can split up that interface exactly the way we split up the repo interface. It’s a safe refactor, and when that step is done we can split up that class, too.

Now we’re turning the tide. Our big, out-of-control classes are getting smaller because the first refactor made it easier to see through the fog and figure out where to refactor next.

Static code analyzers like NDepend can also spot some of these issues for us. If NDepend identifies classes where some methods interact with some fields and others with other fields. Those fields are likely dependencies we’ve injected, or they could be variables for some other use. NDepend will point out that this lacks cohesion. If a class has some methods that use some fields and other methods that use other fields, does it need to be one class? Maybe not.

Why Do These Big Interfaces Happen in the First Place?

I think the underlying reason why big interfaces and their big implementations happen is that we don’t design interfaces with a consumer in mind. That is, we don’t say

The class I’m writing needs to depend on something that does X. Therefore I’ll define an abstraction (likely an interface) that represents that need.

If we do that, our interfaces are small and have specific names. It’s hard for them to become bloated because some new behavior doesn’t fit the name of the existing interface. When we need something different we define a different abstraction. This tends to lead to small interfaces. Many might have only one method. We might realize that we can use delegates or functions instead of interfaces.

Instead, we tend to say,

This class I’m writing needs to depend on something that does X. All of our X-related stuff goes in this interface, IAllThingsXManager, so I’ll add a new method there. It already has twenty-two methods. What’s one more?

Rather than creating interfaces as specific abstractions, we create them as groupings of methods that deal with some related data. We might have customer data and pile anything and everything vaguely related to interacting with it into an ICustomerService or ICustomerManager interface. Once that happens, it snowballs. Anything vaguely related to customer data gets added.

Do we perceive adding methods to classes and interfaces as preferable to creating new classes and new interfaces? I think sometimes we do. It’s a habit to unlearn. It costs nothing to add new abstractions and new classes to fulfill new requirements.

Ask yourself this: How many times have you had a giant class with 500 or 2,000 lines and wished it was smaller? How many times have you had a set of small, specifically-named classes and wished you could refactor them into a single giant one? I’d bet the first one happens all the time and the second one never. So why not write the code we want and stop writing the code we don’t want?

Conclusion

Hopefully I’ve made a case for avoiding big interfaces in the first place. That’s easier than splitting them up. Interfaces that can be split up but aren’t create technical debt. Our code works and we can understand it, but that understanding takes just a little longer than it should. We end up with larger classes that do too many unrelated things. Those small increments of time that we spend reading and figuring out large interfaces and classes add up. Worse, one day we have to make a significant change and we realize that it’s become very difficult to figure out how to do it.

But too often we have no control over that. We might start a project long after it’s gotten unwieldly, or maybe it’s our own mess and we’re just now realizing that it’s hard to work with. Either way, we don’t have to live with those past choices. Splitting up interfaces is one of the easiest ways to improve our code, and it often blows away some fog and makes it easier to see what else we can refactor.

Suggestions for Overcoming Impostor Syndrome and Anxiety

Scott Hannen — Mon, 19 Oct 2020 04:00:00 +0000

Wikipedia defines impostor syndrome as

a psychological pattern in which an individual doubts their skills, talents or accomplishments and has a persistent internalized fear of being exposed as a “fraud”.

I’m occasionally insecure about my skills as a software developer, but haven’t had a problem with impostor syndrome specifically. Why not? Perhaps my answers as well as a few other thoughts will help others who experience that anxiety.

I Am What I Am, We Are What We Are

I try to assess myself realistically and never, ever represent myself as having skills or experience I don’t or knowing more than I do. This is reflected on my resumes, where I divide my skills into three categories:

Primary - this is what I really know well.
Secondary - I’m not an expert, but if I have to work with this I’m not starting from zero.
Other - I’ve worked with this. I might have forgotten everything, but I’m sure I could do it again.

(An upside is that I can list lots of varied tools and technologies under “Other” that would seem out-of-place

in a non-categorized list of skills.)

Perhaps Angular is on my list of secondary skills. If someone sees that along with the rest of my resume, talks to me, and concludes that I can work on their Angular project, what do I have to worry about? I’ve set the right expectation. That’s two-way trust. They trust me because I’m honest. I trust that they know how to evaluate me, and I trust that

they’ll be okay if I’m not an instant Angular expert. That trust is invaluable.

This also takes all the stress out of interviews. I don’t study trivia questions before interviews. Ask me what you want to ask and I’ll try to answer. Ask me weird algorithm questions and I’ll tell you I don’t know the answers. If I ever need to know them I’ll look them up. If that’s good enough, great. If not, them I’m not the right person and you shouldn’t hire me. Ask me increasingly difficult questions about what I do know, and you’ll find where my knowledge ends.

Wit that mindset the dichotomy of skilled developer vs. impostor disappears. I’m just a person who knows some things and doesn’t know others. Hopefully what I have learned demonstrates what I can learn. (The list of things I don’t know even though they’re commonplace is massive, and I might get stressed if I thought about it, so I don’t.)

I’m sure this is harder for developers finding their first jobs. I moved into development internally within a company so I’ve never had that experience. I hear it’s awful, and I’m sorry. The industry needs you, and people know that we need you. We’re just really bad at cultivating your skills. I think everyone figures someone else will do it, and then they can hire you away after you get some experience. My advice: Play that game. Take any first job you can get however you can get it. If it’s great, stick around. If it’s an awful place to work, use them to get some experience and then move on. It gets so much better.

Replace Unhealthy, Negative Thoughts

We may have a tendency to put ourselves down. Maybe we acquired it early in life, even in childhood. We think that demeaning ourselves proves that we don’t have a big ego, and in turn no one else will have

cause to burst our bubble.

Or we might fear the Dunning-Kruger effect, which says that skilled people think they are unskilled and vice versa. (It doesn’t actually say that, but that’s the version we may have learned.) We worry that if we start to think of ourselves as skilled and experienced them the ghosts of Dunning and Kruger will curse us, poking us with pitchforks and putting defects in our code to humiliate us. (I don’t believe in ghosts and last I checked both Dunning and Kruger are alive, but it’s a fun mental picture.)

Either way, this goes back to the false dichotomy of expert vs. impostor. We don’t benefit by judging ourselves as good or bad. It means that unless we think very highly of ourselves (which might itself be a problem) we’ll see ourselves as deficient.

That’s not a healthy way to think about our skills or really anything. If we think bad thoughts about things we can’t control, we’re unhappy. If we think, “I’m a bad software developer,” or, “I’m an impostor and I’m going to get fired,” we’re not framing our concerns in terms of things we can control. What is a bad software developer and how do we go from being a bad one to a good one? I don’t know. Is a software developer really either good or bad? How do we evaluate that? We’re putting a label on ourselves with no way to take it off.

Let’s take a negative thought and reframe it, and see how something negative we can’t control becomes something neither negative nor positive, and likely something we do control.

Instead of

I’m a bad developer! I don’t know how to do anything.

Think

I don’t know how to do any front-end development. A good developer would know that.

That’s still wrong, but now that we’re putting our thoughts into more specific words we can evaluate them rationally. Is it actually true that we don’t know how to do any front-end development? Probably

not. We’re still judging what’s “good” and “bad” and we’re still describing something with no apparent way to control it.

So let’s try it one more time, more specific, less judgmental:

I don’t know how to use any front-end JavaScript frameworks. I wish I knew how to use at least one.

What just happened? First, the negativity is gone. We’re making a statement which we can rationally evaluate as true or false. Second, we can ask, is this something I control? If the answer is no then what’s the point in worrying about it? But in this case we can change it. We can decide to learn Angular or React or whatever. So now instead of a negative thought we’ve got something that we can choose to do or not do.

Changing our thoughts take practice. We probably won’t say this but we might think it:

Learning Angular will be hard and I won’t be able to do it.

Let’s evaluate that. Why would it be true? Have you ever learned anything, ever? If so, what is different about this? Are there only one or two geniuses in the world who understand Angular? No, thousands upon thousands of people have learned it. It might take some effort, but what evidence supports the idea that all those people can do something and you can’t? None. It’s not rational.

One more:

Okay, I’ll learn it, but I won’t be good enough at it.

Again, let’s evaluate it and see if it’s true. What happened the last time you learned Angular? Were you good at it? There was no last time. So what evidence do you have that you won’t be good at it? What is “good enough?” Thinking that we won’t be good at it is a choice. We’re not required to think that, are we? So if it’s up to us, why not think this instead:

Okay, I’ll learn it, and I’m going to be good at it!

Are you excited now? I am! Seriously, I am. That’s what positive thought does.

Something else changes when we replace good-or-bad judgements with factual thoughts. If we decide that we’re a bad developer or impostor, how do the skills we do have and the things we do know fit into that? They don’t. That all-or-nothing judgment allows only good or bad. It becomes confirmation bias, causing us to disregard evidence which doesn’t support our negative thoughts. It’s impossible to think

I’m a good developer. I’ve got some experience with C# and SQL. I’m a bad developer. I don’t know any front-end JavaScript frameworks and they’re too hard for me to learn.

How can we think both at the same time? We can’t. They’re contradictory. But this is easy:

I’ve got some experience with C# and SQL. I don’t know any front-end JavaScript frameworks. I’ll learn one.

What has changed? Absolutely nothing except our thoughts. We’re not brainwashing ourselves or telling ourselves lies. We’re thinking more rationally with less judgment and making a deliberate choice to think more about things we control.

Developing this habit is like physical exercise. We start out-of-shape and improve with practice. We might let it slide for a while, notice it, and start working at it again. It’s not all-or-nothing. We don’t get all the results right away, but we get some.

Don’t Internalize Other Peoples’ Unhealthy, Negative Thoughts

I suspect that a lot of Impostor Syndrome comes from internalized abuse, typically from managers. Sometimes they’re negative or outright hostile, and because they have the power to act as though what they think is true, we internalize it. We believe them, and their thoughts become our thoughts.

Rather than internalizing what they say, it’s up to us to think it through rationally and decide what makes sense, what doesn’t, what we can control, and what we can’t. This allows us to absorb constructive feedback, reject irrational negativity, and sometimes find a middle ground.

Let’s analyze some things a manager might say and consider how we might respond either to them or internally to ourselves. You’ll notice that the way we analyze what they say to us resembles the way we analyze what we say to ourselves:

Your latest pull request introduced a defect. Let me show you how to write a unit test that would have prevented it.

Our response (internal and external) is:

I don’t like that I introduced a defect. Fortunately I’m about to learn something that will help me in the future. Awesome!

What if your manager says,

Your code never works, and I’ve had enough of it. Get better or else!

Is it true that your code never works? Probably not. If you’re having this conversation, apparently something didn’t work, and that’s nothing to be happy about. Should we learn to deflect this and not care whether our code works or not? Of course we shouldn’t!

We might think:

Oh, no! This has happened before and it happened again! I try so hard to be careful but it’s never enough. I’m going to be fired!

If we think that we’re telling ourselves that we have no control over anything, so we’re just going to wait and see what happens to us, and it’s going to be bad. That sucks. There’s nowhere to go from there. How about this instead:

I write code with defects more than I want to. I want to write code with fewer defects, despite trying to be careful I still create defects. I want to change this but I don’t know how.

That’s accurate and honest. Why not ask your manager, “It’s really important to me to learn how to write code with less defects.” Shouldn’t they be able to help you with that?

Or ask another developer. Ask a question on Quora. Ask people on Twitter. You’re not the first person to have this problem. Trust me, you’ll find excellent help.

The Hard Truth About Abusive Managers

Here’s the hard truth about those abusive managers. I hope I don’t harp too hard on this but it’s important.

If your manager is berating and threatening you because of defects or missed deadlines but isn’t helping you to improve, it’s a clear indication that they have no idea what you could do differently. What other explanation is there? They know but they don’t want to tell you?

Let me repeat for emphasis: If they’re yelling at your or threatening your job, they don’t know how to help you. Of they did they help you instead of yelling. It’s that simple. That means even if you ask them how to improve and they tell you, chances are that whatever they tell you is useless. Listen for platitudes like “Be more careful,” or “Spend more time making sure it works”. In other words, listen for exactly what you would say if you had no idea what you were talking about.

Some managers don’t get visibly upset. They just calmly threaten to put you on a “performance improvement plan.” The same principle applies. They have a limited skillset which includes threats and paperwork. They’re hoping that one or the other will get them a different result. And if that doesn’t work they hope the employee they can’t help will leave and they’ll have better luck with the next one.

If you’re a new developer and your manager yells, threatens, or calmly tells you that you’re going on a so-called “performance improvement plan,” all without giving you meaningful help to improve, they are not withholding that help. They just don’t have it to give. Their skillset consists of abuse and paperwork.

Please understand that this is not a rare phenomenon. Some people become software development managers without knowing much about it. They may be former software developers who had managers much like they are now, which means that they survived but learned little because no one taught them. As far as they know the best you can hope for is to survive for a while and become like them.

If you work in a place where this goes on, there are almost certainly huge issues in the development process and the code itself that make it difficult for anyone to work in. Even a more experienced developer would have a hard time. The difference is that the more experienced developer knows why it’s so difficult. If you’re less experienced you might believe someone who tells you it’s all your fault.

What should you do with that information? I don’t know. This article is more about what to think and what not to think. Your first job is probably the hardest one to get. Set aside the question of whether your manager is unpleasant. Are they helping you? Do you feel like you’re growing and learning? If not, don’t wait forever for it to change. Make decisions that align with your goals. Personally I recommend looking into companies that specialize in software development or consulting. Nothing is perfect, but it’s almost certainly a step up. (Even if they don’t hire you they can provide some insight into how you can grow your skills.)

Again, the point is not to stop caring or point the finger at anyone who tells you something you don’t like. I’m not trying to tell you how not to listen to anyone. Listen if they’re telling you how to improve. But if they’re just telling you negative things about yourself, understand that it’s them, not you.

Does this make anxiety regarding your job go away? No. But if you understand which things you control - learning, asking the right people for help, maybe getting another job - then you can focus more on that. Try your best to tune out the rest.

Conclusion

I may not suffer from Impostor Syndrome, but I have dealt with crippling anxiety regarding work and my ability to succeed as a developer and provide for my family. (Maybe that has something to do with my logo.) This article isn’t some random musing after reading a self-help book. This is what got me through it, which is why I’m sharing it.

When I’m worried about whether I can learn something or succeed in my next role, here’s what I tell myself:

Other people know this. Other people do this. I refuse to believe that the world is full of people who can all do something I can’t. If they can, I can.

I still doubt myself all the time, but this affirmation proves true over and over. It’s reasonable and supported by evidence.

The same applies to you (and I don’t say things I don’t mean to be nice.) Software developers are just people who learn how to do a job. Some are more determined to learn it well, and I hope you and I are among them, but the bar isn’t so high that only rocket scientists can get work.

You will hit some bumps. You’ll deal with some unpleasant people. Sometimes it will be so difficult that you’ll question yourself. But you can succeed. Don’t listen to anyone who tells you otherwise, especially not yourself.

Consistency is Our Friend and Enemy

Scott Hannen — Wed, 14 Oct 2020 04:00:00 +0000

Have we heard this argument when discussing disagreements involving code?

We need to make new code consistent with existing code or new developers will be confused.

The purpose of this blog post is not to dispute the consistency argument. Just the opposite - first I’ll argue for it. Rather my intent is to drain some power from it, to put some speed bumps in front of it. When the consistency angel lands on one shoulder and tells us that this code must be the same as that code, let this other voice ask, “The same how? Why? Have we thought this through?” It’s not even a devil; It’s another angel with a different perspective, and often agrees with the other one.

First let’s look at why consistency is beneficial when applied thoughtfully. This will set up a contrast. Consistency is good when it provides these benefits. It’s questionable if it does not provide these benefits and harmful if it works against them.

Good Consistency

Consistency Reduces Cognitive Load

Looking at unfamiliar code is inherently confusing. We’re simultaneously trying to grasp what code does, why, in what context, and often whether it works as expected. Needless inconsistency adds to that cognitive load. If we take that initial confusion and add to it variables that are prefixed (or not) inconsistently or the indentation seems random, it becomes harder for our minds to process the more important details.

We avoid that by establishing conventions. In C# some code bases prefix field names with an underscore. Others don’t. Which is better? It matters less than whether we pick one and apply it everywhere.

Consistency Helps Someone Who Understands One Part of Our Code to Understand Others

Suppose we choose a certain library to fulfill a need: Let’s say we choose Autofac as our IoC container. If we use it consistently throughout our code base then once a developer understands how it works they’ll understand it more easily throughout the code. Sure, they can understand more than one, but why not make it easier for them? We wouldn’t introduce a different one without compelling reason.

If we see a design pattern employed across multiple similar, related classes then once we understand it in one place we’ll understand more quickly what’s going on in another. We should not, however, assume that developers are so limited that similarity between one area and another is the only way they can understand how code works.

Consistency Narrows Our Choices and the Remaining Ones Are Good

What if a design pattern is employed across similar, related classes and now we need to create a new one? We could approach it as if the other classes don’t exist, and we’d have lots of choices in front of us. The decision might take longer and we’d make it with less certainty. If we aim for consistency we can often eliminate most choices and follow a pattern found in existing code.

If we need to choose a library to perform some purpose, that decision is easier if we’re already using a library for that. In that case we just keep using the same one.

In both cases consistency helps us to make choices more quickly by removing choices or strongly recommending others. Fewer choices might sound bad, but especially when starting out in unfamiliar code it’s reassuring to know that we’re doing something the expected way. Decision paralysis can waste time. The code that results from following a pattern will likely be good enough. (likely, not certainly.)

Let’s contrast these benefits with happens when the drive toward consistency goes wrong.

Harmful Consistency

Will New Developers Really Be Confused?

We’ve probably worked in code bases where something weird is done over and over, or something that clearly violates the principles we follow to write the best code we can. I don’t want to use a real example, so I’m going to make one up. It’s no less strange than the real examples I’m not using.

Imagine we have an area of our application in which each class contains an identical set of unused private fields and private methods. It’s obvious that each new class is created by copying and pasting an existing one. The majority of the code in each class is not used. The methods that are used have weird names like Invert and Enrich, and you realize that those names have no connection with what those methods do.

Figuring out what’s going on in these classes wastes some of your time. So when you create your own, you leave out the parts that aren’t used, and you help out the next person by giving the methods meaningful, descriptive names.

But then comes the objection: “Consistency is very important. All of the other classes have these methods and variables, and the names are all the same. If you make this one different… (here it comes!) … new developers will be confused.”

Quite often when I hear that argument I am the most recent developer to join the team. No one ever asks me what confuses me. They tell me what confuses me, and they’re usually wrong.

Yes, it is possible to confuse new developers. It’s certainly possible to confuse me. But I’ve never looked at code and thought, Hmm. They did this odd thing for no apparent reason in ten other places but didn’t do it here. Therefore this code is confusing. If they did the same odd thing here also I would not be confused. Is that plausible?

If a developer found that confusing, how would they cope if we modified one class to accommodate new requirements? Would we add it to all the others just for that one person? Would it actually help them? How?

Unused code is just an example, but the same reasoning applies to other scenarios, whether it’s naming, applying design patterns, or something else. We’ll be consistent when we can because it does make code easier to understand. But any developer understands that two classes or modules that exist for different reasons will differ from one another.

Don’t just say, “New developers will be confused.” Reason on it. Sometimes we’ll find that it makes no sense.

Sometimes There Is No Pattern

Sometimes we see an coincidental pattern in the code and then expect new code to follow it. For example, we might have several classes that perform similar functions. At some point the requirements for one class change, one if its private methods becomes more complex, and we extract it in to separate service. We didn’t do the same with the other classes, so now in that respect this one is different from the others.

We could reason that we shouldn’t refactor the class because then it won’t be like the others. But that doesn’t hold up under scrutiny. The two classes were already different. They had different names, and their methods did different things. Otherwise why are they separate classes? Don’t enforce imaginary consistency.

One or two of anything is not a pattern. Don’t be that person who writes twenty lines of code or a few classes and then tells everyone else to write theirs the same way because it’s “the pattern.” Please don’t.

Don’t Confuse Patterns With Design Patterns

Design patterns are beneficial when used correctly. Sometimes developers justify their peculiar consistency by calling it a “pattern,” perhaps hoping that the goodness of properly applied design patterns will rub off on whatever they’re doing (and want others to do.) It won’t.

This confusion is compounded when the pattern consists of misapplying a design pattern, often using it for no apparent reason. (Think singletons everywhere or “adapters” that adapt interfaces to deliberately identical interfaces.) Now we’ve got double pattern! Or maybe even pattern squared! Pattern is a neutral word. If something is a pattern or we call it one, that doesn’t make it good or bad. Knitters follow patterns. So do serial killers.

Consistency Is Coupling

Coupling occurs when changes to one part of our code affect other parts of our code, often forcing us to change one part because another part changed. A good deal of what we learn as developers is how to avoid coupling. Why? Because it makes code harder to modify. Small changes ripple across coupled code and become larger changes. As the size of the change increases so does the risk of introducing defects.

Now think about what happens when we have multiple classes and determine that they should have characteristics in common for reasons that have nothing to do with the purpose or behavior of any of them. We’ve artificially coupled them together. We may have carefully designed each to minimize coupling, but now we’ve done the opposite. The effect could be minor and harmless, or we might have to choose between writing “bad” code to satisfy that coupling or modifying unrelated code so that our new code is easier to write.

We may weigh the cost of that coupling and decide that it’s worth it for the sake of consistency. It often is. It’s less harmful than other forms of coupling because it exists only until we decide it doesn’t. But the cost is real. Consistency is coupling. I’ve said it twice in the hope that when someone hears the one word in this context they’ll think the other.

How to Detect Good Consistency and Avoid Harmful Consistency

How do we test whether consistency makes sense in a particular case? Here’s are a few suggestions:

Whatever we think should be consistent, try to express it in words. For example,

Do not prefix field names with underscores.
We use __ library to do __. Let’s keep using it to do that unless we need something it can’t handle instead of using two or more libraries.
All classes in this namespace which implement ISomeInterface should have private methods named Invert and Enrich.

This encourages us to reason on

whether or not a pattern exists
exactly what that pattern is
whether it’s intentional or coincidental
whether it makes sense to perpetuate it

If needed, discuss other factors:

Is the alternative to consistency based on concerns about an existing pattern or is it matter of personal preference? If there’s not a good reason to deviate then we likely shouldn’t.
Has the pattern created technical debt, and does repeating it add more technical debt?
Would an alternative be easier to write, understand, test, and maintain? Why? How?

The benefits of consistency might outweigh other factors. Sometimes it makes sense to repeat something sub-optimal to avoid inconsistency. But the point is that we’re deciding to be consistent or inconsistent. We should not be subservient to our past decisions or those of others.

If we choose consistency then perhaps we can document it along with our reasoning. (After all, we’ve expressed it in words.) Perhaps place a ReadMe file in the source code or note it in a Wiki. Even then it should remain open to change. The point is that if we’ve reasoned on it and made a decision, we don’t have to have the same discussion often. And we’ll always remember why we’re doing what we’re doing.

Expressing the reasons for our decisions is powerful. We can document and discuss it, accept it or challenge it. Sometimes we reason about consistency, but sometimes we use it as magic word that ends reason and discussion. Let us view consistency as a principle on which to reason and not as a mere invocation.

Conclusion

If I’ve convinced anyone to be thoughtful about how and why they prioritize code consistency then my work here is done. Consistency is beneficial as a principle, but harmful when we use it as a reason to disregard other principles.

Agile methodology and common sense tell us that we should look at what we’ve done in the past, do more of what works, and stop doing what doesn’t. Apply that to consistency. Where does it help? Ask your new team members what helps them and what slows them down. If something helps, keep doing it. If what hinders them or you is something done consistently, stop doing it.