DEV Community: Anton Sizikov

Show off your DevOps efforts on GitHub

Anton Sizikov — Tue, 16 Mar 2021 20:49:22 +0000

In the era of DevOps some of us write yaml-files on a daily basis. Sometimes it's just a lot of yamls. I would say that in some repositories YAML is more than just 'a data-file', it's a valuable artifact where the core (or just important) logic resides.

However GitHub doesn't think so. You know that there is a Language Breakdown Graph on every GitHub repository:

It looks cool, it highlights the languages used in this repo.
However, your DevOps efforts are not represented on this chart. No Kubernetes or Docker-Compose manifests and Charts, no CI/CD pipelines, no GitHub Actions, and other sometimes tricky configurations.

Luckily this is something we can fix.

GitHub Linguist

GitHub uses a library called Linguist (https://github.com/github/linguist) to analyze your repository. Of course, the Language Breakdown Graph is more of a side-effect and definitely not the main purpose of this analysis. But for now we'll focus on this particular feature of Linguist.

All languages known to Linguist can be found in the languages.yml file in the linguist repository.

Let's check YAML definition

YAML:
  type: data
  color: "#cb171e"
  tm_scope: source.yaml

as you can see, it's treated as a data file by default and thus it's left out from the graph.

Overriding Linguist behavior

We can add (or edit existing) .gitattributes file with the following content to the root of our repository:

*.yml linguist-detectable=true
*.yml linguist-language=YAML
*.yaml linguist-detectable=true
*.yaml linguist-language=YAML

and Linguist will respect that override, by showing us a proper Language Breakdown on your page:

Isn't it cool?

There is more

It's not just the way YAML files are treated. You can fine-tune that to emphasize the work which is important in your opinion.

Do you have a repository full of Markdown files, perhaps a book that you're writing? Well, Markdown can be marked as linguist-detectable as well.

Do you have some samples or generated files that you want to exclude? linguist-generated and linguist-documentation attributes can be used.

You can go wild and mess up with it:

# Example of a `.gitattributes` file 
# which reclassifies `.rb` files as Java:
*.rb linguist-language=Java

Additional documentation and examples can be found here.

Call to Action

I hope you enjoyed this little trick. Now go and celebrate your DevOps work by including it in your repository stats.

Configure CD for Azure WebApp with Terraform Provider for Octopus Deploy

Anton Sizikov — Thu, 28 Jan 2021 20:29:41 +0000

In this post, I'm going to configure the continuous delivery process for Azure WebApp (Azure Function in this case, but that's pretty much the same) with Octopus Deploy. To make it a little bit interesting I'm going to use Configuration-as-Code approach with a brand new Octopus provider for Terraform.

Buckle up and let's get started...

Tools

Terraform

I'm going to need terraform on my machine:

Octopus Deploy

For this post, I've signed up for a free tier of Octopus Deploy SaaS offering. Of course, the self-hosted version will work as well.

When signed up and configured my account I can generate a new API key that I'm going to use with Terraform

All configuration is going to be made against the default Space (Spaces-1), however, that's configurable too.

Azure

To connect Octopus Deploy with Azure I'm going to need a Service Principal.

# (!) make sure to follow the principle of least privilege here 
# and define the role and scope
az ad sp create-for-rbac --name sp-octopus-deploy

Grap the tenant Id, application Id, password, and subscription Id values. We'll heed them later.

Azure Function

For the demo, I'm going to use this super simple Azure Function that replies to your GET request with a list of headers supplied.

The complete source code is published here.

the .csx gist is the following

Azure DevOps

I'm running the CI part in Azure DevOps.

Project configuration is pretty simple and it's terraform set up can be found here.

the pipeline itself it here.

Nothing complicated:

Build Azure Function
Pack Azure DevOps artifact (zip)
Upload it to Octopus Deploy's built-in feed
Create and trigger Octopus Deploy release

I'm still waiting for the fix to my bug-report to be delivered, without it I cannot make this demo end-to-end :(

Init

Terraform provider is no different from other providers and can be found on Terraform Registry

and terraform init will download missing files

Configuration

For the demo, I'm going to have this terraform.tfvars file to provide configuration values to Terraform.

Configure Environment

Now we're ready to start. At first, I'm going to configure a new Environment.

Environments are how you organize your deployment targets (whether on-premises servers or cloud services) into groups that represent the different stages of your deployment pipeline, for instance, development, test, and production.

You'd probably have more than one, but that'll do for now.

Deployment Target

The next crucial part is Deployment Target

With Octopus Deploy, you can deploy software to Windows servers, Linux servers, Microsoft Azure, AWS, Kubernetes clusters, cloud regions, or an offline package drop. Regardless of where you're deploying your software, these machines and services are known as your deployment targets.

I'm using octopusdeploy_azure_web_app_deployment_target, It's quite specific. I could have used a more generic octopusdeploy_deployment_target instead.

This is where I link Azure (my Service Principal) with Octopus.

Now we have a nice and clean infrastructure defined in Octopus.

Project

The next step is to configure our Project.

Projects let you create and manage your deployment processes, releases, and runbooks from the Octopus REST API and Octopus Web Portal.

I'm setting up an Echo Api project here, placing it into Default Project Group with the default lifecycle.

Deployment process

And now the juicy part.

My deployment is super simple actually, just one step that is picking up the package from a built-in feed and pushing it to my only environment.

I wish all deployments were that simple...

Release

Initially, I wanted to automate the release creation and trigger it via my Azure DevOps pipeline, but due to a very recent bug, I'm blocked here. I'll update this post when things are fixed so for now just a manual process.

I'm creating a new release from the zip file in my built-in package feed

And now I can sit back and watch it deploying my function.

Voilà, it's up and running.

Import

The neat part about Octopus Deploy is its resource ID's system. It's always easy to find the id of the object which makes it super easy to import existing resources into your terraform state.

Let's assume that I already have an environment configured.

https://cloud-eng.octopus.app/app#/Spaces-1/infrastructure/machines?environmentIds=Environments-7

The URL looks this way. Environments-7 is the id of my environment.

All it takes to import the environment is to declare the resource and type one command:

this is such a breath comparing to long and not so easy to get Ids in Azure :)

this is a cross-post from my personal blog.

Using GitHub Container Registry with Kubernetes

Anton Sizikov — Thu, 08 Oct 2020 12:15:10 +0000

GitHub Container Registry was introduced on the 1st of September 2020. It's still in the Beta stage, so it's rather not recommended to use it in production. However, it offers us free private storage for our Docker images, at least until the end of the Beta period.

Private storage, free and unlimited download... looks like a good enough option for local development.

In this post, I'm going to configure my local Kubernetes cluster to pull images from my GitHub Container Registry.

Enable GitHub Container Registry

First things first. We need to enable GitHub Container Registry for our account (Process for your organization is almost the same).

Note GitHub Container Registry is in public beta state at the moment of writing, so it's subject to change. I recommend to follow the official documentation, it's more likely to be up to date.

Create Access Tokens

For now, Container Registry supports just one authorization option, which is Personal Access Token (PAT). I'm going to create two tokens here (note the scopes).

One token with read:packages scope will be used for my local Kubernetes cluster to perform pull actions. And another token I'll use to manage images in my registry (remember Authentication step from GitHub Packages configuration page?).

.dockerconfigjson

Once I have my PAT token created I can build an auth string using the following format:

username:123123adsfasdf123123 where username is my GitHub username and 123123adsfasdf123123 is the token with read:packages scope.

let's Base64 encode it first:



echo -n "username:123123adsfasdf123123" | base64
dXNlcm5hbWU6MTIzMTIzYWRzZmFzZGYxMjMxMjM=

With this string I can compose a new .dockerconfigjson:



{
    "auths":
    {
        "ghcr.io":
            {
                "auth":"dXNlcm5hbWU6MTIzMTIzYWRzZmFzZGYxMjMxMjM="
            }
    }
}

Which we can Base64 encode again:



echo -n  '{"auths":{"ghcr.io":{"auth":"dXNlcm5hbWU6MTIzMTIzYWRzZmFzZGYxMjMxMjM="}}}' | base64
eyJhdXRocyI6eyJnaGNyLmlvIjp7ImF1dGgiOiJkWE5sY201aGJXVTZNVEl6TVRJellXUnpabUZ6WkdZeE1qTXhNak09In19fQ==

and store it at as dockerconfigjson.yaml file.

Note: base64 is not an encryption, so you should not commit this file! This is just for a demo.

And now we have prepared all the pieces, time to create a new secret



kubectl create -f dockerconfigjson.yaml
secret/dockerconfigjson-github-com created

Which can be used later on when we schedule our pods:

And the final step is to check Pod Events (I'm using Lens Kubernetes IDE here).

Have fun, and enjoy it while it's free!

This is a cross post from my personal blog.

Fixing console logs for Azure Functions running in a Docker container

Anton Sizikov — Sat, 11 Jul 2020 15:54:06 +0000

Local development of C# Azure Functions on macOS is still a bit painful.
Even the simple-ish logging might cause issues. Let's assume that we have Azure Functions Core Tools installed and we have a basic function app with one TimerTrigger function created.

With the default Run/Debug configuration

We can compile and run our function

Our function will start and log to console as expected.

I wish all the functions were that simple, right? Unfortunately, sometimes we need more, sometimes we even have to use some shared libraries. So let's create one:

Nothing fancy there, one class, one interface accepting the ILogger<T> as a dependency.

Let's update function code accordingly: will make is non-static, and use construction injection to get the IAsyncGreeter injected:

In order to register this external dependency and use a propper .NET Core-style DI we need to install Microsoft.Azure.Functions.Extensions NuGet package and create a Startup class like this one:

You would expect it to work, right?

I wish...

As you can see it still just logs from the function.

Why would one logger print it's output to console while another one just flushes our logs to void?

Because they are two different loggers with different settings. One logger is instantiated and configured in our Startup.cs class and another one is built and configured by function host and injected into the method instead. Oh, well.

I want to run this function in docker, so let's create a Dockerfile.

Note that I'm setting AzureFunctionsJobHost__Logging__Console__IsEnabled environment variable here.

The missing link here is the hosts.json logging configuration.

    "logLevel": {
        "default": "Information"
    }

And voilà, we've got logs in our container:

I hope that would save you some time. Happy logging!

This is a cross post from my personal blog.

Provision your Azure environment with Terraform and Octopus Deploy

Anton Sizikov — Sun, 22 Mar 2020 09:58:18 +0000

My current project has reached the point where we have to manage our infrastructure in a more organized way rather than ad-hoc manual configurations.

Current landscape

We use a pretty standard setup for a .NET web application.

We keep our code in one monorepo hosted in internal GitLab. We use TeamCity to build our code and we use Octopus Deploy to run our deployments to Azure.

We have several environments (let's call them feature, test, and prod) and they all are different. Each one has a different configuration, different accounts, different topology, different numbers of replicas and shards, you name it.

Things like connection strings, account and passwords are managed by Octopus.

At this point, we would like to utilize the configuration management feature provider by Octopus, and that's why we would have to jump through several hoops to marry terraform configs and Octopus variable substitution mechanism.

Scope of the problem

We'd like to move to an Infrastructure-as-a-Code approach and provisioning our system automatically as a part of our CI/CD pipeline.

To do that we're going to add Terraform to the game.

At the time I'm writing this post the versions for the software are the following:

TeamCity 2019.2.2
Octopus Deploy v2019.12.0 LTS
Terraform v0.12.24
azurerm provider 2.2.0

Repository Setup

This part is not very important, but to provide you with the full picture I'll put it here.

We keep our Terraform files in the same repository alongside the application code and database schemas. Basically, it's just an \infrastructure directory in the root of the repo.

Terraform Setup

We run our application in Azure, hence the choice of Terraform backend and a provider: we keep Terraform state in Azure.

Please refer to the official documentation for a detailed step by step guide on setting up storage for Terraform State.

In short, you'd need to run the following:

#!/bin/bash

RESOURCE_GROUP_NAME=terraformstate-rg
STORAGE_ACCOUNT_NAME=terraformstateci
CONTAINER_NAME=terraformstate

# Create resource group
az group create --name $RESOURCE_GROUP_NAME --location eastus

# Create storage account
az storage account create --resource-group $RESOURCE_GROUP_NAME --name $STORAGE_ACCOUNT_NAME --sku Standard_LRS --encryption-services blob

# Get storage account key
ACCOUNT_KEY=$(az storage account keys list --resource-group $RESOURCE_GROUP_NAME --account-name $STORAGE_ACCOUNT_NAME --query [0].value -o tsv)

# Create blob container
az storage container create --name $CONTAINER_NAME --account-name $STORAGE_ACCOUNT_NAME --account-key $ACCOUNT_KEY

echo "storage_account_name: $STORAGE_ACCOUNT_NAME"
echo "container_name: $CONTAINER_NAME"
echo "access_key: $ACCOUNT_KEY"

Once you've successfully created a storage for your state you can update your main.tf file with the backend configuration:

For the purpose of this blog post I'll be creating just a new resource group with one SQL server.

Let's declare our setup in a new sql.tf file:

note that I'm using variables in a gist above.

Variables are defined in a variables.tf file:

variable "sql_server_admin_pwd" {
  type    = string
}

This works fine, but for the local testing I keep the terraform.tfvars files with the values:

resource_group_name  = "test_rg"
sql_server_admin_name = "admin_name"
sql_server_admin_pwd = "P@$$w0rd1!" # dummy pwd used for local tests

values are not encrypted, but they are just for the local tests and experiments. They will be replaced with the environment-specific vars by Octopus later.

TeamCity Setup

Octopus has a specific syntax for variable substitution. Usually, it looks for #{Variable} markers in your config files. As you can imagine they are not always valid as resource names.

As I mentioned before our terraform.tfvars has realistic values for local tests. It helps our infrastructure engineers verifying their work by running terraform plan commands.

However, when we build our packages we want to convert tfvars file to replace variable's values with the syntax suitable for Octopus.

I had to invent a convention here.

Every variable declaration var_name = "abc123" will be converted to a var_name = "#{var_name}" line.

To do that we've set up a build step with the following PowerShell script:

After that we can build our code and then create a release package with our Terraform config.

A TeamCity build configuration has a custom %octopus.package.suffix% variable declared. This is a suffix that is composed of the branch name. It's empty for the master branch build and has something like feature-AB-123 for a feature branch with an AB-123 name. Yes, this is a JIRA ticket name :)

With this variable we can prepare a zipped artifacts file:

+:infrastructure/** => Terraform.%build.number%%octopus.package.suffix%.zip

that would produce a file Terraform.1.123.9-feature-AB-123 which will later be uploaded to Octopus.

Octopus Setup

Let's create a new Step in Octopus project:

We currently use an official Apply a Terraform template.

However, there are few things which we had to configure in order to make it work for us.

First of all, Octopus cannot keep up with Terraform development, so you may need to update a terraform.exe version yourself. Just grab the latest version from Terraform Downloads page and put it in a folder on your Octopus server. Something like C:\Program Files\Terraform would do.

Then you need to provide a path to your custom terraform.exe executable. To do that you need to set up a variable: Octopus.Action.Terraform.CustomTerraformExecutable. Give it a C:\Program Files\Terraform\terraform.exe value.

As you remember the release package contains a terraform.tfvars file with templates in it. So don't forget to tick this box:

And of course, you will have to define your variables for each environment:

note the variable name format. It should be the same as the variable names in your terraform.tfvars file. Here we can have different values for each environment. Also, we store passwords encrypted.

And now we have a new player in our CI/CD pipeline:

Crossposting from my personal blog

Should I do that test assignment?

Anton Sizikov — Thu, 22 Nov 2018 19:03:09 +0000

Test assignments aren't really a good thing. In fact, it's quite the opposite. It might be one of the worst things that have happened to the hiring process.

The hiring process, in general, is quite broken in many senses. It's not really clear how to hire an engineer. New ways of testing our skills come and go. It shows us that it's nearly impossible to check interviewee skills in 30-90 minutes time frame.

I'm not going to talk about different puzzlers or tricky questions about the compiler, nor I'm going to cover any stress-interview practices here. Today is all about test assignments.

From a company's point of view

Hiring is a very expensive process and it's not very reliable either. Hours of interviews ain't free at all:

An hour of one or two engineers conducting an interview
20-30 minutes to prepare before, and to compose results after an interview
Recruiter's time

If you sum it up, you'll end up with a good number. And it's fair to expect the business to cut the costs. There are many ways to make it cheaper: phone screening and tests are very typical in the industry.

And there is one way which outstands the others. An assignment which is given to the candidate. Typically it's a piece of rather boring code to write with a possible trick in it. Sometimes you'd have to do it even before a phone screening phase as an extra filter. And most of the time you take it before an on-site round.

Often it's assumed that you would spend (should I say waste?) a few hours to finish up the assignment. That's an estimate made by the hiring manager or an engineer, and it's based on the assumption that you are familiar with every tech and every aspect of the task.

How long does it take to check your code? Well, it's about 15-20 minutes. Doesn't look fair, right?

That's correct. A company is trying to put the expenses on your name. They don't know how to evaluate your skills and your experience during a short interview, nor they know how to do that by phone, hence the need for you to spend hours in order to convince them simply to talk.

By the way, a person whose job is to evaluate your task is not going to give it too much effort. They have enough on their plate and they don't really like switching their attention. Imagine looking at the same code over an over again. Often they would quickly go through a checklist.

A candidate's point of view

Unless you're aiming for a top tech company like Google, Amazon or Facebook (they don't really give test assignments out, btw), you will not be dying to get a particular job. A vast majority of companies out there are doing similar things: storing the form's input in a DB. Well, sort of, but you know what I mean.

For the candidate test assignments are not very useful. On one hand, often they are similar to each other, on the other hand, they are hard to reuse. Every company is trying to make it a bit special. And you have to start a new project, configure all the parts, pick an ORM or set up a linter, over and over again.

Speaking of... How often do you start a new project at work? Like from the very beginning. You know, a git init and File->New Project. I've done so a couple dozen times doing various test assignments, and handfull times at work. Most of the time you either join an existing project or start one and keep working with it for a long period. So, honestly, it doesn't look like a relevant skill to me.

How much does your time cost?

Think about it. How many man-hours do you have on an overage day? Well, if you're young and single, you might even have all 24 of them. You can't keep it up this way for a long time, but you can make it for a day or two.
However, it's a different story when you're older or have a family. You have like what? 12-14 hours a day. The remaining part of the day you do your chores and sleep.

You have to spend from eight to ten hours at work and commuting. That leaves you about four spare hours to use. So a hiring manager is going to take those four priceless hours. They want you to spend all your available resources so that they can save some time on their side.

How many resources does a company have? There are tens of engineers, managers and HR specialists there. If we speak about a small to middle size company. So they have hundreds of man-hours at their disposal. An extra 20-40 minutes to evaluate a candidate doesn't seem to be too crucial.

And that is an obvious asymmetry in the process. You have less time than the company, but you have to spend more resources upfront.

I don't really know what should we do about it. It's up to each individual. If you feel confident if you value your time and you're applying to an overage company you may refuse taking an assignment home. There are many ways to do so, you may ask them to pay for your time. You may offer them to take a look at your portfolio first. Show them your blog, your public projects if you have any. Let them download an app you built and try it first. Whatever works for you.

However, you might not be in a position to refuse. You can simply be running out of money or you might be talking to a dream job and you don't want to ruin it over the pride. I understand that.

Though I recommend you to publish the assignment somewhere. Let it be GitHub if we talk about code. Or Dribble or any other place which is popular among specialist in your field. Just be clear about it upfront: you are going to publish it because you own the result of your work. Unless they want to buy it out.

Hopefully one day you'll be offered with a similar assignment. Just send them a link back. It might work and could probably save you some time.

It can't be that bad, can it?

Well... Let's see.
Do you remember that time when you've spend many hours, build a perfect thing, neat and polished? And you were hired right after that with no questions asked whatsoever. Ugh, that was just a dream.

Unfortunately, often a perfect result will simply be a reason to meet you in person. And that's where an actual interview will take its place. They would ask you some questions to make sure that you are the author. Sometimes it feels like an unnecessary step to go through. They could have given you a simple 15-20 minutes test over the phone.

Imagine you found that you don't like the office, the team or anything else about the company. You know, there are things with you only can find out during a face-to-face phase. And in order to get there, you've spent hours working on that test assignment. What a waste...

One more reason in favor of test assignments. They should equalize candidates who have time for a public activity like blogging or OSS contributions and people with family or those who cannot publish any code due to their work contract limitations. Honestly, I don't see how that helps. If you have no time to work on your pet project why would you have time to work on a test assignment?

What do you think about it? Do you often see a take-home assignment given before the actual on-site interview?

This text was originally published in Russian in my blog.

You are mocking it wrong.

Anton Sizikov — Wed, 27 Dec 2017 22:03:20 +0000

Well, probably you are not, but let me grumble a little bit anyway.

Mockingbird knows how to mock.

I've been working with various code bases throughout of my career, and there is one pattern which I see rather often. As you may have already guessed it's the unit-tests and mocking I'm going to talk about here. To give it a nice catchy start, I'd claim here, that mocks should be used when you have to, but not when you can.

I'll give a few examples of somewhat useless and even harmful tests.

All the examples are going to be made up, but I hope you'd get the point.

So, let's start with the typical Greeter example. This is a "Hello, world" of Unit Testing. In one way or another, this sample gets repeated in all the articles, posts and books dedicated to unit-tests and mocking frameworks.

IGreeter{
   string Greet(string name, string title);
}

public class Greeter: IGreeter{
  public string Greet(string name, string title){
    return "Hello," + title + " " + name;
  }
}

public class Client{
  private IGreeter _greeter;
  public Client(IGreeter greeter){
    _greeter = greeter;
  }

  public string FormatPageHeader(string name, string title){
    return "<h>" +  _greeter(name, title) + "</h>";
  }  
}

 public class ClientTests {
    pubic void Test(){
        var mock = new Mock<IGreeter>();
        mock.Setup(greeter => greeter.Greet("John", "Mr.")).Returns("Hello, Mr. John");
        var result = new Client(mock.Object).FormatPageHeader("John", "Mr.");
        Assert.AreEquals(result, "<h>Hello, Mr. John</h>"); //All good here, what does it test, tho?
    } 
 }

So far so good. Tests are green. The Greeter interface isn't perfect, though. Two strings as a parameter? So easy to mess up, isn't it? Most probably you have a method like that in your project.

Ok then, imagine that you decided to make this method less error prone. You don't have much time for a proper refactoring because you have a feature to work on. Let's just swap the parameters. It's much more natural for a human to put the title before the name. Depends on the culture, I know.

("John", "Mr.") is more awkward compared to ("Mr.", "John") and ("Dr.", "Smith").

So, we'll end up with the greeter like that:

//Version 2
public class Greeter2 : IGreeter{
  public string Greet(string title, string name){
    return "Hello, " + title + " " + name;
  }
} 

IGreeter{
   string Greet(string title, string name);
}

add., commit, push. Our beloved build server will pick the changes up, will run the tests and will fail of course. We forgot to update the test for the IGreeter implementation. Once they fixed we're good, aren't we?

Not really, remember the Client class? Now it's incorrect, we still pass the title and the name in the old order, even though our test claims that everything is fine.

Here is the paradox, we introduced mocking so that we can test our class in isolation, but we had to put some code to make that mock return something. It's still the logic, isn't it? Now IGreeter has at least two implementations. Most probably in your codebase, you have tens of implementations for the mocked class. Just because it's a very much reused dependency, and you have to mock it all over again.

We can improve that test, tune it, but the only way we can make it fail is to repeat the Greeter logic in our mock set up. But wait a minute, if we have the same implementation, why don't we reuse the existing code?

The more complex your mocked object is the more complex your mock setup becomes. If that's not the case, most probably you're testing just the interaction with your dumb mock, i.e. you're not testing anything. Just like the ClientTest above. The only positive outcome is the extra 20c which your company pays to Amazon for that CPU time you waisted on the build server.

It's not rare to have more than one dependency. I can imagine that our Client could use some IHmlRenderer which would consume the IGreeter result. You would mock that one too, right?

Sweet as. You now have a test which verifies how two mocks are integrated with each other. How does that prove your code correctness, thought? I don't know.

I'm going to step back now, and talk about it from another point of view. What does the typical system look like? I assume, it's not the bunch of isolated classes, but it's more like a spiderweb of dependencies. If you look at type dependency diagram you would see a group of clusters, where each cluster is a somewhat tightly coupled set of classes. This is how we manage complexity, we break down one large class into small pieces, but those pieces would never be used in isolation. Each of them would be responsible for a little piece of work. You would write a separate test suite for them, mock out all the dependencies (they all belong to the same cluster).

How is that different to the attempt to test a private method?

That little convenience wrapper for the standard library class is nothing more than an implementation detail.

But integration tests are slow...

That would be an expected note. If we come back to our ClientTest. What performs better, the test with mock, or the test with the concrete Greeter implementation? I guess the answer is obvious. We shouldn't forget that that test doesn't prove anything. It is slower, it allocates more, and it's wrong. I'd say it's harmful.

So you're going to send out those emails every time you test?

Ok, that is a good question. Remember I said that we should mock when we have to, not when we can? That's exactly the right situation for the test isolation.

You don't want to make HTTP calls while running your unit tests suite, you don't want to send out emails, neither do I.

Hey, I tried not to mock, and I got sick of initing all the dependencies

This is where it gets painful. I saw systems like that, it's a nightmare to maintain. Manually setting up the dependency of the dependency is really not the way to go. It's hard, and everyone would avoid writing a new test. But that's an easy task to solve.

How do we build a dependency tree in the runtime? I hope you're using the DI framework which wires up interfaces and implementations together, it knows how to create a new dependency, it knows when and how to rebuild a dependency tree. If you're building a web service you should reset the state between requests, so keep as many dependencies request scoped as it's possible.

The same pattern is applicable for your test suite. Everything is request scoped and stateless already, so treat each test as a 'request' and let the DI container to build the system under test for you.

Of course, you would have to set up the DI container differently.

Module (or cluster) level test would build the dependency tree for that module but would mock out the rest of the world. Like no HTTP calls, no DB, no emails, you've got the point.

At this point, you would realize that you don't need that test which verifies that your utility class can split the string, but you would be sure that your MortgageCalculator does the job. After all, that is your business rule, and that is the feature you're building. Unless you are the low-level framework developer of course. Most of us are not, though.

Once you've got all the clusters well tested, all the interfaces established you may try to break it down a little bit. Or you may want to leave it as is, it's up to you.

Imagine you want to extract some logic into the separate class. Now your system has a new dependency, but that logic has already been tested. You don't need to set up a new mock, you don't need to repeat the logic of the extracted class in your test set up, you don't need to update the test to instantiate the tested class. It's just there, and your safety net still works. If you extract and modify the logic, you would break the test. Do you see the beauty? Extracting the class and introducing a new dependency is as easy as moving the logic into a private method.

Summary

The approach above would give you the following benefits:

Fewer meaningful, useless and harmful tests
No need to maintain duplicated implementation (mock set ups)
Feature driven tests, tests that verify that the cluster does the job
An easy to refactor code base
You can improve module level access (no need to make the class public just for the testing)

From my experience tests, based on that cluster approach, are much more reliable. When they break, they actually mean that the system is dysfunctional, when they pass you can be sure that you did not break the logic.

We still have to run integration tests to be sure that the third party integrations are working, that the system's runtime configuration is valid.

I am more than open to any criticism, feel free to tear that post apart. :)

Crossposting from my personal blog