DEV Community

Samuel Ko
Samuel Ko

Posted on

2

An interesting observation on C# code coverage

Introduction

Code coverage is an indicator, not an metric.

In this post, I will share an example for your entertainment.

The code

public class Animal
{
    public bool CanTalk { get; init; }
    public bool CanWalk { get; init; }
}

public static class AnimalValidator
{
    public static bool IsHuman(Animal animal)
    {
        return animal.CanTalk && animal.CanWalk;
    }
}
Enter fullscreen mode Exit fullscreen mode

The good test

Let's test all the input combinations.
It gives us 100% code coverage:

public class AnimalValidatorTest
{
    [Theory]
    [InlineData(true, true)]
    public void IsHuman_WhenAnimalCanTalkAndCanWalk_ReturnsTrue(bool canTalk, bool canWalk)
    {
        var animal = new Animal { CanTalk = canTalk, CanWalk = canWalk };
        var isHuman = AnimalValidator.IsHuman(animal);
        Assert.True(isHuman);
    }

    [Theory]
    [InlineData(false, false)]
    [InlineData(false, true)]
    [InlineData(true, false)]
    public void IsHuman_WhenAnimalCannotTalkOrCannotWalk_ReturnsFalse(bool canTalk, bool canWalk)
    {
        var animal = new Animal { CanTalk = canTalk, CanWalk = canWalk };
        var isHuman = AnimalValidator.IsHuman(animal);
        Assert.False(isHuman);
    }
}
Enter fullscreen mode Exit fullscreen mode

The 100% coverage test

One day, someone removed two test cases because they thought that the tests were running for too long .

[Theory]
// [InlineData(false, false)]
[InlineData(false, true)]
// [InlineData(true, false)]
Enter fullscreen mode Exit fullscreen mode

Interestingly, we still get 100% code coverage.

The explanation

Let's put the code into SharpLab.

This code:

return animal.CanTalk && animal.CanWalk;
Enter fullscreen mode Exit fullscreen mode

will compile into:

if (animal.CanTalk)
{
    return animal.CanWalk;
}
return false;
Enter fullscreen mode Exit fullscreen mode

It means:
CanTalk = true contributes 50%.
CanTalk = false contributes 50%.

The value of CanWalk doesn't matter.

The brittleness of 100% coverage test

Another day, someone changed the order of boolean because they thought it would improve performance because majority of animals cannot walk.

public static bool IsHuman(Animal animal)
{
    // return animal.CanTalk && animal.CanWalk;
    return animal.CanWalk && animal.CanTalk;
}
Enter fullscreen mode Exit fullscreen mode

Now, our test coverage drops from 100% to 50%, because both of our test cases are CanWalk = true.

Conclusion

C# code coverage is based on compiled code.

Writing good unit tests is more important than code coverage.

Hope you find it interesting.

Imagine monitoring actually built for developers

Billboard image

Join Vercel, CrowdStrike, and thousands of other teams that trust Checkly to streamline monitor creation and configuration with Monitoring as Code.

Start Monitoring

Top comments (0)

Image of Timescale

Timescale – the developer's data platform for modern apps, built on PostgreSQL

Timescale Cloud is PostgreSQL optimized for speed, scale, and performance. Over 3 million IoT, AI, crypto, and dev tool apps are powered by Timescale. Try it free today! No credit card required.

Try free

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay