An interesting observation on C# code coverage

Introduction

Code coverage is an indicator, not an metric.

In this post, I will share an example for your entertainment.

The code

public class Animal
{
    public bool CanTalk { get; init; }
    public bool CanWalk { get; init; }
}

public static class AnimalValidator
{
    public static bool IsHuman(Animal animal)
    {
        return animal.CanTalk && animal.CanWalk;
    }
}

The good test

Let's test all the input combinations.
It gives us 100% code coverage:

public class AnimalValidatorTest
{
    [Theory]
    [InlineData(true, true)]
    public void IsHuman_WhenAnimalCanTalkAndCanWalk_ReturnsTrue(bool canTalk, bool canWalk)
    {
        var animal = new Animal { CanTalk = canTalk, CanWalk = canWalk };
        var isHuman = AnimalValidator.IsHuman(animal);
        Assert.True(isHuman);
    }

    [Theory]
    [InlineData(false, false)]
    [InlineData(false, true)]
    [InlineData(true, false)]
    public void IsHuman_WhenAnimalCannotTalkOrCannotWalk_ReturnsFalse(bool canTalk, bool canWalk)
    {
        var animal = new Animal { CanTalk = canTalk, CanWalk = canWalk };
        var isHuman = AnimalValidator.IsHuman(animal);
        Assert.False(isHuman);
    }
}

The 100% coverage test

One day, someone removed two test cases because they thought that the tests were running for too long .

[Theory]
// [InlineData(false, false)]
[InlineData(false, true)]
// [InlineData(true, false)]

Interestingly, we still get 100% code coverage.

The explanation

Let's put the code into SharpLab.

This code:

return animal.CanTalk && animal.CanWalk;

will compile into:

if (animal.CanTalk)
{
    return animal.CanWalk;
}
return false;

It means:
CanTalk = true contributes 50%.
CanTalk = false contributes 50%.

The value of CanWalk doesn't matter.

The brittleness of 100% coverage test

Another day, someone changed the order of boolean because they thought it would improve performance because majority of animals cannot walk.

public static bool IsHuman(Animal animal)
{
    // return animal.CanTalk && animal.CanWalk;
    return animal.CanWalk && animal.CanTalk;
}

Now, our test coverage drops from 100% to 50%, because both of our test cases are CanWalk = true.

Conclusion

C# code coverage is based on compiled code.

Writing good unit tests is more important than code coverage.

Hope you find it interesting.