DEV Community

Samuel Ko
Samuel Ko

Posted on

2

An interesting observation on C# code coverage

Introduction

Code coverage is an indicator, not an metric.

In this post, I will share an example for your entertainment.

The code

public class Animal
{
    public bool CanTalk { get; init; }
    public bool CanWalk { get; init; }
}

public static class AnimalValidator
{
    public static bool IsHuman(Animal animal)
    {
        return animal.CanTalk && animal.CanWalk;
    }
}
Enter fullscreen mode Exit fullscreen mode

The good test

Let's test all the input combinations.
It gives us 100% code coverage:

public class AnimalValidatorTest
{
    [Theory]
    [InlineData(true, true)]
    public void IsHuman_WhenAnimalCanTalkAndCanWalk_ReturnsTrue(bool canTalk, bool canWalk)
    {
        var animal = new Animal { CanTalk = canTalk, CanWalk = canWalk };
        var isHuman = AnimalValidator.IsHuman(animal);
        Assert.True(isHuman);
    }

    [Theory]
    [InlineData(false, false)]
    [InlineData(false, true)]
    [InlineData(true, false)]
    public void IsHuman_WhenAnimalCannotTalkOrCannotWalk_ReturnsFalse(bool canTalk, bool canWalk)
    {
        var animal = new Animal { CanTalk = canTalk, CanWalk = canWalk };
        var isHuman = AnimalValidator.IsHuman(animal);
        Assert.False(isHuman);
    }
}
Enter fullscreen mode Exit fullscreen mode

The 100% coverage test

One day, someone removed two test cases because they thought that the tests were running for too long .

[Theory]
// [InlineData(false, false)]
[InlineData(false, true)]
// [InlineData(true, false)]
Enter fullscreen mode Exit fullscreen mode

Interestingly, we still get 100% code coverage.

The explanation

Let's put the code into SharpLab.

This code:

return animal.CanTalk && animal.CanWalk;
Enter fullscreen mode Exit fullscreen mode

will compile into:

if (animal.CanTalk)
{
    return animal.CanWalk;
}
return false;
Enter fullscreen mode Exit fullscreen mode

It means:
CanTalk = true contributes 50%.
CanTalk = false contributes 50%.

The value of CanWalk doesn't matter.

The brittleness of 100% coverage test

Another day, someone changed the order of boolean because they thought it would improve performance because majority of animals cannot walk.

public static bool IsHuman(Animal animal)
{
    // return animal.CanTalk && animal.CanWalk;
    return animal.CanWalk && animal.CanTalk;
}
Enter fullscreen mode Exit fullscreen mode

Now, our test coverage drops from 100% to 50%, because both of our test cases are CanWalk = true.

Conclusion

C# code coverage is based on compiled code.

Writing good unit tests is more important than code coverage.

Hope you find it interesting.

Sentry image

Hands-on debugging session: instrument, monitor, and fix

Join Lazar for a hands-on session where you’ll build it, break it, debug it, and fix it. You’ll set up Sentry, track errors, use Session Replay and Tracing, and leverage some good ol’ AI to find and fix issues fast.

RSVP here →

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay