Unit proving, not testing

In 1910 mathematicians Alfred North Whitehead and Bertrand Russell published Principia Mathematica. A three volume and nearly 2000 page collection of mathematical rules and axioms used to establish a firm basis for the topic. It takes 762 pages before it is possible to prove that 1 + 1 = 2, at which point the authors state "The above proposition is occasionally useful."

Imagine no-one had figured out how to perform multiplication. Instead of calculating 3.217 x 49 would you prefer to add 3.217 to itself 49 times, or can you demonstrate that adding 49 to itself 3.217 times will produce the same result. That is what mathematicians do (build proofs) and the rules and axioms from Principia Mathematica provide the foundation for building more complicated rules.

It is my contention that unit tests serve a very similar purpose in software engineering. After all, if a developer does not write unit tests how can they possibly say their code is complete and works as they had intended? Equally, long after the original developer of a unit has left the project, well written unit test provide firm evidence the code performs as the developer intended.

Despite the name, in my mind, unit tests are not really about testing but enhance the development process in many other ways such as:

1. Providing repeatable evidence the unit works as the developer had intended.
2. Providing early indication of the adverse impact of a breaking interface.
3. Demonstrating how a unit is expected to be used without bias from a specific use case.
4. Helping to identify redundant code and aid safe removal especially when used in combination with code coverage analysis.
5. Facilitating fault investigation. Got a bug, write a test case to repeat the flaw and then make the correction. When the test case passes the bug is fixed with evidence.
6. Supporting the process of refactoring with confidence, which is vital when engaging in Test-driven development (TDD).
7. Providing an alternative and more formal design specification prior to development. A bit like a remote form of pair-programming combined with TDD.

Writing a good unit test proof

Unit tests are a suite of test cases that (usually) exercise a unit by calling its public methods. Private methods are (usually) exercised courtesy of public methods. There are three conventions that go along to ensuring unit tests and test cases are well written and achieve their primary purpose.

F.I.R.S.T rules

• Fast - slow unit tests do not get executed so keeping unit tests succinct and responsive is important so they can be executed regularly.
• Isolated/Independent - Test cases should run irrespective of the environment, outside of what is being controlled. They should also be independent from each other and not be reliant on the running order.
• Repeatable - Test cases should be written so they run the same each and every time. They should not be dependent on external data sources that could vary - ever.
• Self-validating - Test cases should provide evidence they executed and publish the results so the developer does not have to trawl through a log or something to determine the outcome. A conventional approach to reporting the results of a test case is through assertions, which are special test commands that perform the comparison between an expected value and the actual. The assert (aka expect) statements usually report the test case that failed, where in the test case it failed and what the error was.
• Thorough - The range of test cases should exercise all of the positive cases "Happy paths" but also as many negative cases as can be easily identified; there will always be more. Using coverage analysis in conjunction with unit tests can help identify test cases but both paths and routes need to be considered. What's the difference - let me explain.

Path and routes through a unit

Consider the following, very simplistic, code.

function underTest( testDatum) {
    if (testDatum % 2) {
        console.log(`${testDatum} is Odd`);
    }
    else {
        console.log(`${testDatum} is Even`);
    }
    if (testDatum > 42) {
        console.log(`${testDatum} is greater than 42`);
    }
    else {
        console.log(`${testDatum} is less than or equal to 42`);
    }
}

To exercise all the paths we only need two tests.

underTest(43);  // 43 is Odd and greater than 42
underTest(42);  // 42 is Even and less than or equal to 42

But they only test half of the routes. For that we can add the following test cases.

underTest(41); // 41 is Odd and less than or equal to 42
underTest(44); // 44 is Even and greater than 42

The AAA test case structure

The AAA approach to writing a test case employs the following steps:

• Arrange
• Act
• Assert

Arrange

The first step involves preparing the test environment, including any test data, required to ensure as consistent a test is performed as possible.

Act(ion)

We then conduct the action that performs the test condition, which should be consistent with the name of the test case.

Assert

Once the action has completed the test will check a number of assertions to confirm the action had the desired effect on the test environment.

Personal additional step
There is always the possibility the test environment changes or is not quite as expected for the test. This can lead to unexpected failures or worse a false positive that may not be detected and masks an actual fault.

To safeguard against this I occasionally include an Affirm step between Arrange and Act. This involves adding assertion statements prior to executing the act(ion) to confirm the environment is as expected.

Conclusion

Building a good body of unit tests can establish considerable confidence in the stability and robustness of the source code. This is particularly important when engaged in a refactoring task where the interface and desired effect of the code remains the same but the underlying implementation is being changed.

Supplemental

For anyone interested, I have recently released a post picking up on my 4A Unit Test pattern.

Comments

[Rahul Ladumor]

Great read! You've laid out an insightful comparison between mathematical proofs and unit tests, emphasizing the indispensable role of testing in the software development lifecycle. I particularly appreciate your dive into the F.I.R.S.T rules and the AAA test case structure, bringing an almost scientific methodology to what's often considered an 'art.' Your inclusion of the 'Affirm' step is intriguing—essentially a pre-assertion to ensure environmental consistency, which is so crucial yet easily overlooked.

-Mr. Rahul