Property-based testing: finding bugs that example-based tests miss

Example-based tests check specific inputs and expected outputs. They're the standard approach, but they only test what you think to test. Property-based testing generates random inputs and checks that certain properties always hold true. It finds bugs at the edges of your logic.

A property is a statement that should always be true for your function. For a sort function, the property is that the output is sorted and contains the same elements as the input. For a phone number formatter, the property is that the output contains only digits and dashes.

Property-based testing frameworks generate random inputs and check your properties against them. Hypothesis for Python, fast-check for JavaScript, and QuickCheck for Haskell are popular choices. The framework generates hundreds of test cases, and when one fails, it simplifies the input to the minimal failing case.

Common properties include: idempotence, commutativity, and round-tripping. These properties capture the invariants of your system. Idempotence means running the function twice produces the same result. Commutativity means the order of operations doesn't matter. Round-tripping means serialization followed by deserialization returns the original value.

Property-based tests are especially effective for parsers, validators, serializers, and state machines any code that processes complex inputs. These are the areas where example-based tests often miss edge cases.

Integrate property-based tests into your test suite alongside example-based tests. Run them with fewer iterations during development and more iterations in CI. Property-based tests are slower than example-based tests but find deeper bugs.

The investment pays off over time. Once you write the property, the framework generates new test cases automatically. As your code evolves, the property continues to verify correctness.

-

Rizwan Saleem | https://rizwansaleem.co