DEV Community

Discussion on: From 1 to 10,000 test cases in under an hour: A beginner's guide to property-based testing

individualit profile image
Artur Neumann

What would you say is the difference to fuzzy-testing?

ssimontis profile image
Scott Simontis

Isn't fuzzy testing just randomly generating values with no rhyme or reason (or perhaps within a large range)? It seems like property-based testing is more about guaranteeing that logical assertions hold throughout a large number of test cases versus ensuring that random input is handled correctly.

I suppose I need to go learn about fuzzy testing now!

individualit profile image
Artur Neumann

looks to me that the input values in the examples in this blog are pretty random (in the defined scope e.g. int or json) that is why you need to use @example to make sure this specific value will always be tested.
Maybe verifying that the results are correct is more the point in "property-based testing", you have random (fuzzy) input and because of that you cannot really check the details of the output so you "only" make sure properties (that you can be sure of) of the output are correct

Thread Thread
fornwall profile image
Fredrik Fornwall

In general, a property based test can be formulated as:

for all (x, y, ...)
such as precondition(x, y, ...) holds
property(x, y, ...) is true

One way to look at it (which not everyone agrees with, and is more theoretical than practical) is that fuzzing is a subset of property-based testing which focuses on the property that code should never crash (segfault, throw an exception, give an internal error response code, overflow buffers, etc) regardless of input.

While we have generated rather random inputs in this article, one will often use custom generators (such as the some_object generator in the test_json5_loads example) to build up more restricted, higher level input. It's also possible to use a mechanism like hypothesis.assume to filter away certain generated input that doesn't match the desired precondition.