Having good failure messages does not sound very important. In many ways I agree, in D without an extra library or a custom message a failure message boils down to a file and line number. A quick printout of the result and rerun of the test has generally been sufficient.
My day job entails writing tests and working with others writing tests. These are visible to the developers as we run them in a CI pipeline. In this case a file and line number is not sufficient.
I've been working on our messages (expected 1 actual 0) because they didn't help guide the reader to the right course of action. In this case xUnit is being used and it does not allow custom messages on any assert except Assert.True()
For this reason it is important that all assertion happen with true, this allows a message like "expected to get the value 0 when expecting the result to be 1" this provides a lot more on what is the value and which is the result.
Seriously I want to talk about lists, checking optional equally with order. The contains function can find that an element exists but not the order in the list. Jumping to the correct list location and checking equality provides the expected and actual but doesn't give context on the expected or actual position.
This has led me to dynamically build an expected list (this really is only beneficial for short lists). By using the actual results to fill in expectations the message printed states the actual results as expected.
auto result = [1, 2, 3, 5];
[1, null, null, 4].fill(result)
== [1,2,3,4];
Now the error from a friendly assert will provide the two lists and standard collection asserts don't need special training for optional values.
—--------
Consider that I am not saying you should spend a lot of time on the errors messages. I have found that it is important to my situation.
I will also say that the process of making more information available created a common set of code for data retrieval, reducing code duplication and surfaced more of the action steps in the test function.
Top comments (0)