One issue with unit testing is the tendency to tie the test to the result of the code, not the code itself. In other words, testing that a markdown filter produces the actual content in your filesystem as intended. It's less intuitive, but more durable, to test the code against a mocked markdown file, in that case. Content may change at any moment, and it doing so shouldn't break the test, because the code has not changed.
This doesn't make any sense. If you are unit testing code by supplying input (domain) values and then checking/verifying/asserting the output (range) values, and if when tests run repeatedly you get different output values with identical input values, you know that something is terribly wrong with your design. You should never see that scenario, ever!
I think you are responding to a different comment, since you are not responding to what I wrote. My point is exactly when input changes, it shouldn't break the test.
Sorry, probably misread your comment. Still not sure I understand how is it possible that a change in the input value breaks the test? Please explain.
If a test's expectation is linked to the exact desired output, then changing the input, for example, the wording in the hypothetical markdown file, will break the test. In my experience, this is something not to do, because it adds unnecessary fragility to the unit test. That's what I was saying.
Have a great weekend!
Agree. That's a problem in general with unit testing infinitely variable textual values. It's almost never a case for unit testing if you end up trying.
We're a place where coders share, stay up-to-date and grow their careers.
We strive for transparency and don't collect excess data.