Thanks for picking me up on this 🤣 I don’t disagree with you. Deciding what to test and at what level is one of the hurdles that trips people up. Overtesting is a serious problem. For unit tests I usually avoid testing anything ”static” like HTML or CSS. Those are tests that I’d rather leave to QA, or write up as system/acceptance tests in some other way.
To come back to my statement... yeah, there’s a huge assumption hidden within it, I‘ll see if I can find a way to make that clearer, as discussing how to test well is waaaay beyond the scope of this post 🤗
Does this really exist? I would be interested in hearing some examples. 🙏
I can definitely see a problem in time spent in testing. And that is most likely spent in manual testing because of not having knowledge on how, on what level and what exactly to test.
It might be something I’ve just dreamed up, or maybe there’s a better name for it, but here’s how I think of overtesting.
Overtesting is what happens when your write tests that cover more surface area than necessary. One way this happens is relying on end-to-end tests instead of unit tests. Greater surface area means more chance for tests to break when code is changed. In other words, you get brittle tests.
Each test is additional code which must be maintained. If the tests need to be updated with every code change, even though the functionality they are expecting to test isn't changing, it becomes a liability.
Clearly that isn't because there are too many tests, but it kind of is. The mantality this come from is that every test is valuable. While you could write another test, clearly this is covering more and may find an issue it just isn't this time.
Thanks for picking me up on this 🤣 I don’t disagree with you. Deciding what to test and at what level is one of the hurdles that trips people up. Overtesting is a serious problem. For unit tests I usually avoid testing anything ”static” like HTML or CSS. Those are tests that I’d rather leave to QA, or write up as system/acceptance tests in some other way.
To come back to my statement... yeah, there’s a huge assumption hidden within it, I‘ll see if I can find a way to make that clearer, as discussing how to test well is waaaay beyond the scope of this post 🤗
Does this really exist? I would be interested in hearing some examples. 🙏
I can definitely see a problem in time spent in testing. And that is most likely spent in manual testing because of not having knowledge on how, on what level and what exactly to test.
Thanks for a great post! 🍻
It might be something I’ve just dreamed up, or maybe there’s a better name for it, but here’s how I think of overtesting.
Overtesting is what happens when your write tests that cover more surface area than necessary. One way this happens is relying on end-to-end tests instead of unit tests. Greater surface area means more chance for tests to break when code is changed. In other words, you get brittle tests.
Hopefully that makes sense?
Each test is additional code which must be maintained. If the tests need to be updated with every code change, even though the functionality they are expecting to test isn't changing, it becomes a liability.
Clearly that isn't because there are too many tests, but it kind of is. The mantality this come from is that every test is valuable. While you could write another test, clearly this is covering more and may find an issue it just isn't this time.
It makes!
I think this is due to focusing the tests on the wrong level.
Too small or inappropriate details are verified in a test that is done by eg. Calling the service over HTTP.