This article originally appeared on IG's blog
It is funny how things turn around. For fifteen years I have been preaching TDD, or at least for d...
For further actions, you may consider blocking this person and/or reporting abuse
We decided to use screwdrivers for everything.
Finally got that nail in. I think screwdrivers are hard to use. For our next project let's use hammers for everything. They look simpler.
Rolf. Very accurate summary :)
😄👍🏻
This reminds me of what I read from Uncle Bob and DHH (linked from Uncle Bob's article): blog.cleancoder.com/uncle-bob/2017...
Recently I've come to the conclusion that we made an Idol out of unit tests and greatly undervalue integration tests. The examples above are clearly show code that could be tested as part of the bigger picture, because in itself it doesn't mean anything: it's glue code and integration tests are supposed to check exactly that... The gluing of components together!
So the bottom line is: write a good mixture of unit and integration tests and, most of all, THINK! That's our job.
The most important thing you have pointed out is : "THINK".
There is a perfect gif to illustrate your point: 9gag.com/gag/a0pbDeX/programmers-w...
I believe that 100% test coverage is not a good idea in applications. However, I find it very very valuable in libraries. I have a few personal recommendations:
Avoid white-box testing. It makes refactoring harder. We should be able to achieve 100% test coverage using only the public API of the library.
Every reported bug should be reproduced with a unit test before it is fixed. These unit tests are more important than achieving 100% coverage.
Yaaas! Blindly following metrics and using tools without applying critical thinking of cost vs benefit is the worst! I especially hate when it causes other devs or management to get a bad taste in their mouth around quality and start pushing devs to skip things that are actually important.
I've found tools like cucumber are awesome at the acceptance test level, but the cost vs benefit breaks down very quickly as you try to apply it at lower levels. You're basically maintaining this alternate human readable text and its mapping to code in addition to your regular unit test code. But is any non-developer going to read that feature file? No. So why do it when developers can easily (and probably more easily) just read well-written unit test code??
Definitely going to refer my teams to this article. Thanks for writing it!
Thanks a lot for the feedback!
When doing TDD I think it's important to treat your tests just as importantly as you treat your code. That means also applying the KISS and DRY principles when writing your tests. When I'm writing tests for a class or something, I'll usually write a test that I can instantiate the thing first, but after I have written the tests testing what the class actually does, I'll prune off all the pointless tests that served more as progress markers during the TDD process rather than actual, meaningful tests.
That is a very mature and, in my experience, unusual practice. Congrats!
I haven't done much formal testing myself, and definitely not any unit testing. While trying to understand it a bit more, I came across a really good paper: rbcs-us.com/documents/Why-Most-Uni...
What are your thoughts?
Mr. Coplein is above my pay-grade, so I will let Uncle Bob argue with him :)
m.youtube.com/watch?v=KtHQGs3zFAM
Personally, I always say that system level test are the best and only ones that we should write, if they could be run in a few seconds, in isolation by several developers at the same time.
In my personal experience, the systems that I work with are composed of several docens of moving parts, so system level test end up being too slow, brittle and hard to debug.
I wasn't there, but I think that is a big reason why unit test were born, out of pragmatism.
As Stéphane says in the comment, you need a good mix of test, but tests are not a substitute for thinking!
I would recommend Rich Hickey talk "Hammock Driven Development" m.youtube.com/watch?v=f84n5oFoZBc (all talks by Rich Hickey are excellent!) and Uncle Bob blog blog.cleancoder.com
Thanks for the question and the pdf!
100x this!
Oh and don't forget: Collaborate with the business representative (product manager, product owner, business analyst, whatever) on your high-level test cases! You will be surprised how good both of your understanding of that tested feature will become.
A better response to "How do I know then if the code works?!" might be to say, "No amount of unit tests can guarantee that your code works."
All tests are a cost benefit trade-off. You absolutely need them, but beyond a certain point you get diminishing returns. Writing useful tests is all about value, getting the most bang for your buck.
Testing simple accessors and "glue code" typically has little to no value. However, if it's the case that the "glue code" has a lot of churn, or something about it feels fragile, that it could break, a high level integration or acceptance test can prove very useful. Even if it doesn't always pinpoint the exact "glue code" that broke, you at least have broad coverage over multiple possible points of failure.
Very true.
Most (maybe all?) things in live, not just programming, have trade-offs.
If you don't stop to think about those trade-offs, you cannot make any good decisions. In fact, you are not even making decisions.
We are paid to think, or that is what I want to believe.
Thanks,
Dan
Thank you for this article. I can now go back and remove some tests which I felt at the back of my mind were a waste of time! :-)
Having said quite a bit of learning does come about in terms of architecture (is that too big word?) when writing unit tests but only if we know what we are testing.
the problem with this is that it's subjective. as organization grows, as people of different levels and personal preferences come and go, subjectiveness in these best practices will inevitably lead to lower software quality.
The only reason not to test something is if it's hard or tooling is missing. Those cases require case-by-case analysis and thorough code review.
Hi Alexander,
Thanks a lot for the comment.
I am going to argue that “there are no conditionals, no loops, no transformation” is quite objective. This rule can be follow by any developer of any level. There is no room for interpretation.
On the other hand I find that “not to test something if it’s hard” is very subjective, as it will depend a lot on each developer’s experience what they find hard to do.
That said, I think you are right that a big chunk of what we do as developers is subject to a high degree of subjectiviness: good architecture and design, code quality, simple code, easy to understand, obvious code, engineering vs craftsmanship, dynamic vs static...
I think this is why a lot of people say that what we do is “art” and not engineering. What I think is a masterpiece, you can think is a big pile of crap, and neither of us can provide objective enough reasons to convince the other.
I think the only reason to not automate a test is if it brings no or negative value. The value of a test is he cost of writing and maintaining it vs its benefit: time saved by catching bugs. Unfortunately, as you point out, the value is very subjective.
I am curious about the “tooling is missing” statement. Do you mean UI tests?
Cheers,
Dan
Very resonating post. If one wants to be called software ENGINEER, he/she must start thinking about business tradeoffs and invest time where it's most valuable.
An article that has been really enlightening for me in the past:
blog.stevensanderson.com/2009/11/0...
Hi Raptis,
Thanks a lot for the link, it is indeed very good.
Cheers,
Dan
This is where I draw the line between experience and knowledge of developer tools. Most developer feel that they must use every tool, apply every methodology without critically thinking through the use cases.
I will share with my team.
Well, at least you found something :). Also, in a couple of months you can reflect on how useful your task was. I would love to hear about it.
There is always a tension on how big is the unit under test, how many paths the test can cover and how confident you are about your test suite.
Hard to get right, I wrote some very fuzzy and not very specific advise.
From my experience code coverage metrics are rarely defined by the development team but a management team. For every marginal increase in test coverage there is a less proportional increase in value. Sounds like this organisation has more money then sense to be pursuing 100% coverage. Apply tests (both unit & integration) where it makes most sense.
Docker and testcontainers are changing the game. Pair that with Jenkins on Cloud wth auto scale group - you get what you wanted below - it's not seconds but it's tens of seconds for single test and minutes for full test suite. Oh and it's easy to debug.
I think the real problem is this DTOMapper. Sounds like a really sad architecture/ framework. It should have never been created at first place, then no one would have idea of testing it.
I know - if you follow "industry standard" of having anemic domain model with javabeans - you quickly get that abominations - your choice.
Don't get me started with the new industry standard: annotate-all-the-things.
See youtube.com/watch?v=-6zT60l5hDc for a good presentation about it. Maybe you are familiar ;).
Cheers,
Dan
This is where I draw the line between experience and knowledge of developer tools. Most developer feel that they must use every tool, apply every methodology without critically thinking through the use cases.
This article is on point, I will share with my team.
Thanks for sharing!
Part of the tragedy is that code coverage doesn't necessarily relate to actual product quality. There is no reason to believe than increasing the coverage number will actually help route out more bugs and keep the product stable.
Programming quality through tests requires specific targetting and deduction. Some code will have may more than 100% coverage, and other bits will have virtually nothing.
Testing also has the potential problem that people rely on tests too much while refactoring code. They assume that since they have tests they can make random changes and will be safe.
Writing a good test suite is very hard. As you said, requires thinking ;).
In my opinion, a bright developer should know what pieces of code are worth to test and those that are not. Even if they are required to meet a coverage number I had expected a way better response from those devs. Were they very junior developers? Maybe not so exposed to unit testing before?
Hi Israel,
I don't think writing good tests is easy, no matter how bright you are. Writing good test is as difficult as writing good production code.
I see a big cargo cult around testing in the Java world (not sure about other communities), which is difficult to see from within, and more difficult to go against, specially if your team lead and your teammates are part of it.
Thanks for the comments!
Dan
100% test coverage gives the illusion of complete QA. Senior management like easily measurable illusions.
Wow, this is my first time seeing Cucumber used for unit testing. That is ...interesting to say the least.
Very informative, thanks!
Thanks you for reading!
:)
What number were you aiming for?
Did you found any bug while trying to increase the coverage?
Loved this article.
Loved that you loved it! Thanks!
Really good article explaining that you have to be pragmatic when doing anything, instead of blindly following best practices.
Thanks a lot for the feedback. Really appreciate it.