About a month ago, I started a well received discussion about unit-tests. What people think about them and how they help them to design systems or fix bugs.
The Responses
Overall, the responses were in favour of unti-tests or at least automated testing in general. Some people were even advocating Test Driven Development (TDD) as a crucial part of their software design.
Only a minority didn't write unit-tests and all of them seemed like senior developers to me, people who don't have the feeling unit-tests would help them with their current problems.
I also had the impression, while many people write unit-tests, they use them in different ways. Sure, they check a small part of their code and they check it to ensure some kind of quality, but the approaches seem different.
The Two Main Groups
To me it seemed like there are two main groups. People who start with a test and people who add them later.
Test Driven Development
Some use TDD to design their units, they think about how they want to use a part of their code, write a bunch of tests that reflect the desired behaviour and then they implement the tested part of code until the tests stop to fail.
Others also use TDD, but they find writing "good" tests to be the wrong way. They know what their code needs to do and how they want to use it later, so why bother writing tests for it? Instead they write tests for the edge-cases and look that their code fails when it gets wrong data.
And then thare are even people who say, TDD is just TDD. Like, if you write unit-tests somehow, then you are doing TDD.
All these approaches have the similarity that they form the structure of your code. If you want to do TDD, you need to write your code in a way that allows to access the units for testing from the outside.
With TDD your code may end up being more modular and with better encapsulation than without it.
Still one could argue that structuring code for tests instead of the real problems at had shouldn't be the goal, on the other hand if you write code with SOLID principles in mind, you end up with easily testable code too, so maybe the SOLID and TDD are simply two sides of the same coin?
I think this is why some of the senior developers don't see much value in TDD, they have the impression it doesn't add anything to the code they already write.
But there are plenty of senior developers out there who do TDD, because it's an easier way to enforce SOLID principles without thinking. You write your tests and then your code and can be pretty save that the resulting code is reasonably good while you have enough mental capacity left for other problems of the project.
Bug Driven Testing
Lets call the next kind of developers Bug Driven Testers (BDT). They code their software without any automated tests and later, when the manual testers or production users report some errors, they track down the bug and write a minimal test case to reproduce that bug. Later they fix it so the test passes.
Some say they only write such tests for nasty bugs or bugs that are hard to reproduce manually.
Some say they keep the tests forever and some say, if the bug hasn't shown up for a year, delete the test. Because if you write a test for every bug, you can end up with hundrets of them after some years, which slow the CI/CD process down quite a bit.
But the generall idea here is, design your code and add tests when it fails. Don't use the tests as a core design practice, but as a way to enhance what you already designed.
Variations of Testing
There are also some variations of the existing test practices. They could help people who don't have the experience to write good tests and don't want to clutter their code-base with bad ones.
Property Based Testing
Another apporach comes from FP. It's called property based testing and seems to allow a mix of the two TDD apporaches mentioned above.
The idea is, you statically type the code units you want to test. Then you write a test that puts some data in your code, but instead of writing it like a normal test, you write it with a property testing framework, that calculates all the possible inputs for your code based on the types you allowed.
Instad of writing tests for a few integers, you write the "integer test" and the framework generates all the integers for you. This allows you to write unit-tests for the good part of your code and adds the tests for (possible) the bad part on the fly.
Mutation Based Testing
The main idea of the tests in general, independent of TDD or BDT, is, you want to be save that if your code breaks your test should reflect that.
Mutation based testing breaks your code and looks if your tests still pass.
It mutates the units of code you want to test, removes some return statements, changes some variables, etc. pp.
If your tests don't fail, either the change wasn't critical or the tests would have also failed you with real problems later.
My Testing Practice
I'm still not happy with my testing practice.
In my last small project I tried the TDD approach and had the feeling it wouldn't add anything, but it seemed to me that some problems simply don't lead themself to unit-tests in general. Sadly these problems are the only ones left, that make my dev-life hard.
I tried to write an API client with the help of TDD, but since the client relied on the API, the tests didn't help much. I mocked the API and after all was done I felt pretty good, but when I used the client later it failed right away, because the API required some additional data that wasn't specified.
I know this isn't a problem of TDD and many people even write you shouldn't mock stuff just to get unit tests running, but well, integrating APIs is one main concern for me so I tried it anyway :D
Probably integration tests are the way to go here, but often you don't have different API stages, only the production API and you don't want to clutter it with your test calls, so mocking seemed like a nice way.
Maybe I even did everything right and the problem was the badly specified API, but as I said, often you can't choose here.
I also often have problems with React-Native UI components looking differently after library updates. Don't know if snapshop testing would help here or if this is simply a device problem? I have the feeling they only help if a library breaks a component, not if it just makes it ugly, haha.
Unit-Tests or Not?
I think: Probably yes?
I really want to like it, and many people, most who are smarter than me, love it, so I feel like an ignorant or dumb for not using it.
Top comments (18)
This is an awesome summary of what was a great discussion. Thanks for sharing!
Your description of the relationship between SOLID and TDD is super valid. Test-driving new code (or refactors to old code) is usually my first activity when working with a client who I know is struggling with issues that stem from poor application of SOLID principles.
On the UI components breaking visually, I wouldn't call that a concern for unit tests or even traditional snapshots but rather Visual Regression Tests. There are a bunch of libraries out there, like BackstopJS, that make this reasonable easy to do. There's also a PR here that outlines how to do visual regressions with jest using jest-image-snapshot.
Overall, I wouldn't feel ignorant or dumb for not being completely sold on the value of unit testing. Skepticism is healthy and I'd be much more worried about the person who just accepts that unit tests are good for them without thoroughly examining the landscape first.
Haha, case of the unknown unknown :D
I knew I wanted something like "Visual Regression Tests" but I didn't know how they were called so I couldn't search for it.
I found Storybook and Loki which seem very promising and work for React & React-Native.
Thank you very much :)
I think the reason senior level coders are less likely to unit test is because we developed our coding style prior to testing being in vogue. And it's hard to change your go to style. Regardless of what we say are our reasons
Probably.
To me it still feels like a chore, but I wish I had an aha-moment that would change this, hehe.
Try practicing TDD on something that isn't just a CRUD app. That was a huge hurdle for me at first. Why do I need to test that I got the data!?! It wasn't until I started writing code that manipulated things that I understood how unit tests can help.
I don't follow "Test-Driven Development," in that I definitely don't write "units" and their corresponding "unit tests". In other words, by TDD's official definition, I don't use it.
However, I do write extensive behavioral and integration tests. I helped author the Live-In Testing standard at MousePaw Media, which uses PawLIB Goldilocks to ship the tests with the end product (C++).
I can't speak to Javascript development, though, as that's entirely outside of my bailiwick.
A couple weeks ago I posted an article on miss conceptions of TDD here: dev.to/mrlarson2007/common-myths-a...
The bottom line TDD is great but does not solve all issues. We need integration tests and acceptance tests. Each of these tests solve different issues.
Referring to your personal experience, unit tests are really helpful with handling error conditions because it is really easy to induce all the possible errors you clould get from the API. But you still need a integration test to check the happy path. All of this is really context dependant and we often need a combination of these tests to find these issues.
In my code, I tend to unit test mainly the code that contains business logic or algorithms. And, by that, I mean all what is in
lib,app/services, andapp/validators.For the models, controllers and all the code that is simple, I usually tend to write "smoke tests" like
expect(response).to have_http_status :ok.In any case, as a deference for the next developer, I always write a test (even a really simple one) for every single class I write.
I find unit tests are mostly useful in keeping all of my function signatures modular.
At first I subscribed to the idea that I was doing tdd/bdd. But what I realized was that unit testing (and my laziness) forced me to make my inputs and outputs dead simple to pass in and receive.
I was driven by the need to make every test written in only 3 steps (or even 3 lines):
If I ever found myself writing multiline setup code (eg "initialization code"), I would refactor my function to accept data raw or write unit tests for new production code to do the work my unit test did and have my production code integrate the 2.
My general unit testing mindset is this: If I make my tests dead simple, the production code that also needs to call these functions will also be dead simple. Apply this idea recursively and you've basically made your entire code base modular. Your units will start looking like math expressions. Your integrations will have few to no branches.
I can give an easy problem that I encountered recently: Given an array of arbitrary booleans, write a function that returns an index pair where the pair indicates a range in the array that is contiguously set to true.
My first crack at this was something like this:
for each item in container:
if item is true and I went from false to true:
remember item's position as the range start
else if the item is false and I went from true to false:
remember item's position as the range end
exit for loop
return remembered pair of positions
Now I could have been done with this and moved on with my day. But when I looked at unit testing this, I kept thinking: That's a lot of cases to cover. Let's recursively break this down into subfunctions so that my test cases for these subfunctions are dead simple.
eventually my function looked like this:
range_start = find( array_start, array_end, true )
range_end = find( range_start, array_end, false )
return (range_start, range_end)
Now when I think about unit testing this "find" subfunction, I'm thinking that I just need to unit test "find". It turns out that "find" is already a 3rd party API which should have already been well unit tested.
So at this point, I realized I'd be wasting time unit testing the integration of calling "find" twice and ended up writing an integration test that exercises the overall business case instead.
This kind of epiphany in modularization is not a one-off. I've encountered this type of situation in so many different algorithms and system workflows that I rely heavily on this kind of unit testing mindset to help me break down problems into trivial compositions.
I hope this example gives you ideas on how to make unit testing work for you.
Great wrap up of the discussion about testing.
IMHO there is silver bullet in subject of testing. I use a mix of unit and integration tests in my projects. I didn't use Property Based Testing or Mutation Based Testing. Maybe there will be opportunity for that in next projects.
Generally use Unit Tests and TDD when I'm writing some business logic or piece of code which is or could be easly separated from other parts of the system. When it's dificult to separate the code, for instance in the case of legacy systems, I prefer integration tests. I try to estimate which approach is the best in particular case based on my experience (of course sometimes I make wrong decision but that looks work of a software developer).
Thing I always try to achieve is the automation of tests. In my opinion tests which aren't triggered after every build (unit tests) or before release (integration tests) are worthless because they don't bring us the information that something is wrong.
Wow, great roundup K!
Thanks :)
I'm a bit late to the conversation, but I wrote a blog post recently (partly inspired by this thread, partly inspired by a lot of similar conversations I've had with others): Tried TDD and didn't realize the benefits? Try it the next time you get writer's block.
For me, I didn't really start to "get" TDD or unit testing until I started using it to help me when I was completely stuck with "writer's block" or overwhelmed by a feature I was working on.
Being able to break down code into smaller functions is often touted as one of the best reasons to practice TDD (and unit testing in general, even if you don't write the tests first), but I think what's usually missing from that conversation is why you'd want to do that, even if you know that it makes code easier to read or refactor. While readability and refactoring are great, I think they can seem a bit "abstract" sometimes, especially when you're really focused on just getting something implemented.
And while the usual reasons to do TDD are great, I've found that they can often make it seem like "something you should do if you want to be a good developer", and that's not as healthy a way of looking at it as using it as a tool to help yourself.
When I started thinking about how I could use TDD as a way to help myself, I began to use it more, understand it more, and got better at development.
A very nice summary indeed and as this topic really interests me - here are my 2 cents.
From my personal experience TDD is just a tool that helps you write your code in small bite sized chunks. This tool is amazing when you try and work out some complex piece of business logic.
That said just having the unit tests is not enough and a brilliant talk that I have seen around TDD is "Ian Cooper: TDD, where did it all go wrong" vimeo.com/68375232
There he argues the idea that once your unit tests are written and green there are not safe from refactoring too, and that sometimes deleting the tests is a right thing to do, or refactoring those into a higher level test.
In modern day development especially if you do any sort of serverless development - you can have the best unit tests out there but they will unlikely to catch most of the issues you will experience: for example not giving right permissions to your function - something that is impossible to test on a unit level.
I personally also have not totally nailed it down and keep on getting challenged with how to best apply TDD as the more I work in the field - the more I lean to higher level tests - especially integration.
I have a question, the unit tests should be written by the dev? or by somebody else like QA, tester, PM or peer developer? will it save time to the developer if tests are written by somebody else? so the developer can focus con the code? At the end wasn't just one, sorry XD
Depends on your philosophy I guess.
If you wa to break as much as possible, to find all the bugs, it's better when someone else writes the tests.
Unit testing can be a pain sometimes. I generally don't unit test repository code. But some integration tests can help there.
Unit tests really shine when you have a method that takes in a few parameters. Easy to test.
That method that does all the things and is hundreds of lines long, not so much ;)
Great write-up, first off!
Secondly, testing is a tool for ensuring software quality. You don't want to have fear when changing things around, and testing helps that. It's not a catch all. Some things don't need to be unit tested - empty destructors, overloaded functions that map to higher arity versions of that function, etc. Use your testing hammer to ensure that array accessors fail only when you expect them to, sorting algorithms actually sort, etc.
Thirdly, some languages eliminate whole classes of errors. For instance, you're never going to have a NPE in Haskell because... there's literally no type for NULL/nil. Make sure that you're only testing spots that your language won't eliminate for you.
So in short, definitely test... but not on things where it doesn't make sense, and a lot on the things that do! Enough to make you feel secure enough that you can change implementation details, but not so much that it feels like a chore.
Looking forward to hear more about your journey through testing philosophies!