I always assumed that unit tests are just part of the job. They are just there, and it's a no-brainer to always test every new piece of code that you add to the codebase.
Until I recently stumbled upon a guy who leads an engineering team with a radically different approach. They refused to write even a single automated test for their entire production application.
You might say, "Well, this guy is just fooling around with his 10 users."
But when I dug a little deeper into the discussion, I found that there are teams at Facebook, Twitch, and Netflix that followed a similar strategy.
So what's that all about? Should we all just stop unit testing from now on?
✅ Supposed Benefits of Unit Tests
You can probably find more, but these benefits are mentioned over and over again:
Unit tests help you to:
- find bugs early on before you commit them to production,
- catch bugs when your code changes,
- improve your code design,
- document how your system works.
If we assume you are not writing garbage unit tests, these benefits are all reasonable. But they come at a cost.
💰 The Cost of Unit Testing
Unit tests are expensive to set up and maintain.
The question is not whether unit tests are beneficial but rather if these benefits are worth it.
Developers constantly need to think about how to design their tests in a meaningful way, how to mock stuff, how to get more code coverage, and whether they covered all potential edge cases.
On top of that, when parts of the code change, they might have to refactor dozens of tests, no matter how well they were designed in the first place.
And all of this can lead to engineers becoming very shy to touch existing code that has been cemented in place by excessive unit testing. They tend to avoid rethinking existing solutions, trying out new things, and being bold in general.
🦧 What can you do instead?
1. Use a strong toolset
You can catch a lot of stuff in advance with a strong toolset. Code reviews, linters, strict type checking, and some solid coding guidelines provide a good basis to produce a robust product.
2. Manual testing
If you do need to test something, test it manually by hand. Yes, you will repeat yourself, but you are probably closer to the production environment and can easily test scenarios that might not even be testable automatically.
3. Rely on users to find bugs instead of engineers guessing them
It's likely you are already addressing the obvious stuff. But most of the bugs you'll face come from edge cases that you couldn't have guessed in your wildest dreams. If you accept that code will inevitably break in production, your focus will shift more towards how to fix it fast.
4. Add safety nets
Speaking of fast fixes. Focus your effort on creating safety nets to detect and address bugs as quickly as possible. These include gradual rollouts of new features, using good bug and performance monitoring tools, and feature flags to quickly roll back in case anything fails.
5. Write stuff down
If you run into weird edge cases, leave a quick comment in the code to explain why stuff is in there and discuss it with your teammates. That way, you can remember everything when trying to refactor pieces of your code.
6. Code is written to be rewritten
Establish that in your team. You will never get your code right on the first try. Encourage frequent improvement and don't let anyone shy away from touching someone else's code.
7. Work on your code, not on the supporting structures
If your code is fragile and keeps breaking when you add or modify parts of it, focus your energy on making it more robust instead of making the tests around it more complex.
🤔 So, should you ditch your tests?
There are cases where the cost of a single failure is simply unacceptable. Take banking, for example, or critical infrastructure. These systems need to deliver 100% of the time, and it's just not worth messing with that.
Also, at a certain scale of a company, unit tests will for sure be worth it. When there are too many people involved and things keep constantly breaking.
But this point comes later than you probably think. Netflix apparently went to 100M users without a robust testing suite in place (Ex Netflix engineer on unit testing).
So if you are a startup or medium-sized company and your priority is moving fast, then you can probably get away without unit testing for longer than you think. And going even further, it might even benefit your company because you can focus on setting up processes to fix bugs faster.
What do you think about unit testing?
EDIT:
Going without automated tests is not an excuse to go wild and crazy with your code. The time you gain needs to be invested into alternative failsafe mechanisms. It's like driving a sports car in race mode to make progress fast, but you better know what you are doing. It requires you to be always alert and you need to feel comfortable with running into some road bumps along the way.
Side notes:
If you want to dive deeper, check out this interesting discussion on the topic:
https://www.youtube.com/watch?v=pvBHyip4peo
Latest comments (64)
You don't have to listen to me or anyone else with more than 3 years of XP working at multiple companies from decades old established organizations to startups: this attitude will always lead to a software system's downfall. You are going to pay for it severely later.
Failure to write unit tests is laziness, I see it in younger engineers and old-timers on their way out who simply want to do things the way they always have. Unit tests are seen as a chore. I admit it's not glamorous, but it has saved me in the future when I made changes thinking everything is happy, only to find out that wasn't the case.
There is always a significant portion of software that you can unit test without going full "Enterprise Java". You will not write unit tests for everything because the verification requires a production environment or near simulacra (requires network, OS interface, file system, etc.) and abstracting EVERYTHING for the sake of unit testing nets minimal gains at high cost, but you would do well to automate what you can, including integration tests. My experience tells me that if you are still around after decades of manual-only testing, it's because you have cornered a niche market with no competitors (yet).
I write quality unit tests so that I can refactor code safely. Code without tests is legacy code.
Sounds like an excuse to write bad code. You're basically saying that testing is hard and the users will deal with your bugs for you. That's a horrible attitude. Unfortunately, large companies like FAANG can afford it. People will continue using them even when their software is bad. For those who still need to build trust with users, you do not want to copy this attitude. Test your code, the time it costs upfront will save countless hours on the back end. This is true even at FAANG, unfortunately it's much easier to hide those costs in a larger organization. The manager that finishes more projects will get the raise faster, than the one who saved the company a few hundred million in losses from software bugs.
First, I thought that the huge list of things to do as an alternative to unit testing was, in fact, part of a joke to end saying that unit testing is the faster way. But you are really convinced against unit testing.
If you have enough knowledge and experience with testing and TDD, you don't make coupled tests, or tests too hard to maintain.
There are many books to get help from about creating good tests, from Kent Beck, or James W. Grenning, to mention the best ones I've read.
I work in a startup where we are building micro saas & never introdcued units tests in our Codebase. It totally depends on your product & its customer base.
I work a lot with Nuxt (a framework atop Vue.js) lately. I barely find valid cases for classic unit tests here. Often you "only" fetch data from backend and display them. Or collect data from a form and send them to backend. This can't be unit tested, this has to be E2E tested. Okay, maybe I can unit test for example getters I write in my Pinia state stores, but they are usually so straightforward that you just see the results immideately during development and once estsblished, they are quite unlikely to change. So in this case having to write dozens of dumb unit tests that just
"test" whether you can use
Array.prototype.filtercorrectly is quite a waste of time, if you ask me.I recently came across the concept of "visual testing" with Backstop.js - it opens your app in emulated browser, performs defined scenario (visit route, click a button, etc), takes a screenshot and stores it for future reference. If you mess something up during development, the test starts failing, becuase actuall screenshot looks different. I think this is good way for quite complex and yet easy-to-manage (because you dont have to describe the outcome) testing. On the other hand there is some overhead as you have to store the screenshots somewhere.
Very interesting approach with that screenshot testing! Thank you for sharing!
I believe that unit tests are written poorly the majority of the time -- they should test the functionality of the code, not just the happy path, but most of the time I see the failure path(s) ignored because the code coverage threshold had been met. This makes ensuring the signature of that unit less reliable and any refactor will most likely miss preserving it.
I find the idea of testing in production behind flags or certificates a better solution to the problem. You write the integration and end-to-end tests against that system with known dummy accounts, and those tests should run often enough to avoid breakages. Once a feature is passing the tests and has a level of completion to release the flag can be removed.
This could be something that a more senior developer could setup and maintain, while the business code can be done by the entire team. Having this in place would ensure that good test coverage that is aligned with the requirements.
My thought is if your application doesn't have a good documentations then you must write the unit test cases of every functionalities.
It helps you to prevent the bugs in the dependent modules...
I have long considered observability to be far more important and impactful than unit tests.
Cool article.
Explains why my Netflix kept crashing
Some comments have been hidden by the post's author - find out more