You Don't Have to Test All of Your Code...

#testing #cicd

Earlier this year, I noticed a sign in my dentist’s office that said, “You don’t have to brush all of your teeth, just the ones you want to keep.” I smiled. I like this saying for many reasons. It serves as a good reminder that all values require choice (if you want to keep your teeth, then you should brush them). It also reminded me of a recent conversation with a tech leader who wanted to get her engineers involved in testing. “Which parts should we test?” she asked. As I stood there in the dentist’s office, I said to myself, “Test the code you want to work.”

On its own, it’s a bit pithy. We can expand on the idea. Let’s look at some ways we can get from zero to fully tested.

The finish line

Teams that self-test release higher-quality software faster. This is what DORA provides evidence for year after year. It’s not a tradeoff between speed and quality; it’s both.

To ensure quality, we want fast validation that our software isn’t broken. To this end, we want our tests to run as part of our build pipeline, giving the team immediate feedback when something is off.

In fact, my typical pipelines are dominated by validation. Even before I’m through with my DEV environment, look at all the validation steps I have:

Each validation step is designed to give fast feedback. The faster the feedback, the earlier the step is in the pipeline. Linting and unit testing are lightning-fast; they run first. Integration and acceptance tests take longer to run and are executed later.

The constraints

Some of the following constraints are metaphysical; you cannot escape them, while some are practical. All of them are good. The best performing teams I’ve encountered use them.

Time/Money

Engineering teams never have enough time to do everything they’d like to do. This is normal. We have to carefully choose what we spend our time on. Your tests should be pushed down to the appropriate layer (unit/integration/acceptance). Sometimes this means you don’t test everything.

Authorship

Your engineers should write the tests. Or, put another way, whoever writes the test should write the code. Your test code should be as ruthlessly clean and organized as your application code.

Additionally, your engineers should own the build pipeline. They don’t need to own the platform the pipeline runs on (e.g., GitHub Actions or Jenkins), but they should control the pipeline’s structure and be able to make changes to keep it fast and functional for the applications they own.

Mockless

Do not mock things you own. This goes for classes, modules, and services. Mocks are harmful. You may consider mocking stuff you don’t own, but only if necessary. This is (currently) an unpopular viewpoint and will be the topic of a forthcoming article, so hold on to your pushback until then.

The starting line

If your team is writing its first test right at the beginning of the application’s development, congratulations. You are on the path to success from day one. This is the best position to be in.

Unfortunately, many teams have started development with either no testing or with an external team handling testing. For all of them, they are now behind the pack and need to catch up. What to do?

The race

Write a test.

The best time to write your first test is today. The next best time to write your first test is tomorrow. That is, start now; it never gets any easier.

What should I test? If you are building a new feature, test that feature. If you aren’t in the midst of a feature, pick the easiest existing feature to validate and test that. Building a test suite is hard at the beginning, so start small and build more tests as you gain confidence.

Won’t this slow me down? Yes. And you will make it all back as you go forward. An F1 driver will take a few seconds away from the race to make a pit stop and change tires so he can finish even faster than he would have without the pit stop. It’s an investment toward winning, not a sacrifice.

The strategy

Each new feature or bug fix brings a corresponding test (or tests) to validate it. Slowly, the percentage of self-tested features will increase.

You can backfill missing test cases as you have time, or you can wait for a problem to emerge before covering it with a test. Both are viable strategies. The risk on the first is that you test something that may not break or has little impact if it did. In other words, you might have wasted your time. The risk on the second is that, while you now know this is an excellent place for a test, the customer who found the problem may be upset.

Write the test first. This helps reduce the scope of work and keeps you focused on the task at hand. Once the test goes green, tidy up and push to master. Which leads us to...

Practice continuous integration. Merge your changes into master at least once a day. Stop using feature branches. Continuous integration is closely aligned with a test-first methodology; they work well together. Again, this is (currently) an unpopular viewpoint that deserves an entire article. I will have more to say on this in future writings.

The payoff

Better software faster. Who doesn’t want that?

Testing plays a role in both the “better” and the “faster.” You are better because your automated test suite prevents you from backsliding (breaking existing functionality) and, therefore, maintains the application’s high quality. You are faster because your tests are automated and part of your build pipeline. And, because your engineers own the code under test, the tests, and the pipeline, they have both the incentive and the capabilities to keep it running.

Once a team achieves self-testing, it will start running circles around teams that don’t. And from personal experience, it’s a whole lot more fun to build this way.

Happy building!