loading...
Cover image for Do Your Tests Wobble?

Do Your Tests Wobble?

arleneandrews profile image Arlene Andrews Originally published at distort-roving.blogspot.com on ・5 min read

Cover image by delo from Pixabay

“The green reed which bends in the wind is stronger than the mighty oak which breaks in a storm.”
― Confucius

~

I was asked to pair Blog with Lena Weiberg - much to my shock and delight. This was the perfect opportunity to hear another point of view on something, and share it with the wider community. Thank you so much, Lena: it is a delight reading your words, and even better working with you! Please take a read of her look at this topic at https://testing.pejgan.se/2020/06/13/reacting-to-change/ - I think you'll be delightfully surprised.

~

Part of being a good tester, to me, is making sure I stay aware of the trends in development. Knowing how the team is working can help get comments and questions to them at the proper times, and has the information that can be acted on. In this quest, I attended a recent online conversation where the topic was primarily on the size of 'steps' taken in software development: making sure each step takes a reasonable amount of time, that it leaves the system in a state that isn't worse than it was before the change, and that could - if needed - be deployed at this point.

The concept of “wobble” (I'm not sure who originated the term)- how much the entire system is going to react to a change in one area – was presented, and caught my attention. Thankfully, this was near a break time, so I had a few moments to explore what this means to a team. And found several ways that this might change how they look at both ‘done’ and stability in a system. Does this fit the model they are using? And is the model wrong in this way?

And I wonder how much this concept has shaped the 'normal' path software goes through. Is this one of the reasons that testing has traditionally been placed at the end of the process? Either by those writing the code itself, or as a chance to make sure that a release has what it needs to be able to get into the user’s hands?

I will use the example from the quote above: a clump of reeds with many points of contact – intentional and otherwise – is close to a flurry of tiny changes. This could be produced by having any change instantly sent out to everyone. So a button could shift colors several times in one day, or flip into a different position – as a deliberately extreme example. None of these are going to ‘break’ the system, but would certainly irritate me!

Or it could be more like an older, stiff tree: reacting quickly to a potentially damaging threat just isn’t possible. Large changes that appear with infrequent updates happen, but it is a shock to the user's workflow. They may have to puzzle out where the information is now accessed – and if the frustration of finding and relearning the ‘tree’ is large enough, you may lose that user, or a lot of them!

I mulled this over for a few hours, and got this clarified in my mind. When a system changes during coding, it should be easy to see how those changes influence nearby code, as well as the code it touches. This is a normal thing for many developers: it allows them a chance to make sure that the system isn't going to collapse. A change of something like a variable name will cause waves across the application - but this will settle out unless there is a conflict. Changing the way data is collected or displayed may cause more reaction, and might lead to a cleaner code base. This, to me, is an area of little wobbles: things are deliberately changing, but we minimize the wobble to avoid future work.

Deliberately causing a large wobble is something that should be thought over carefully. Making sure the team knows of potential questions and issues, and a way forward to resolve them opens a chance to take time to look at the entire section. The amount of influence on the system as a whole is one consideration for how carefully you need to look. Changes like this most often are without problems, but they may infrequently cause damage, or at least confusion. And since the code has changed, there are the test and automated checks to update, and add to - no better time to make sure that this section of the code has more coverage than it did.

A bunch of small changes can add up to a lot of 'tech debt'. It is very easy to lose focus, in this situation, on the larger change we are wanting to make. Changing the name of a variable (a legitimate change, and a 'step' of the proper size) shouldn't set off a flurry of tests. The 'flow' of the development would be disrupted, and cause even more wobble. But always waiting until the very end of the process might cause large wobbles, if a functionality is missing or changed beyond recognition.

Our tests themselves suffer from wobble. Exploratory testing, in some ways, seems to be built on the concept of wobble, and where it will take us. The tools we use to help find the errors – even something as simple as a set of tester notes – can suffer from changes that might appear simple. Those changes may have an effect on more-distant parts of the application, and don't get captured in the original set of notes on an area. Future testing and checking might be incomplete, and adds to the potential of missing things and ‘tech debt’.

Our developers have a focus on getting through the application for the specific use they see. Users, however, will very likely not follow this path, at points - or at all. Letting the system wobble may save hours of coding updates, even if it seems to be needless duplication in finding alternate ways to get to information. Ensuring that all of these ways work to make the user’s experience easier is not always the priority, but it should at least be thought about. This helpful wobble may delight someone who has been frustrated by a less flexible “happy path’ that other solutions offer.

Wobble, by itself, isn't bad. Reeds bend in wind, and the resistance they gain makes them stronger. Allowing the system to wobble a bit will help reinforce and guide future changes, rather than allowing the system to be inflexible. Change will happen, and the whole team is responsible for making sure that this is a positive thing.

Posted on Jun 13 by:

arleneandrews profile

Arlene Andrews

@arleneandrews

Recently found the world of testing and how it fits into the system. I still enjoy coding, but I fit better in the testing mindset.

Discussion

markdown guide