This was originally published on my blog, https://unremarkabletester.com/.
Recently, I was asked to give a 15 minute introduction on Quality Strategy in DevOps for our upcoming "Efficient DevOps with SAP" course and I happily accepted. And then, to my horror of horrors, I realised that I needed to figure out what a Quality Strategy is and what makes it unique in a DevOps world.
Instead of trying to come up with a definition, I started considering what are some quality specific aspects within the CALMS framework that would help a DevOps transformation.
So, here are a few ideas on the Culture, Automation, Lean, Measurement and Sharing aspects to consider that might improve the overall quality of a system, from idea to end user.
Culture
When it comes to the online narrative of Quality in DevOps environments I see two major changes in mindset. The first one is that when we talk about quality we shouldn't narrow the discussion to functional testing activities. There are more things that affect the quality of our product, such as for example a well-thought-out user experience. Once we get beyond the Quality = Testing mentality, it is easy to see the second required change. And that is that quality is everyone's responsibility. Releasing a quality product is not a final check but a continuous state of mind for every task we work on. More often than not, this is easier said than done of course.
Automation
Automation is the part that gets the most attention in any DevOps transformation. When it comes to quality, automating various functional checks increases confidence in the system’s expected behaviour. The time that automation saves us can be used to explore unexpected behaviours within it and look into ways to anticipate them better in the future. But the automation we build also needs to be of good quality. Having flaky tests that nobody trusts is something that anyone who has been bitten by it painfully knows.
Lean
Including Lean aspects to improve the quality of our product might appear obvious. For example, a reliable, time efficient, compliant Deployment Pipeline where everyone is responsible for having it in a good state to reduce handovers and "red tape" seems like a pretty good idea. But even if your team has mastered DevOps maestro levels, if your neighbouring teams still struggle with releasing their changes in good quality then the quality of the whole product suffers. The discussions need to focus on continuously improving the quality of the entire system and not just that of the individual parts if we want to remove bottlenecks and constraints.
Measurements
A common quality metric is the number of issues opened by the customers. But what about users that can't find the proper way to report a bug? What about those that instead of complaining just stop using a product? What if instead, we measured our users' satisfaction and the net promoter score they give to better understand the quality of our product? We can make a hypothesis of how we can improve, implement it and try asking again. And implementation doesn't necessarily mean building new features. Sometimes it is removing unnecessary functionality, sometimes it is renaming the buttons to something more meaningful. A small bug might be forgiven and not be considered a threat to the quality of a product but the inability to do the work you want on a constant basis might be a bigger problem.
Share
Sharing knowledge seems to be an intuitively good practice, but implementing is not as straightforward as one might think. And that is because it implies investing time both for the side providing information as well as the one consuming it. For example, we can say that we share testing tasks. For that to become a factor of improving quality, testers need to take the time to show their team how to do it efficiently and the team needs to take the time to learn how to do it properly. Sharing is good to spark the flame for something new but to keep it burning the oxygen of time investment is needed.
So what is a Quality Strategy and how does it fit in DevOps environments? For me, in the end, CALMS is a Quality Strategy. It is a framework for improving the value for the end users of a product while optimising the process of creating it. This might not be a perfect definition or by any means a definitive one. But it is one that I can work with in the future.
Top comments (0)