How many testers should ensure Dodo IS high quality

#testing #qa #developers #softwaredevelopment

How many testers should ensure high quality in an IT company? I have been asked this question so many times that I have finally decided to answer it.

First of all, I have to admit that asking "How many testers does a company need to ensure high quality?" is too abstract, because there is no right answer. Anyway, I did a little bit of research and found some tips such as "one tester for every three developers". Such answers are very similar to tips like: "You should drink 2 liters of water" or "You should take 10k steps per day". These tips apply to everyone and to no one at the same time. There is only one correct answer to this question - "It depends.

Let's have a look together at what influences the ratio of testers to developers.

Critical products, where people's lives, safety and health depend, need more testers. Your ratio of testers to developers needs to be higher if you are developing a software for self-driving cars than if you are developing apps for selling pizzas.
Level of developers and testers. If you have three senior developers, a junior tester is not going to take over the tasks of those three developers. And the other way round, a senior tester is over-qualified for a team of three junior developers.
The culture of the company. If there is no culture in the company where all team members are responsible for product quality, but developers are only responsible for writing code and testers are responsible for testing this code and product quality. You need more testers than a company that has this culture. By the way, I have no idea how testers who don't write code for a product can be responsible for its quality.
The size and the complexity of the product. If you have a huge and complex product, you will need more testers than if you have a small and simple product. But remember that without deep knowledge of the product and its context, it's easy to underestimate the size and complexity of a product.
Process maturity If the whole team is responsible for quality and developers can test the product themselves before the tester starts testing: In this case you only need a few testers. You will need even fewer testers if the developers write not only unit tests, but also integration and end-to-end tests. You will need fewer testers if you have good monitoring, canary releases, or other ways of detecting problems quickly.
How to deploy. If you have a complex way of distributing your products, e.g. software for cars, and your customers need to visit the service center to get their software updated, you will need more testers compared to a company that only updates its products online.
The automation potential of the product. If your product is difficult to automate, you need more testers. Consider a company making printers. You can cover drivers with tests, but it's hard to automate integration of hardware and software. I really don't know how to automate the next case: Checking the text printed on the paper. Maybe we can use computer vision. But how much does it cost? And what about the ROI of this job?

Speaking of how many testers we need to ensure the high quality of Dodo IS: Let me give you some examples.

We have two teams with two developers and no testers. These teams own non-critical services. These services are mostly covered by unit, integration and end-to-end tests. These services can be rolled back in a few minutes. The developers in these teams can test the services themselves. Sometimes, before they build some features, they call in testers to help them with acceptance criteria.
There is another team that has three developers and one tester. This team owns our system's core service. But the engineering culture in this team is high: fast releases, fast rollbacks, and high coverage by tests - after every problem in production, this team will create a new set of automated tests that will cover that problem. Great monitoring and alerting allow testers from this team to not only test their own features, but also do a lot of work for the QA guild.
There are a number of teams that consist of 6 developers and 1 tester. Those teams develop mobile applications. Here the situation is a bit more complex. The developers are not as active in creating automated tests and in most cases rely on the testers for all quality and testing issues. For mobile apps, the cost of failure is higher than for web apps. You can't easily roll back your app on customers' devices. But for such teams, one tester is still enough because they have two backend developers, two Android developers, and two iOS developers, which means this team can develop two features simultaneously instead of a team of three fullstack developers developing three features simultaneously. That is why a single tester will be sufficient for this team.
And we have another option: two teams (7+3 devs) and 1 tester. In this case, one tester is enough, because the devs can partially cover some testing tasks. But there is one thing that might be unpleasant: the tester has to be part of two PBRs, two daily scrum meetings, two planning meetings, etc.

So, as you can see, there is no right answer in general. As far as I'm concerned, I'm sure that every two-pizza team should have at least one tester/QA. This person provides testing and QA expertise, but testing and creating autotests can be done by the whole team. Then review the above criteria and determine whether additional testers are needed. Now we have 22 testers/QA and 34 development teams. We haven't reached my absolute minimum number of testers. But we're moving in that direction. What is your ideal ratio? Or what kind of unique ratio have you found?