DEV Community: José Peñaherrera

Quality Strategy as roots inside a software development team

José Peñaherrera — Fri, 14 Apr 2023 15:09:05 +0000

The most common structure inside a Software Development project is a development team (DEV), quality assurance team (QA), business team (BA), and project management team (PM). As you may know, ideally a team is conformed around 5 to 10 team members and, something really common in this kind of team, is that we have people from around the world with different ways of thinking, expertise, and culture, so, one of the biggest questions is, how we are going to align the team on getting the things done? The easy answer is: with a team-defined Quality Strategy (QS).

To be able to define a good QS with the team there are several things that we need to understand first:

QS is not the same as a Testing Strategy or Testing Pyramid
QS is a Critical Quality Agreement between the people that are going to be involved in the team
QS is a living document that is guided by a QA champion but it can be updated for any team member

Now that we have a basic idea of what is a QS we need to think about “Which are the critical quality agreements that will help my team?”, based on this, we will discover that this document doesn´t have a common structure and that it will change based on the team necessities but, we can define some usual topics that are discussed in here, for example:

Ways of working (Agile practices that the team will perform)
Effort Measurement
Path to Production & Roles and Responsibilities
Definition of Ready
Definition of Done
Risks
Testing Strategy
Bug management process

As you may notice, these topics are open to having their own article but we will try to give you an idea of what all of these means:

Ways of working (Agile practices that the team will perform)

The most critical alignment that we need to get an agreement in the team is the development methodology that we are going to use inside our team, the most common is to use Agile practices, as the name says, agile means that we are going to define what are the best practices that we can add on our daily work and how to adapt them to our necessities, for example, we can define that we are going to perform 20 min daily, we are going to have Sprint planning every 3 weeks because the Sprint will last 3 weeks long, we can define things like Retrospectives, when to execute a postmortem meet, which will run the daily, when to do a demo and so many things that will guide us based on how we want to work and the availability of each team member.

Effort Measurements

This agreement between the team is about, how we are going to estimate the effort that will cost for the coming tasks, the most common way to measure the effort of a story (in agile practices) is with Story points, which means that we are going to define what does a 1, 2, 3, 5, 7, etc. Represent inside our team, also, need to understand what will be the best approach when we have a task with points like 13, we can try to split them, or let 2 team members work on that task.

Path to Production & Roles and Responsibilities

Here we are going to define how we plan to get things done inside our team, usually, this takes many hours that can be split into different sessions, most of the common things that we will define is what is the lifecycle of a story inside our team, we define things like where a business necessity could come from (business new requirement, stakeholder necessity, improvement the infrastructure, code refactoring, technical changes), who are the responsible to get the necessity transformed into a story (for example, tres amigos method to have three different points of view and try to get the most of the considerations), how we will construct our kanban board (here we can also consider the different testing environments), which columns we are going to define in here, what are the process to deploying the new develop or fix to prod (CI/CD) and how we are going to let the business know about our progress, we need to consider that, inside on each step that we are defining we need to have the responsibility of them based on the roles that we have inside our team, also, is a good idea to get the expectations that every teammate have on the different roles that the team is conformed so we can properly be aware to who is the champion of what specific task

Definition of Ready

This concept is used to understand when a specific task is ready to be picked and developed by the team, we can have our definition of ready that the task has already been reviewed, approved, and estimated by the team so it is ready to be added to our next Sprint in the To-Do column, also, we can add a Story structure that will help to guide the team, we can add thing like Purpose of the story, Examples on how the system should behave after the development, Out of the scope, etc. We can also keep in mind things like our Kaban board and the different testing environments, we can define what means for us that something is ready in QA environments, or in UAT environments.

Definition of Done

The difference in this concept is that here we can understand when a task has completely gone through all of our development processes, has been developed, tested, accepted by the customer or stakeholders, and also is already deployed on production, of course, this applies when we develop something that will be deployed in prod, we can also develop artifacts or libraries to be used for the organization itself so things like being approved or deployed can be modified.

Risks

This is more like a knowledge-based thing to understand what are the risks that can affect our deadlines, for example, common things that need to be considered are holidays, sick leaves, changes on the task, dependencies on other teams' development, etc.
The main objective is to recognize what can affect us as a team and how we can try to mitigate this to make us more reliable, always keep in mind that a risk is something that we can just mitigate but not remove from our project.

Testing Strategy

We always need to keep in mind that testing is not performed to catch bugs, is created to prevent them before it arrives in our system, here we can define with the team what are the different testing practices and methodologies that we are going to apply to our project, we should always try to use the most common testing approaches: static testing and dynamic testing.

Static testing is related to performing tests without using the system, the main question is, how can we test that something is working without using it? and the answer is with documentation, one of the testing principles is related to having working software over extensive documentation, the word “over” doesn't mean instead, so we also use documentation in agile but with a different shape like stories, bugs, epics, etc. One common approach to static testing is the Shift Left testing approach, to start from the very beginning of the concept of the business necessity that will be transformed into a Story or Task and analyze that we have added a common understanding of what needs to be the expected result, for example by using the Specification by Example method. Keep in mind that these static tests can be performed by anyone on the team.

Dynamic testing is the most common method performed by all the development teams, usually, some different testing approaches that we have here are used to define the testing pyramid that the team will use, the two main techniques are White Box Testing and Black Box Testing

White box testing is performed by the dev team because they have the knowledge about what is happening inside the code so they can provide unit tests for each module and/or integration tests between the different modules and define the percentage of code to be covered with this type of test to give us the assurance that, internally talking, the functions and methods are not being affected by new or refactored code.

Black box testing is performed by the QA team to validate that the new or fixed code matches the functionalities that we have defined in the stories, we can perform functional tests for this, also there are more types of testing that will help us to confirm the correct behavior of the system in different types of level, we have end-to-end tests (to confirm that the whole path that the teams develop is working as expected), exploratory testing (using a good knowledge of the system behavior, understand what are the code changes to review if any other functionality was affected by it), user acceptance tests (to confirm with the stakeholders that the system is doing what they were expecting), smoke test (to confirm that the new deployment hasn’t affected any of the main or most used functionalities by the customers), etc. We can also consider that we can add automated test scripts to avoid having a huge amount of repetitive manual tests and improve the performance and time of the QA people

Bug Management Process

This helps us understand what is considered a bug and how we will resolve it, usually, a bug is something that we found on production, so we need to report it to be analyzed, and based on that we can create a ticket, a common structure to report a bug is:

Steps to reproduce it (with images or a video)
Current Behavior
Expected behavior

We need to consider that, if we find something that is not working correctly but is still not in production, means that we can negotiate with the developer to move the tasks again to the To-Do column so we can discuss with the team what is missing to define in the ticket.

As you can see before, a Quality Strategy provides critical agreements that teams usually tend to take as granted that everybody has the same ways of interpreting things thus the team will have the same ways of working, is a good practice that any of the QA takes the lead on defining all of these aspects inside the team so we can have a common understanding between everyone and avoid having disagreements with processes that are already in place, also with these agreements embodied in a document, we can be sure that any new team member will be able to understand and adhere to them, so keep that in mind, this document is handled and maintained by the whole team, we do have a champion, but this person is not the one who will decide all the QA aspects that need to be defined, and last but not least, remember that this is a living document, this will change based on team changes, team new members, business necessities, etc.

Continuous Deployment for QAs

José Peñaherrera — Tue, 02 Aug 2022 20:51:00 +0000

We have heard a lot about Continuous Integration and Continuous Deployment A.K.A. CI/CD but, how and why do we, as QA people, help in these settings on our project?

First, we must understand that the quickest and most well-defined answer will always be: it depends. All the settings of a project will be accordingly to how the team and the business work, so we, as QAs, the people who perform different kinds of testing on the lower environments and the production environment must help our team in connect the business necessities with an approach that assure us that we are reviewing what we need, in the required environment with the required changes. In practice, Continuous Delivery means we deploy the new changes as soon as we have them approved in the repo and we can do this with an answer that is simple to type but hard to set: pipelines, we do not focus on a specific tool for this, we need to focus on the general concept of “How can I guarantee that I am testing (functional or non-functional) the right environment with the latest updates?”

What has helped me a lot is understanding our application and not just the business side, but also how the deployment cycle is working, we usually have more than one testing environment, let's use as an example these environments:

Test
Stage
Preview
Prod

We have so many of them because we use different settings and connections between these environments, let´s say that Test will have a connection with the testing database. Still, Stage has the latest changes from the API team that we don´t have on test yet and Preview consumes similar APIs and databases as Prod so, we can do performance tests without interrupting Prod behavior. With all these common configurations on our testing environments, is easier to realize why we need to understand our pipelines so we can set different types of automated tests on them based on the system and necessities, for example:

We can create an accessibility test for the Test environment.
We can add e2e test on Stage because we know that it always has the latest changes on the APIs.
We can execute Load tests on the Preview environment as this use similar services to Prod.
We can add a performance test on Prod to confirm that the User Experience will not be affected (of course this will be after deploying the changes to not interrupt the pipeline flow)

As you can see here, we have different types of testing for the different environments, and you are wondering, how can we handle this in the pipeline? Well, there are several approaches that we can use for solving this, a DevOps engineer will help us more in this configuration but one that I really like to use is splitting, having one pipeline for each environment, and setting the YAML file based on each of them, we can also connect our repo to execute the task for all of them but Prod at the same time, these means, when we do a PR on the development branch we can trigger the pipeline for Test, Stage, and Preview and have the master branch to trigger the Pipeline for Prod

As I always tell you, this is the experience that I got from working on different types of projects and are patterns that have helped me and my team, if you have any comment or suggestion please let us know in the comments below.

Mobile Testing

José Peñaherrera — Mon, 03 Jan 2022 21:27:30 +0000

Have you heard before about mobiles testing? This is a completely new world for the testing methodologies and the testing techniques, I feel that this will open your mind and let you realize that the testing world is so extensive and interesting, so with nothing else to say, let's begin!

We first will start that we have multiple kinds of apps for multiple kind of devices, for example we can have:

- Native apps: This means that the app was created and can be installed in a specific mobile operative system.
- Hybrid apps: This feels like and app but is a website packaged on a native wrapper.

With this in mind, let's say that we have a Native app that we would love to automate it testing phase, usually for web development, we try to prioritize the browser and it's version when executing our tests but, for mobile, this is more difficult to define because we have so many testing approaches and things that we should validate based on the app functionality. Some different tests that we can execute for a mobile app are:

Devices that will support the app (we don't have an standar device)
Sensor and device peripherals.
Availability under various network conditions.
Installability
Compatibility
Performance efficiency
Usability

As you can see, there are so many things that we can consider to reach a complete mobile test suite for our app, but the first thing that we should review before starting should be define the region in which our app will operate to search for a average mobile device with an average android/ios (using the most used ones as example) operative system. Now that we have a starting point, we can start developing and testing our work, the thing is, as I mention at the beginning there are so many considerations for this kind of testing, like on starting phases we can use simulators for simulate our app behavior but, with the project growth, we should also start doing our test in real devices in real locations to have a more real test environment, for this kind of approaches there are several companies that sells this service of connecting with real devices to your application so you can execute real test cases, also, if you want to go deeper, you can leave your building and test the app on the street! Using real 4G or 5G connections, real time location, etc.

Now that we have an idea on how a mobile testing procedure can escalate, we should review again the different mobile testing types that we can apply based on our app main functionality:

Testing in different displays ().
Test Devices Input sensors (GPS on and off, etc).
Testing Various Input Methods (gestures).
Testing for typical interrupts (calls, chargers connected, etc).
Testing Access Permissions (position, denied permissions).
Testing notifications (low battery, conditions).
Testing connectivity methods (2G, 3G, 4G, WiFi).
Installability testing (install, update and de-install).
Stress testing (out-of-memory, high CPU usage, etc).
Security testing (access to sensitive data, unsafe storage, encryption of transferred and/or storaged data).

We can continue listing a whole more testing types that we perform for mobile but I don't want to scare you haha. The thing is that, usually is not a single job person to execute all this testing types, all this testing types are splitted in several teams to confirm their correct behavior.

Now we will talk about the CI/CD approach for mobile, usually this keeps an standard flow like the one that we perform for our web based pipeline but with some extra steps, let's say that we usually have a deployment pipeline like this:

Build pods (containers)
Perform Unit tests
Deploy to test env
Deploy to stage
Perform e2e tests
Deploy to production

Now for mobile we will have some steps related to the app itself and also with the stores where the app will be visible:

- Source phase: here is where the code come from and what will trigger the pipeline execution.
- Testing phase: Unit test, UI test, Integration test.
- Building phase: here is the extra step that we have for mobile, in this step we will build the artifact and the code signing step to be able to send the new app version to the store of the different operative systems.
- Deploy phase: here we can have Alpha, Beta or production environment.

As I mention at the beginning, it's really exciting how mobile have some many testing types and considerations that we can develop and start playing with and yeah, I know, also is a lot of work to do...

Oh, before we finish this section, there is something that we should consider, the risks that we can find in the previous mentioned testing types:

Market fragmentation: to avoid this one is the suggested step of defining a most common used device to develop and test with.
Cost of multiple platforms: this can be minimized analyzing to support only the most common platforms.
New technologies and/on devices: when we get a new 5G connection for example, we can try to test using pre-production version of those new technologies.
Lack of devices availability: Crowd testing services or Remote devices services.

And finally, that's it, of course is easier to read about this than actually execute this mobile testing types but at least we now have a better idea on how to do it, and I hope that I make you feel excited about how testing can be so big and interesting. Don't forget to ask any question or let your comments so we can discuss about it.

Why you should NOT automate your tests

José Peñaherrera — Sat, 18 Dec 2021 22:51:36 +0000

I know what are you thinking, "why can anyone say that we don't need test automation?" and you're right! But, we need to also consider the context, the most of articles that I have read about "why you should automate your test" always talk about the benefits like reduce effort on repetitive tests, greater consistency to our testing approaches or easier access to info (results) about testing. The thing is, we are not always working on a well defined processes, for example we are a typical company that have our testers and had heard about the Automated Testing process and benefits so, the company, want to apply this to their daily activities. Here is when we should review if we should or not add the automated test activity to our business.

A common error that we get when we want to add the automated testing approach to our system is thinking that this will be the key solution to improve the Quality of our product, and that is not the case, the automated test are expensive to maintain, and will not cover the whole testing process for the products, usually, the automated tests cover only the happy path on a system, let's say that we have a product that sell clothes online, you will create an automated test that start from login, to adding clothes to your cart and will end with placing an order (in a lower environment of course, we don't want to add fake order in production and mess up our analytics). Now, what happen if, in the login page, the user type the wrong credentials, what if the clothe size is out of stock, what if the user set a wrong address for the delivery, all of this scenarios should not being automated, I mean you can do it, but if we decide to change the flow or, even, we make a small change on the login button selector, we must update all of our test so they can start working properly again.

Another thing to consider when you want to add automated test in a product, is that we usually tend to underestimate the time, cost and effort on the tool that we want to use for automating (Selenium, Cypress, TestCafe, etc.), this means that it cost a lot to add the tool and we will start seeing the benefits of it in several months, it will not give us value since the beginning of the implementation.

To finish this, we should keep in mind that Automating doesn't mean that we will replace other testing processes like Unit Testing, Integration Testing or Manual Testing. This will help us to improve some of the testing activities, more oriented (in my personal opinion) on the pipeline flow to confirm that the happy path works and we are good to deploy on prod without any breaking change. What do you think about it? You can leave your opinion in the comment section :D

How to build a Quality Culture in our team

José Peñaherrera — Mon, 13 Dec 2021 14:36:22 +0000

This is one of my favorites questions, and actually, is one of the hardest one to achieve, we should keep in mind these two premises:

Quality is not a one person job/responsibility
We don’t have only one Quality approach for a project, it changes based on the phase

What does this mean? Basically, the first one talks about keep the stakeholders (internal and externals) aware that the quality of the project relies on the whole team, and to achieve this, we can do it keeping a good communication to avoid misunderstanding on the system behavior as well as preventing defect in the code (this also could affect functionality), some techniques that can help us with this are:

Acceptance test-driven Development (ATDD): this approach is related with Test-driven Development (TDD) but is more focused on the stories, as TDD creates tests for the units to confirm that we are doing the things right, ATDD is defining automated test scenarios before the development to confirm that we are doing the right thing.
Behavior-driven Development (BDD): the main difference with ATDD is that BDD focuses on the behavior of the feature using the Given-When-Then structure to keep a common language between the team, and ATDD focuses on capturing the accurate requirements when included the stakeholders in its definition.
Specification by example (SBE): the main approach of this is capturing and illustrate the requirements using realistic examples instead of abstracts statements.

The main objective of these three approaches is to have a collaborative definition of the requirements and the business understanding inside the team, with this, we will avoid misunderstandings and will keep everyone focused on objectives, also this help us to we achieve the second approach, because as the quality change based on the phase on the project, we will let the team know the main quality approach for each phase, of course we always have a general objective for the overall quality like, talking about a online clothes store, the main objective will always be let the people buy our clothes online, but what happen if we want to add some offers for the Christmas season? We will have a new main quality objective of adding the offers to the page, right? What about when Christmas season finish? The approach will change to, remove the offers to avoid any issue with out customers, and so on.

Importance of QA on a team

José Peñaherrera — Tue, 07 Dec 2021 01:08:24 +0000

Have you ever had the doubt about, what does it means deliver a quality product? or, what does it means to work based on quality? Well, this questions have an easy but tricky answer, because quality depends on the perception of each person, so, based on this, let me show you what is my perception of quality. In general Quality means deliver a functional product that achieves the expectations of our stakeholders (yeah based on software delivery of course), but, for me, Quality also means, that the team is proud with the product that we are presenting to the stakeholders (insert emotional music here).

Ok, now that we understand the most common definition of Quality (we don't use the one that says "Quality means absence of errors" because, based on Quality Principles, Absence of error are a Fallacy) we need to understand from where this Quality comes from, as I mention before, in my perception of Quality, the whole team is responsible of achieving the best Quality level possible for the product, most of the people tends to asume, that the Quality Assurance person or the Tester are the only one responsible of doing this, that is true (in some way) because we should always have a Quality Champion (QC) on our team, who is in charge of negotiate with everyone to define the best practices that will adapt to the project Quality Strategy (QS) but, at the end, this will always be a team responsibility to understand, practice and improve what everyone defined on it.

My main purpose here is, make people understand that Quality is not a person or department, quality is a commitment that a team uses to define the best practices that can be applied to their specific project (there is not a common path or a typical Quality Strategy for all the teams).

To finish this post, I’ll let you know some desired activities that I get from my stakeholders on the QA activities that I performed on a team:

Define with the team and maintain the QS.
Guide the team on keeping track of what we defined on the QS.
Perform testing activities like static and dynamic testing (manual and automated testing).
Be aware in the coming activities and the actual activities that the team is performing to raise flags when needed.
Give the QA perspective in the team activities like deployments, pipelines, developments, business activities, etc.

I can type a complete book about this topic but I’ll leave it here for the moment, this is my first post and I’ll try to continue giving the expertise that I get from my daily activities and I hope that I can guide the people in understanding QA inside the development industry.