DEV Community: Jonathan Thompson

Expanding On the Tester Bill of Rights

Jonathan Thompson — Thu, 25 Mar 2021 14:34:14 +0000

I have been reading through Agile Testing: A Practical Guide For Testers And Agile Teams, by Lisa Crispin and Janet Gregory, when I found a small portion which discussed the existence of a "Bill of Rights" for programmers and customers. Lisa Crispin noted that a "Tester Bill of Rights" was absent, so she and Janet set forth to create one.¹

The "Tester Bill of Rights" contains six articles, each describing a primary right or responsibility of the tester within an agile team. Unfortunately, there is a lack of detail regarding each article. New testers may find themselves asking questions of these articles, specifically:

What does it mean to provide estimates for a sprint story?
Why should I advocate for the "whole team" approach to quality?
How can I approach my team about adopting new tools?

This article seeks to answer the above questions while providing insight into the articles that Janet Gregory and Lisa Crispin originally penned.

Article 1

"You have the right to bring up issues related to testing, quality, and process at any time."

It is your priority as a tester to raise items pertaining to application quality at any time throughout the sprint process. These could be bugs, design inquiries, agile practices, or any other item which could be a target for continuous improvement.

While a QA Automation Engineer at TransLoc, I noticed that my team's grooming process was more focused on fleshing out stories rather than refining the backlog. This caused the team to spend more time discussing stories, ultimately ending with "grooming" a single story per session when we should have been doing at least five. As a result, we were holding two grooming sessions a week which cut into development and testing time.

I raised this as an issue to my team and discussed potential solutions with my scrum master. We decided that we would test grooming strictly as refinement time while asking for story writers to flesh out their tickets at least one week prior. The result was a net increase in efficiency as we were able to cut our time spent grooming and discussing tickets in half.

Speak up early and often about issues that you see, even if they do not directly pertain to quality and testing.

Article 2

"You have the right to ask questions of customers, programmers, and other team members and receive timely answers."

I am personally guilty of succumbing to imposter syndrome and spinning over something that I would not reasonably know while testing a ticket. I have yet to kick this ugly habit despite being in this industry for half a decade. It has caused me to waste time on development tickets that were otherwise straightforward. The solution?

Ask questions, not just of your development team, but of product stakeholders and customers as well.

One of my favorite elements of quality assurance is collaborating with my development team. I am a proponent of "desk checks" where, in an office setting, I will message a developer working on a feature and ask whether I can swing by to chat about how it is going. The developer walks me through what they have implemented so far, and I will take the time to ask any questions related to application quality.

It should be agreed upon that there are no stupid questions during a desk check. Instead, you should feel free to ask any question, big or small, that can lead to greater understanding of the feature.

I will also take time to speak with product owners and customers to better determine how our users consume the application. This not only helps when determining critical workflows or prime automation candidates, but also in becoming a stronger advocate for the customer.

You should feel empowered to ask your team, or a customer, any question which will help you better understand the application and how it is used.

Article 3

"You have the right to ask for and receive help from anyone on the project teams, including programmers, managers, and customers."

I ran into a situation recently where I was stretched thin and could not feasibly regression test all of my release tickets by myself. I had two choices:

Risk pushing the release back by testing everything alone
Ask my team to help test with me

I chose to ask my team to help me test, knowing full well that they are just as busy as I am. Our release went as scheduled and we were able to provide value to the customer.

I understand that for many testers it can be difficult to approach a team of developers and ask for a hand with testing, or with understanding how a feature was implemented. I have experienced this first hand, most notably when starting for new teams or new companies. What drives me forward is the understanding that if I do not ask for help, I could be sandbagging my team.

Do your best to help your team. Ask for assistance when needed, whether it be testing a ticket, clarifying a feature implementation, or understanding a process.

Article 4

"You have the right to estimate testing tasks and have these included in story estimates."

I cannot begin to tell you how often I hear from testers that their efforts in automation or testing are not factored into story estimations. Simply put, this is an anti-pattern, and one that can cost your team sprint velocity should it spiral out of control.

While it may be difficult to estimate the amount of time, t-shirt size, or effort points (whichever practice that your team follows) for testing items, you should do your best to provide a ballpark figure for the team to work with. The biggest concern that testers have when estimating is how to quantify the time they spend automating and testing.

For example, you have a ticket that is estimated at 5 story points for development which should be about 2–3 days of work. Testing for the ticket, based on history with the product, will take about half a day. Automation is being built in parallel with development and will likely take about a day to complete when all is said and done.

In my experience, I would estimate this story as an 8. I would personally add 0.5 points for testing and between 1–1.5 points for automation, resulting in 7 which rounds to 8 as the next Fibonacci number in the sequence.

Estimate your testing activities. Not only will it provide your development team with a better idea of how much effort and time it takes to test, it will also bring more accuracy to your velocity measurement.

Article 5

"You have the right to tools you need to perform testing tasks in a timely manner."

As a tester, you should feel empowered to ask for tools that may help you do your job, whether it is an open-source tool or a subscription. The best way to navigate this is by showing your leadership team the data as to why the tool is necessary for your day-to-day work.

When I started out at Vodori, Inc. the development teams were utilizing the free version of Hiptest (now Cucumber Studio) which, at that time, only allowed a total of 10 users at a time. Our team was constantly adding and removing users as developers took part in writing and executing test scenarios. It was a fairly cumbersome and time consuming process.

After about six months of wrestling with the free version, I raised that Vodori should consider the enterprise version. In an effort to state my case, I reached out to Hiptest and asked for a trial to be put in place. They gave us the enterprise version for one month with the stipulation that after the month was over, we would either need to pay or downgrade to the free version.

I charted out the amount of time we spent removing and adding licenses against the cost for the enterprise version. I also noted the benefits of using enterprise, such as access to "Living Documentation".² I then presented this data to my QA Manager and the CTO of the company, arguing that we would be saving time and effort by using enterprise.

My case was denied. Simply put, Vodori was not in the position to rely on enterprise software at the time. We just did not have the funding to justify the expenditure.

You should not feel apprehensive to ask for tools that can help you do your job better. State your case, show leadership the data, and be graceful should they deny your request.

Article 6

"You have the right to expect your entire team, not just yourself, to be responsible for quality and testing."

This is quite possibly the most important of the articles found in the "Tester Bill of Rights". Your team should understand that quality is an effort that the entire team should be working on, not just those responsible for the testing of the product.

My team at Pendo.io has been diligently working toward bringing quality from the right swim lane into the left-hand columns. Rather than focus on quality at the end of the sprint like a waterfall team, we try to discuss quality concerns as early as possible. We have currently taken the following steps to ensuring that quality is a "whole team" effort:

Regular discussions on application quality and design in Slack
Inviting testers to blueprinting, grooming, and planning meetings
Noting quality impact during epic blueprinting, grooming, and planning
Developers taking part in manual and automated testing
Team forums on how to test and what to look for when testing

As Lisa Crispin and Janet Gregory emphasized many times in Agile Testing, quality is a "whole team" initiative and not the sole responsibility of the testing party.³

Summary

You cannot know everything, especially about new feature implementation. In these instances, ask your peers questions to strengthen your knowledge. Remember that you are not on an island, feel free to ask your peers for help whenever you feel the need to. Estimate quality tasks into sprint stories so they may provide a more accurate picture of your team's velocity. This will help fine-tune your sprints.

Ask for the right tools, but be prepared to show leadership the data on why you need it and what it can do for you. Inquire about application quality, process, and testing in order to continuously improve. Most importantly, remember that quality is a team effort.

Resources

Agile Testing: a Practical Guide for Testers and Agile Teams, by Lisa Crispin and Janet Gregory, Addison-Wesley, 2014, p. 50.
"Living Documentation." Living Documentation | CucumberStudio Documentation, support.smartbear.com/cucumberstudio/docs/bdd/living-doc.html.
Agile Testing: a Practical Guide for Testers and Agile Teams, p. 15.

Jonathan Thompson is a Senior Quality Engineer at Pendo.io specializing in test automation. He currently resides in Raleigh, NC with his wife and a Goldendoodle named Winston. You can connect with him on LinkedIn, or follow him on either Twitter or Github.

Sharpen Your Testing Skills by Debugging in Cypress

Jonathan Thompson — Wed, 24 Mar 2021 18:46:40 +0000

Debugging - we have all been there. Staring at lines of code for hours on end wondering why something broke the way it did. Debugging automation code can be a frustrating and mentally exhausting experience. No matter the toolset, rooting through lines of code to determine where automation broke down is challenging.

I personally am guilty of spotting an error, then immediately running my code again. For some reason I continue to think that the second time around my code will pass without issue. As if a 50% failure rate is anything to feel confident about - it is not.
Thankfully, Cypress comes with methods and features which greatly increase an engineer's ability to successfully debug automation code in a quick and efficient manner.

This tutorial assumes familiarity with Cypress and with test automation. We will use the DemoQA Elements page as a basis for our testing and debugging activities.

Setting Up the Debugger

In order to properly debug using Cypress, you must configure the browser window to open with developer tools. This can be done by adding the following code to your index.js module within the plugins directory.

NOTE: Launching a browser instance without the above inside of your plugins index.js module (or without the console manually opened) will not allow for debugging to take place.

You can verify the code has worked by opening the Cypress GUI and running a test in headful mode. The browser window should now open with developer tools prominently displayed.

Debugging Methods

Cypress allows for two separate debug methods:

JavaScript debugger statement
Cypress’ cy.debug() method

I have personally found the Cypress cy.debug() method to be much more useful to a testing engineer so I will be focusing on its usage over the debugger statement. This is largely due to the fact that Cypress captures element criteria when the debug method is called.

For example, you are building a test for selecting the dynamic click button on the DemoQA Buttons page. The dynamic click button is the third button shown on the page, below the double click and right click buttons. Writing out a simple call to get a button and click it fails as there is more than one button element on the page.

A quick and easy way to find which button to select would be to use the debug method immediately after getting all buttons on the page.

Running the above code will return the following in your browser window:

As you can see, there are four buttons on the page with the dynamic click button occupying the third index. Opening the index within the console shows information for the selector, ranging from childNodes and innerText, to onClick data. For this particular issue, we will focus on the innerText entry as the button has a unique text node that we can work with.

Using the cy.contains() method with the text “Click Me” is not going to work in this instance, as there are three buttons with “Click Me” on the screen. Instead, we will resort to using a regex pattern and match it to the exact contents of the innerText data found when using the debug method.

Our test will now pass without issue as we are selecting the correct button on the screen.

This may seem like a rudimentary example. The intention is to demonstrate the practice of using the cy.debug() method for finding element criteria which can help build a selector for test consumption.

Past and Present

One of the original features that drew me to adopt Cypress version 1.0.0 was the before and after DOM screenshots for page actions. Prior to Cypress, engineers would relied on two patterns for debugging via image screenshot:

Manually enter screenshot calls within the test code
Screenshot on after failure calls

The trouble with the first was that it required the engineer to know specifically where to enter the screenshot call. In extreme cases, engineers would add screenshot calls before and after every action. Each automation run would then fill a directory with screenshots to sift through without context, further muddying the engineer’s ability to accurately troubleshoot automation issues.

Screenshot on failure was only useful to determine application state when the action failed. It was not at all helpful with viewing application state prior to the failure.

Cypress solves these issues by providing DOM screenshots before and after an action is taken on the page. Below is an interaction on the DemoQA Buttons page. When a user double clicks on a specific button, a message is shown in a container below the button rows.

The gif shows the Cypress test runner with “before” and “after” buttons at the bottom of the screen. The “before” button shows application state prior to action while the “after” button shows the result. Toggling the “after” button displays a screenshot with message text stating that a double click has occurred on the correct button, thereby confirming that a double click has taken place on the page.

While this information is only available when running in headful mode, it allows an engineer to review actions that have been taken within the application and the application’s state immediately prior. This can prove to be extremely helpful during debugging by providing a base for when to add cy.debug() methods.

Capture It On Video

By default in headless mode, Cypress captures video files for each test which has completed — whether it passes or fails. These videos can provide a glimpse into application state during the test while showing the overall workflow under test. Engineers can use these videos to quickly spot errors within application state, taking into consideration what actions are occurring so as to mark where debug statements need to be entered.

As a test engineer, you should be reviewing the videos after every failure in order to determine where to begin troubleshooting. While they are not interactive, they do provide adequate context.

Pause for Effect

Many programming languages feature a Read-Evaluate-Print Loop (REPL) which allows an engineer to step into the code during execution. From here, the engineer can write out steps and watch the result of their commands in real time. I am intimately familiar with this pattern as a good portion of my automation experience is with Python and Ruby, both programming languages that feature REPLs.

Each language allowed me to write automation code, open a headful window, then step into it using binding.pry for Ruby and breakpoint for Python, respectively. Once inside, I could write out the code for the next step of the test within the REPL and watch the results of my interactions. This process allowed me to see the actions execute and what sort of issues I should be looking for, such as slow-loading elements that need to be waited upon.

Unfortunately, JavaScript does not feature a REPL. However, the creators of Cypress did provide us with an alternative: the cy.pause() method.

Using the pause method stops your automation code and provides two additional features:

A play button
A next button

The play button will simply run the test as normal. It is the next button that is critical to troubleshooting automation code.
Here is the button in action:

Using the next button, we can view the action and result for each step of code in our test. This is extremely powerful as it allows the engineer the ability to view results outside of the confines of video or screenshots. Instead of static assets, the engineer is directly controlling Cypress. This is perfect for troubleshooting page loads, finicky selectors, or all manner of other issues.

I personally use this pattern whenever I find myself troubleshooting with Cypress, no matter the size of the issue. The cy.pause() method is far too powerful to not use when writing or maintaining automation code.

Summary

Debugging test automation does not have to be a painful experience. By using Cypress’ debug method, you can spy application elements for use within test automation code. Reviewing DOM screenshots and videos allows you to build context for entering debug statements. Finally, the cy.pause() method is a powerful tool which allows an engineer to step into the test code as it runs and manually execute test steps.

Each of these tools will greatly enhance your ability to troubleshoot and debug automation code.

Resources

“Debugging.” Cypress Documentation, 5 Mar. 2021, docs.cypress.io/guides/guides/debugging.html.
Cypress-Io. “Proposal: Add Command-Line Flag for Opening Dev Tools during Run · Issue #2024 · Cypress-Io/Cypress.” GitHub, github.com/cypress-io/cypress/issues/2024.

This article was originally published on Medium.