Sometimes we fail our tests so we can see that when it passes it really is passing but there are times when our tests fail and we don’t know why. When that happens it’s time to debug.
In this post we will take a look at how to debug our failing tests right in VS Code as well as debugging using the trace viewer. Let’s dive in and find those bugs.
In the last post we used our Codegen tool to create a test for us and we added an assertion to make sure there was a product in the cart and that when we clicked on the remove button the product was no longer in the cart.
Let’s break our test by making a simple typo and changing the remove button to be called removed. Let’s make sure we have show browser checked and then run our test. As expected our test fails. Because we have show browser checked we have access to the browser window and now in VS Code we can hover over each line of test to see the locator highlighted in the browser. Yet where the test is failing we can see no element is highlighted because Playwright can’t find that element.
However we can start editing the name of this locator in VS Code and see it being highlighted as we fix our typo. By removing the letter 'd' from remove we can now see our locator highlighted in the browser meaning Playwright can find it. You can see if we remove even more letters and have only the name of re it will tell us that there are more than 1 of these locators. By scrolling down the page we can see that this matches the refund policy in the footer of the page. This would mean that it is not a very good locator and your test would fail as Playwright is strict by default and therefore can only accept one matching locator.
If we save this and run the test you will see the error message come up in VS Code saying strict mode violation and showing the two elements that Playwright found for this locator.
As Playwright doesn’t know which one it should click on it fails the test. This is why it is important to always choose good locators that have exact matches. Check out the locator docs for more info on how to create exact matches and also use chaining and filtering.
In VS Code we can also set breakpoints by clicking next to the line we want to pause our test on. You will see a red dot appear. We can then run our test in debug mode by right clicking on what was previously the play button but is now a red X, then from the drop down menu select debug test. You can also click on the debug test icon next to the test name in the testing sidebar.
Now in our browser window we will see our test has stopped on the element and is highlighting the name we have in our test and that Playwright has found 2 elements with the same name. If we click on the play button of our debug menu the test will give us the strict mode violation error. We can then fix the error by changing the name to remove and re running our test in debug mode. Now we can see we only have one element found for that locator and pressing play will continue to run our test and our test will pass.
We can remove the breakpoint by clicking on the red dot.
Another way to debug your tests is by using the trace viewer. First lets break our test again. This instead of the label cart lets change it to bag. Now lets go to our testing sidebar and click on the show trace viewer option and then run our test as normal. Playwright will try to find the element with the label bag and will retry it until the timeout has reached. After the default of 30 seconds it will show us a trace of the test up until the moment it fails.
We can now walk through our test step by step to try to understand what went wrong and why. If we click on the source of our test in the menu on the right side we can see each line of code as we click on each action of the test. Line 12 is underlined in red and this is where our test is failing. Playwright is waiting for an element with a label of bag and can’t find it.
We also can’t see a label of bag on the page as we can just see an icon and it looks pretty much like a bag to us. However we can easily check what the correct locator is for this bag by clicking on the pick locator button and hovering over the bag icon. We can see that under the bag icon the locator is highlighted underneath and it is in fact a getByLabel cart not bag. If we click on this element it will add the locator into the pick locator box. We can then use the copy button on the right hand side to copy it to our clipboard.
Now in VS Code we can go to line 12 where our test is failing and replace the locator with the locator we have just copied from the trace viewer. We can then save our test and re run it and as you can see our test now passes and Playwright can find our element and it is being clicked on just like expected and the test continues. As we click on each action we can see the element highlighted in the the DOM snapshot and we can even see what happened before this action and what happened after it.
Don’t forget if we need to inspect the HTML or CSS of our code we can simply click on the pop out button to pop our DOM snapshot into a browser window and then open dev tools by right clicking on the browser window and choosing inspect. We can then easily understand where our cart label is coming from. In our case the word cart is coming from the aria-label of the button.
We can then decide if we want to change this label in our code or just modify our test to use the word cart instead of bag. Either way what is important is that now we have a test that will catch this bug should someone change this in the future meaning our app won’t break in production and our users can easily add items to their shopping bag or cart or whatever we decide to call it.
Now that we know how to debug tests it’s time to run our tests on Continuous Integration so that tests run on each commit and pull request. By running our tests on CI it means all bugs will be caught before they make it to production. Nice. See you in the next video.
Check out the Video
Useful Links
- Playwright docs
- Join our Discord server
- Subscribe to the Playwright YouTube channel
Top comments (2)
Hello. I hope you are well. First off, I am deeply grateful for Playwright and the Playwright team--y'all are amazing!
Second, would you be able to provide more resources on troubleshooting timeout issues in a WordPress/WooCommerce environment? I have spent a great deal of time researching both your first party docs as well as third party docs in search of an idea that might help me figure out transient timeout issues. The tests will run fine (I've even had them run 56 times consecutively without issue). Then, for some reason that I cannot discern, they timeout. I am guessing this is related to some idiosyncrasy related to the WordPress environment, hence me asking you to address that in future writings/blogs/videos.
Thank you in advance,
Jesse Lind
Thanks for the post!