FlakeGuard aims to find tests that pass one time, and may fail another time, despite not changing any written code.
Curious about what flaky tests are - checkout this article, before diving into the benefits of utilizing FlakeGuard!
What is FlakeGuard?
FlakeGuard is a free, open-source tool that allows developers to run Jest tests to automatically detect and report flaky tests in the development process.
Flaky tests can be flagged, and viewed visually through a variety of charts and metrics, see here.
Set-Up
Install: npm i flake-guard --save-dev
Run FlakeGuard: npx flake-guard <filename> with the name of the test file that you want to examine.
FlakeGuard will analyze your E2E tests for flakiness by executing multiple test runs and analyzing the results.
Once all tests have run successfully, simply press enter in the terminal to be redirected to flake-guard's website.
Configure FlakeGuard
The default number of test runs is 10, but this can be adjusted as described below.
To adjust FlakeGuard configuration variables, you can create a file in the root directory of your project calledfg.config.json.
How FlakeGuard Works - "Under the hood":
The flake-guard npm package is automating repeated runs of your test file. It will do a basic parsing of the results locally to log an object in your terminal with all of your test assertions and their pass/fail metrics. It sends off the raw Jest results objects to the FlakeGuard server for further analysis which you can view at
FlakeGuard Web Application:
FlakeGuard Sample DashboardView analytics including, but not limited to:
- Passing vs failing tests with the color coordinated flake severity table
- Analyze Runtime metrics & API response times
- See flaky test flags
- View pointers showing fail points within test code
- Running test order permutation
- Display flake improvement over time
Other articles:
Flaky Tests - How to Overcome Testing Nightmares
https://medium.com/@ashleyhannigan88/flaky-tests-how-to-overcome-testing-nightmares-cdf661b51464
Top comments (0)