Playwright is a remarkable tool that lets you author cross-browser cross-platform end-to-end tests for your web app. It even lets you generate test code as you navigate your app (but that's probably a topic for another blog post)!
Unfortunately, as of now, out of the box, it does not support measuring code coverage for the app being tested (see #7030 and #9208). This post demonstrates a workaround.
Specifically, this post shows you how to instrument a Next.js app using a custom babel.config.js and how to measure code coverage using the playwright-test-coverage plugin. If you want to jump straight into the demo, you can do that here:
anishkny
/
playwright-test-coverage-demo
Demonstrates usage of playwright-test-coverage - a Playwright extension that collects code coverage from running end-to-end tests
Playwright Test Coverage Demo
Demonstrates usage of playwright-test-coverage - a Playwright extension that collects code coverage from running end-to-end tests.
Read more in this blog post.
Usage
npm install
npm test
Code coverage should be generated in coverage/ folder.
Sample
License
Step 1: Instrument your code
If you want to measure code coverage, first you will want to instrument your code (i.e. insert counters without causing any side-effects) using istanbul. Most modern web apps (React, Vue etc), use babel in their toolchain (which converts modern ES2015+ code into backwards compatible code to support older browsers). The recommended way to instrument code in this scenario is using babel-plugin-istanbul:
npm i -D babel-plugin-istanbul
Now, we need to instruct babel to use this plugin. The details may vary by framework, but for a Next.js app, create a file named babel.config.js in the root folder:
// babel.config.js
const plugins = [];
// Instrument for code coverage in development mode
if (process.env.NODE_ENV === "development") {
console.log(
"Detected development environment. Instrumenting code for coverage."
);
plugins.push("istanbul");
}
module.exports = {
presets: ["next/babel"],
plugins,
};
This will instrument code only in development mode e.g. when running next dev. That's good because we do not want instrumentation cluttering our production code.
Step 2: Write tests using the playwright-test-coverage extension
Now that your code is instrumented, you are ready to measure code coverage. For this you will need to install this Playwright plugin:
npm i -D playwright-test-coverage
This package exports overrides for Playwright's test and expect methods which handle code coverage measurement correctly.
// tests/app.spec.js
const { test, expect } = require("playwright-test-coverage");
test("basic test", async ({ page }) => {
// Use Playwright as usual
await page.goto("http://localhost:3000");
await expect(title).toHaveText("Title");
// ...
});
If you have existing Playwright tests, simply drop in the following replacement near the top:
// Replace this:
const { test, expect } = require("@playwright/test");
// With this:
const { test, expect } = require("playwright-test-coverage");
Step 3: Measure code coverage
Finally, you are ready to measure code coverage.
Start the Next.js dev server using the Istanbul command line tool nyc - which enables code coverage measurement:
npx nyc next dev
Then, in another terminal window, run the playwright tests as usual:
npx playwright test
If all goes well, you should see a code coverage report similar to:
Running 1 test using 1 worker
✓ tests/app.spec.js:5:1 › basic test (1s)
1 passed (2s)
----------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files | 100 | 100 | 100 | 100 |
index.js | 100 | 100 | 100 | 100 |
----------|---------|----------|---------|---------|-------------------
Wrap up
Hopefully you now have some understanding of how to measure code coverage for a Next.js (or any Babel based) app when running Playwright tests.
You can check out the full source code for the demo here:
anishkny
/
playwright-test-coverage-demo
Demonstrates usage of playwright-test-coverage - a Playwright extension that collects code coverage from running end-to-end tests
Playwright Test Coverage Demo
Demonstrates usage of playwright-test-coverage - a Playwright extension that collects code coverage from running end-to-end tests.
Read more in this blog post.
Usage
npm install
npm test
Code coverage should be generated in coverage/ folder.
Sample
License
The extension code is available here and contributions/issues are welcome:
anishkny
/
playwright-test-coverage
Extends Playwright test to measure code coverage
Playwright Test Coverage
A Playwright extension that collects code coverage from running end-to-end tests. Assumes that code has been instrumented with babel-plugin-istanbul during the build process.
Prerequisites
- Playwright test framework
-
babel-plugin-istanbulplugin -
nycfor running tests
npm i -D @playwright/test babel-plugin-istanbul nyc
Installation
npm i -D playwright-test-coverage
Usage
Write your Playwright tests as usual, except require test and expect from this package as follows:
// tests/foo.spec.js
const { test, expect } = require("playwright-test-coverage");
// Use test and expect as usual
test("basic test", async ({ page }) => {
await page.goto("https://playwright.dev/");
const title = page.locator(".navbar__inner .navbar__title");
await expect(title).toHaveText("Playwright");
});
Then, instrument your front end source code for coverage using the babel-plugin-istanbul plugin.
Finally, run your server via nyc…
Cheers!
PS: Using with fixtures
Playwright lets you define custom fixtures in accordance with the Page Object Model pattern. This library should work out of the box with custom fixtures.
You can see an example here.
Top comments (3)
Great package and great tutorial! It's what I'm looking for!
We are getting this error now: "TypeError: Property left of AssignmentExpression expected node to be of a type ["LVal"] but instead got "BooleanLiteral" "
The internet says we should not be using a custom babel config file
Hi Anish! Nice post.
I'm trying to make this work with code coverage cover for all files, even those that not have test files associated. I'm trying to set nyc configuration, but I'm even so the code coverage just returns for the files that are being tested. Do you have any idea how to make it work?