Comprehensive coverage Jest+Playwright in Next.js TS

Before I start, disclaimer:

• I assume You know how to setup Next.js, Jest and Playwright (without coverage)
• This approach will create two json coverage files, which will be merged together by NYC. Therefore the results will be purely local. If You don't mind using online tools like Codecov or Coveralls for merging data from different tests, then go ahead and use them. They will probably also be more accurate. But if You still want to learn how to get coverage from E2E, then please read through
• There might be slight differences in interpretation of functions/branches/lines between Jest and Playwright (I have noticed some minor problems with import statements)
• E2E will run against locally served app
• It will hurt performance during testing and build as Babel (not SWC) will be used

Why even bother?

If You are data junky like me, then You want to see improvements in coverage percentages throughout coding. And if You don't like waiting for coverage reports from CI/CD pipeline then this approach might be for You.

I won't be focusing on setting Next.js, Jest or Playwright here as I assume You already know how to do it, but I will go through all the steps required to adapt existing Next.js + Jest + Playwright project to include coverage

Setup

Dependencies

Of course You need to install some dependencies. Here's the command, I will describe all of them below:

npm install babel-plugin-istanbul nyc ts-jest ts-node

Let's have a closer look on those dependencies:

babel-plugin-istanbul: it is responsible for collecting coverage from Playwright tests. Here's the link to GitHub repository. You'll also need baseFixtures.ts file from this repo to make it run

nyc: a tool I use to merge coverage reports together. It will create final report, which is an union of coverages from Jest and Playwright

ts-jest: quite usefull as the focus is to test TypeScript app

ts-node: needed to compile Jest config TS file back to JavaScript

Code

You need to make some changes in Your code. Let's start with config files. Suprisingly playwright.config.ts is fine as it is. But there are some changes in jest.config.ts. Let's start with it.

jest.config.ts

import type { InitialOptionsTsJest } from "ts-jest";

const config: InitialOptionsTsJest = {
    preset: "ts-jest",
    testEnvironment: "jsdom",
    globals: {
        "ts-jest": {
            tsconfig: "tsconfig.jest.json",
            useESM: true,
        },
    },
    testPathIgnorePatterns: ["./tests/"],
    collectCoverage: true,
    collectCoverageFrom: [
        "**/*.{js,jsx}",
        "**/*.{ts,tsx}",
        "!**/node_modules/**",
        "!**/__tests__/**",
        "!**/.*/**",
        "!**/*.config.*",
        "!**/coverage/**",
        "!next-env.d.ts",
    ],
    coverageDirectory: "./coverage",
    coverageReporters: ["json"],
};
export default config;

preset: self explanatory - You want to test TS code

testEnvironment: jsdom is required for correct interpretation of .tsx files (and React components)

globals: ts-jest: here is where the magic happens: as Next.js does not allow changing "jsx":"preserve" to any other value in tsconfig.json You need to set it separately in a special file. If You don't, You'll see:

This setting is needed for correct coverage reports. That is why I created a special file, which I point to in globals. useESM also helps to interpret ECMAScript 2015+ features.

coverage settings: I like to set it in config file. To sum up it collects coverage from js/jsx/ts/tsx files excluding tests and configs and saves it as json (important!) to coverage directory

tsconfig.jest.json

This is the file I mentioned before. As jsx is set here in separate file Next.js cannot fix this automatically and will use adjusted TS config:

{
    "extends": "./tsconfig.json",
    "compilerOptions": {
        "jsx": "react-jsx"
    }
}

baseFixtures.ts

This is a file from babel-plugin-istanbul page. Original file is available here, but I made some changes to make it even simpler (I removed uuid and changed output location to coverage):

import * as fs from "fs";
import * as path from "path";

import { test as baseTest } from "@playwright/test";

const istanbulCLIOutput = path.join(process.cwd(), "coverage");

export const test = baseTest.extend({
    context: async ({ context }, use) => {
        await context.addInitScript(() =>
            window.addEventListener("beforeunload", () =>
                (window as any).collectIstanbulCoverage(
                    JSON.stringify((window as any).__coverage__)
                )
            )
        );
        await fs.promises.mkdir(istanbulCLIOutput, { recursive: true });
        await context.exposeFunction(
            "collectIstanbulCoverage",
            (coverageJSON: string) => {
                if (coverageJSON)
                    fs.writeFileSync(
                        path.join(
                            istanbulCLIOutput,
                            `playwright_coverage.json`
                        ),
                        coverageJSON
                    );
            }
        );
        await use(context);
        for (const page of context.pages()) {
            await page.evaluate(() =>
                (window as any).collectIstanbulCoverage(
                    JSON.stringify((window as any).__coverage__)
                )
            );
        }
    },
});

export const expect = test.expect;

.nycrc

Last new file in NYC config. Here is how it should look like:

{
    "all": true,
    "include": [
        "**/*.tsx",
        "**/*.ts"
    ],
    "exclude": [
        "**/*.config.ts",
        "**/*.config.js",
        "**/*.d.ts",
        "**/pages/api/**/*.*",
        "__tests__",
        "tests"
    ],
    "reporter": [
        "html"
    ]
}

all: You need to tell NYC to include all files in project directory for coverage. It is quite counter-intuitive that there is this option and also include/exclude, but without all option set to true it won't include files in report, which had 0% coverage

include/exclude: what You want to see in report and what not

reporter: I set it to html to be able to see results in human friendly way. Other options available are: lcov, json, text

package.json

I'll also post my changes in scripts to package.json file:

"test": "jest",
"test-e2e": "playwright test",
"create-report": "nyc report --reporter html --reporter text --reporter json -t coverage --report-dir coverage/summary",
"coverage": "npm run test && npm run test-e2e && npm run create-report"

You can run them with npm run <command name>.

Execution

Finally after long setup it is time to run the tests and see what is still not covered. The steps are as follows:

1. Start development server in background with: npm run dev
2. Run Your Jest tests: npm run test
3. Run Playwright tests with: npm run test-e2e
4. Create coverage report: npm run create-report

Or You can use shortcut for steps 2-4 by running: npm run coverage

Please keep in mind that NYC requires json files to be able to merge them. baseFixtures.ts always returns json file, make sure that You also set json as output format for Jest.

The result should be available under coverage/summary/index.html

Summary

It is not perfect, but this approach can give some kind of overview of coverage of code, also in places where unit tests are just not enough. It works best with line coverage, so I would suggest on focusing only on it when You extend Your tests

Sources

To prepare this article I used:
ts-jest docs (GitHub.io)
Next.js issue regarding jsx option (GitHub)
New value for jsx parameter (Stack Overflow)
Playwright coverage plugin (GitHub)