Writing integration tests with jest and puppeteer

For quite some time already I'm working on a personal project every friday which contains a part that involves heavy detection work being executed in the browser.

Even though my project is generally divided into multiple files with small, well-isolated methods, classes, and other elements—each thoroughly tested with Jest unit tests—I occasionally encounter situations where I need to test parts of the project in a more realistic environment. This typically arises when combining various methods to achieve a specific goal that works well in the browser but isn't easily covered by Jest unit tests running in a Node environment. In these cases, attempting to use mocks, stubs, or specialized test packages can introduce unnecessary complexity, which is something I strive to avoid.

The challenge with developing these browser-targeted algorithms lies in the difference in context. Jest operates in a Node environment, running directly on your hardware, rather than in a browser context where JavaScript usually executes.

Thankfully, Puppeteer—a programmable headless Chrome—provides a solution. By using Puppeteer alongside Jest, we can create robust integration tests for frontend code intended to run in the browser.

The general setup is as follows.

1. We define a folder in our project, eg. stubs which we can use to store snippets to test against. You can for instance download a webpage as html and store the downloaded page for later use
2. We copy - or when using typescript - build the source file and store it in the same stubs folder for easy access during the test
3. Before we start each test we load our stub html inside puppeteer, and then add the script as script tag to our puppeteer instance so we are able to interact with our code
4. Then we run our test, we run evaluate to our puppeteer page where we invoke our code inside the browser context and return the result back to the node context
5. Finally, we remove our script from the stubs folder to prevent committing it later on

Example implementation
Lets look at a practical implementation for something I was working on, an algorithm that extracts the dominant colors based on a 'screenshot' of the page using canvas. Html canvas does not exist in the node context which makes writing tests for this algorithm kind of a pain train. It then loops over all pixels and gives a list of colors based on their weight

I've got my stubs folder up and runnning so lets proceed to step 1: I am using typescript so I need to compile the target I wish to test first. I am injecting it on a global object IntegrationTests

/**
 * @jest-environment puppeteer
 */

import "jest-puppeteer";
import puppeteer, { Browser, Page } from "puppeteer";

import * as fs from "fs";
import { puppeteerSettings } from "../../Constants/Tests";
import { PluckedColor } from "./mapColorsUsingCanvas";

const esbuild = require("esbuild");

esbuild
  .build({
    entryPoints: [
      process.cwd() + "/src/Algorithms/Colors/mapColorsUsingCanvas.ts",
    ],
    bundle: true,
    outfile: process.cwd() + "/stubs/mapColorsUsingCanvas.js",
    target: "es2015",
    format: "iife",
    globalName: "IntegrationTests",
  })
  .catch((err: Error) => {
    console.log(err);
    process.exit(1);
  });

Second step is setting up the tests by starting puppeteer, loading the stub page and injecting my script. The debug flag is for local testing, if you turn that on you get the console.log inside the browser context which is a bit of a maze to plough through

const debug = false;

describe("mapColorsUsingCanvas Integration test", () => {
  let browser: Browser;
  let page: Page;

  beforeAll(async () => {
    browser = await puppeteer.launch(puppeteerSettings);
    page = await browser.newPage();

    // THIS spams your console
    if (debug) {
      page.on("console", (msg) => console.log("PAGE LOG:", msg.text()));
    }

    await page.setViewport({ width: 1366, height: 768 });

    await page.goto(`file:///${process.cwd()}/stubs/text_strategy.html`, {
      waitUntil: "domcontentloaded",
    });

    await page.addScriptTag({
      path: process.cwd() + "/stubs/mapColorsUsingCanvas.js",
    });
  }, 20000);

Also I'm defining my final step of cleanup hereunder, its basically a simple rm on the javascript file created in the esbuild step

  afterAll(() => {
    browser.close();
    fs.rmSync(process.cwd() + "/stubs/mapColorsUsingCanvas.js");
  });

Finally lets look at our integration tests itself now. We run a page eval - the method body operates inside puppeteer in the browser context - which allows us to invoke our injected script we want to test. We return the result however back to jest running in the node context which then allows us to run the necessary assertions!

  it("runs mapColorsUsingCanvas on the stub and returns the dominant colors", async () => {
    const result: PluckedColor[] = await page.evaluate(() => {
      // eslint-disable-next-line @typescript-eslint/ban-ts-comment
      //@ts-ignore-next-line
      return IntegrationTests.mapColorsUsingCanvas(document);
    });

    expect(result).toHaveLength(10);
    expect(result[0]).toEqual({
      color: { r: 255, g: 255, b: 255 },
      weight: 0.9839,
    });
    expect(result[1]).toEqual({
      color: { r: 17, g: 17, b: 17 },
      weight: 0.0057,
    });
  });
);

Conclusion

Using this approach we can add these type of integration tests easily to our project. It has little to do with TDD of course, these kind of tests more act as an an insurance policy that future changes won't affect any real life situation, and they have been proven really useful already on some occasions.

for reference, the whole file hereunder

/**
 * @jest-environment puppeteer
 */

import "jest-puppeteer";
import puppeteer, { Browser, Page } from "puppeteer";

import * as fs from "fs";
import { puppeteerSettings } from "../../Constants/Tests";
import { PluckedColor } from "./mapColorsUsingCanvas";

const esbuild = require("esbuild");

esbuild
  .build({
    entryPoints: [
      process.cwd() + "/src/Algorithms/Colors/mapColorsUsingCanvas.ts",
    ],
    bundle: true,
    outfile: process.cwd() + "/stubs/mapColorsUsingCanvas.js",
    target: "es2015",
    format: "iife",
    globalName: "IntegrationTests",
  })
  .catch((err: Error) => {
    console.log(err);
    process.exit(1);
  });

const debug = false;

describe("mapColorsUsingCanvas Integration test", () => {
  let browser: Browser;
  let page: Page;

  beforeAll(async () => {
    browser = await puppeteer.launch(puppeteerSettings);
    page = await browser.newPage();

    // THIS spams your console
    if (debug) {
      page.on("console", (msg) => console.log("PAGE LOG:", msg.text()));
    }

    await page.setViewport({ width: 1366, height: 768 });

    await page.goto(`file:///${process.cwd()}/stubs/text_strategy.html`, {
      waitUntil: "domcontentloaded",
    });

    await page.addScriptTag({
      path: process.cwd() + "/stubs/mapColorsUsingCanvas.js",
    });
  }, 20000);

  afterAll(() => {
    browser.close();
    fs.rmSync(process.cwd() + "/stubs/mapColorsUsingCanvas.js");
  });

  it("runs mapColorsUsingCanvas on the stub and returns the dominant colors", async () => {
    const result: PluckedColor[] = await page.evaluate(() => {
      // eslint-disable-next-line @typescript-eslint/ban-ts-comment
      //@ts-ignore-next-line
      return IntegrationTests.mapColorsUsingCanvas(document);
    });

    expect(result).toHaveLength(10);
    expect(result[0]).toEqual({
      color: { r: 255, g: 255, b: 255 },
      weight: 0.9839,
    });
    expect(result[1]).toEqual({
      color: { r: 17, g: 17, b: 17 },
      weight: 0.0057,
    });
  });
});