DEV Community: Greg Gorlen

Avoiding False Positives in Node.js Tests

Greg Gorlen — Wed, 04 Dec 2024 11:30:34 +0000

When running tests, it's a great feeling to see dozens of green check marks indicating that a test suite is passing. It's especially gratifying after tackling a tricky bug or slogging through a tough feature. But those passing tests may be giving you a false sense of security.

Often, bugs lurk in passing tests, undermining trust in the test suite and your application. Such tests can cause more harm than good, giving you a hearty pat on the back while hiding broken functionality. It can take months to weed out a false positive test, and that might be only after a customer complaint.

This post examines several common false positive patterns that can crop up in test suites. Taken out of context, the examples may appear obvious, but they find their way into complex, real-world tests all the same.

Prerequisites

I will assume readers are familiar with ES6 JavaScript syntax and have written at least a handful of tests in NodeJS.

Although I don't intend to be technology-specific, inevitably some of the pitfalls are specific to quirks in the JavaScript ecosystem (particularly its UI testing libraries). I expect readers will be unfamiliar with some (or most) of the libraries featured in the post, but I hope the underlying principles will be relevant.

This post is current as of Node 22.11.1, jest 29.7.0, mocha 10.3.0, chai 5.1.0, chai-http 4.3.0, supertest 6.3.3, eslint 8.57.0, @playwright/test 1.42.1, and @testing-library/react 13.4.0.

Let's begin.

Common False Positive Test Patterns in Node.js

First, we'll look at some common patterns when it comes to false positive test results.

Using `equal()` Rather Than `strictEqual()`

As a warmup, check out this Mocha test with Chai assertions:

import { assert } from "chai";

describe("strict equality", () => {
  test("1 equals '1'", () => {
    assert.equal(1, "1");
  });

  test("0 equals false", () => {
    assert.equal(0, false);
  });

  test("null equals undefined", () => {
    assert.equal(null, undefined);
  });
});

Here's the output:

$ npx mocha strict-equality-bad.test.js


  strict equality
    ✔ 1 equals '1'
    ✔ 0 equals false
    ✔ null equals undefined


  3 passing (2ms)

This classically illustrates a surprising JS type coercion, courtesy of the loose equality operator ==. Replacing assert.equal() with assert.strictEqual() (akin to ===) gives the more desirable result — these values being unequal (some output omitted for brevity):

$ npx mocha strict-equality-good.test.js


  strict equality
    1) 1 equals '1'
    2) 0 equals false
    3) null equals undefined


  0 passing (3ms)
  3 failing

  1) strict equality
       1 equals '1':
     AssertionError: expected 1 to equal '1'
  2) strict equality
       0 equals false:
     AssertionError: expected +0 to equal false
  3) strict equality
       null equals undefined:
     AssertionError: expected null to equal undefined

The following assertions also fail, as expected:

// Jest:
expect(1).toBe("1");
expect(1).toEqual("1");

// Chai:
expect(1).to.eq("1");
expect(1).to.equal("1");

The theme here (which will be a recurring theme in this post) is that matchers don't always behave as expected.

These mistakes can be detected by experimenting with assertions to trigger failures. For example, consider the assertion:

assert.equal(add(-3, 3)).to.be(0);

Try plugging in temporary values like .to.be(-1), .to.be(undefined), and .to.be(false) to ensure the assertion fails. If they don't, the assertion is too weak and might be hiding a bug. Failures are good!

Test-driven development can help to avoid overly-permissive assertions. Tests that start out by failing before passing are more likely to function as intended than ones that pass from the outset, written against already implemented code.

Using Overly General Assertions

The next example looks at the dangers of broad matchers such as .toBeTruthy() and .toBeFalsy(), which are present in most assertion libraries. The following test has a serious bug, but passes:

describe("parser", () => {
  it("should parse `text` without throwing", () => {
    const parser = new Parser(text);
    expect(parser.parse).toBeTruthy();
  });
});

The author wants to assert that the return value of the parse() function is truthy; for example, that it returns an object. But the test author forgot to call the function, so the test is only asserting that the parse property exists!

Using .toBeDefined() and .toBeInstanceOf(Object) would fail in the same way. Variables and properties are typically defined and are often objects, so these assertions are too weak. Frequent use of loose assertions can indicate a code smell in the test or in the application under test. Functions should return values with predictable types, and assertions should be strict and specific to these types.

General assertions tend to read poorly and emit vague failure messages. Rewrite assertions like expect(meaningOfLife() === 42).toBe(true) to expect(meaningOfLife()).toBe(42).

See this Playwright question on Stack Overflow for an example of a false positive .toBeTruthy() assertion in the wild.

Using Shallow Equality for Deep Comparisons

Similarly, false positives can occur when comparing objects:

import { assert, expect } from "chai";

describe("deep equality", () => {
  test("[assert] two objects with same contents not equal", () => {
    assert.notStrictEqual({ foo: 42 }, { foo: 42 }); // incorrect, passes
    assert.notDeepEqual({ foo: 42 }, { foo: 42 }); // correct, fails
  });

  test("[expect] two objects with same contents not equal", () => {
    expect({ foo: 42 }).not.to.eq({ foo: 42 }); // incorrect, passes
    expect({ foo: 42 }).not.to.deep.eq({ foo: 42 }); // correct, fails
  });
});

The assertions labeled "incorrect" test identity rather than deep value equality.

Note that .not is a contributing factor to the problem. Avoiding .not whenever possible can improve readability, as it often does in boolean logic in application code. Positive assertions tend to match a narrower set of values and are therefore more meaningful.

Misunderstanding Assertion Behavior

I've discussed some of the gotchas when using overly broad assertions, but fine-grained assertions can hide subtle, surprising behavior as well.

Example: Playwright

Setting aside unit testing for a moment, the browser automation library Playwright offers a Jest-style matcher called .toBeHidden(). Playwright's documentation describes this matcher as follows (emphasis mine):

Ensures that Locator either does not resolve to any DOM node, or resolves to a non-visible one.

The first part of this sentence is surprising. .toBeHidden() is a broader assertion than its name suggests. If an application never renders an element, .toBeHidden() still passes, even if the author's intention was to ensure the element exists, but in a hidden state.

Example: React Testing Library

React Testing Library's getBy queries implicitly assert by throwing an error when an element isn't located, and are often used without expect.

However, it can be easy to forget that queryBy variants return null rather than throwing, and can't be used as implicit assertions like their getBy complements.

The lesson is to read the documentation for your assertions carefully. Names can be misleading. Always force assertions to fail to make sure they're working as advertised and intended.

Forgetting to Call a Matcher

Here's an embarrassing mistake I made migrating a Mocha/chai-http test to Jest/supertest.

expect(response).to.be.json functions as expected in Mocha, but becomes a no-op in Jest when naively updated to expect(response).toBe.json. The property .json is undefined, and even if it were defined, it'd be a no-op without parentheses to call the function. The same mistake can appear in assertions like expect(someValue).toBeTrue, which looks sensible from a linguistic perspective, but is do-nothing code.

Jest offers expect.assertions(number), which holds you accountable to calling a certain number of assertions in a test. This adds a bit more protection against forgetting to call matchers and asynchronous assertions that run after the test ends. Linting can also identify do-nothing expressions and unused variables.

Misusing Mocks

Writing mocks can be time-consuming so you can be tempted to cut corners. Poorly written test mocks can hide bugs in application code. Consider the following Sequelize database query:

const user = await User.findOne({ where: { id } });

If User.findOne was mocked to ignore its argument and unconditionally return {id: 1, name: "Alan"}, the test would still pass, even if the function call argument was incorrect:

import User from "./user";

jest.mock("./user", () => ({ findOne: jest.fn() }));

describe("User", () => {
  it("should create and retrieve a user", async () => {
    User.findOne.mockResolvedValueOnce({
      id: 1,
      name: "Alan",
      createdAt: new Date(),
      updatedAt: new Date(),
    });

    // a bad call made by the code under test
    const user = await User.findOne({ nonsenseArg: 42 });

    // passes, even though the call to findOne is broken!
    expect(user).toEqual(
      expect.objectContaining({
        id: 1,
        name: "Alan",
        createdAt: expect.any(Date),
        updatedAt: expect.any(Date),
      })
    );
  });
});

A solution is to assert that a particular argument was passed to the User.findOne() mock using expect(User.findOne).toHaveBeenCalledWith({where: 1}). Although this helps to avoid a false positive, it can make refactoring and writing tests frustrating.

Similarly, in the early days of React, the Enzyme testing library supported invasive mocks of implementation details, like the setState() hook. Such mocks can easily be misused to strongarm the suite into 100% coverage, failing to test the component's actual behavior by injecting values that cause the component to behave differently than it would at run time.

Using Regex Matching Incorrectly

Testing libraries like Playwright, React Testing Library, and Jest offer many matchers which accept regexes. It's easy to forget that /Hello./g matches "Hello world" and "Hello!" because . is a regex special character, not a period, and the regex doesn't have anchors. Perhaps /^Hello\.$/ was intended. Using a regex when a plain string match would suffice is a testing code smell.

Even when using plain strings, inadvertently using substring matches when exact matches were intended can cause tests to pass when they shouldn't. Luckily, modern UI testing frameworks like Playwright and React Testing Library offer strict assertions and queries by default, failing with a clear error if multiple elements match an overly broad query.

Copy-Paste Errors

Testing mistakes often come down to silly copy-paste errors and typos. These are particularly common in test suites with long chains of similar assertions.

For example, when testing a form with a few buttons, a tester might create the following suite:

import { expect, test } from "@playwright/test";

// simple page for demonstration
const html = `<!DOCTYPE html><html><body>
<form>
  <input type="submit" value="Submit">
  <input type="reset" value="Reset">
  <button type="button">Cancel</button>
</form>
</body></html>`;

test.describe("form", () => {
  test.beforeEach(({ page }) => page.setContent(html));

  test("submit button exists", async ({ page }) => {
    await expect(page.getByRole("button", { name: "Submit" })).toBeVisible();
  });

  test("cancel button exists", async ({ page }) => {
    await expect(page.getByRole("button", { name: "Cancel" })).toBeVisible();
  });

  test("reset button exists", async ({ page }) => {
    await expect(page.getByRole("button", { name: "Submit" })).toBeVisible();
  });
});

Here, the author forgot to adjust the content of the third test block after copy-pasting the first block twice. The test name was updated, so the output misleadingly makes it seem like the 'reset' button is being tested. Removing the <input type="reset" value="Reset"> element from the page doesn't cause the test case to fail as it should.

In the era of GPT-generated testing code, a hallucination like this is easy to overlook.

On more complex tests that trigger JS code, a coverage tool can help catch these mistakes. Static analysis tools available as IDE plugins can also detect duplicate code.

Using Incorrect Properties in Configuration Objects

Configuration objects are a readable way to pass arguments to a function, making up for the fact that JS doesn't support named arguments. Modern UI testing libraries use them liberally.

However, if you're using plain JS rather than TypeScript, it's easy to make a call to a function with a typo or incorrect property name in the configuration object:

screen.getByRole("heading", { value: "Hello" }); // incorrect!

This React Testing Library query should use the name: key rather than value:. JS will silently ignore the argument, possibly retrieving an unexpected element or failing to implicitly assert the header's user-visible text.

This scenario can also creep up in global configuration files and strings in general. In addition to type checking and linting, the usual strategies for false positive avoidance described throughout this post apply.

Misusing Snapshot Tests

Snapshot tests let you diff a live UI component (for example, a React or Vue component) against a version-controlled, stringified snapshot of the component. If the strings match, the test passes.

Be careful, though: snapshot testing can lead to false positives if the reference snapshot is updated to match a broken component. Jest makes it effortless to update all failing snapshots in one command, even when some shouldn't be updated. Hastily updating snapshots without careful examination can bake false positives into tests.

Unlike traditional assertions, it's not as apparent from glancing at a large snapshot whether it's accurate or not, allowing bad snapshots to be handwaved through a code review. In some cases, the content of external snapshot files may not be examined at all.

Having a strong code review culture that examines tests as critically and thoroughly as application code is another tool to mitigate false positives.

Wrapping Up

After reading this article, I hope you'll walk away with a more critical eye on your test suites. This is far from an exhaustive list, so be on the lookout for other pitfalls, especially when you're onboarding a codebase or using an unfamiliar testing library.

The testing errors we've examined are a subclass of logic errors in general applications. Working cautiously to guard against and surface silent errors helps ensure the code you're working with stays correct.

Happy debugging!

P.S. If you liked this post, subscribe to our JavaScript Sorcery list for a monthly deep dive into more magical JavaScript tips and tricks.

P.P.S. If you need an APM for your Node.js app, go and check out the AppSignal APM for Node.js.

Puppeteer in Node.js: More Antipatterns to Avoid

Greg Gorlen — Wed, 21 Jun 2023 11:00:00 +0000

Puppeteer is a powerful browser automation library for web scraping and integration testing. However, the asynchronous, real-time API leaves plenty of room for gotchas and antipatterns to arise.

This article is part of a series, starting with Avoiding Puppeteer Antipatterns and Puppeteer in Node.js: Common Mistakes to Avoid. In this post, we'll add another dozen antipatterns to the list. There will be no overlap with previous installments, so you may wish to start with those.

While these antipatterns aren't quite full-fledged mistakes, weeding them out of your scripts (or being judicious when employing them) will increase the reliability of your Puppeteer code.

Let's begin.

Prerequisites

We will assume you are familiar with ES6 JavaScript syntax, promises, the browser DOM, and Node, and have written a few Puppeteer scripts already.

At the time of writing, the version of Puppeteer used was 20.3.0.

Onward to the antipatterns!

Antipatterns to Avoid in Puppeteer for Node.js

Underusing `page.goto`

I often see scraping scripts automating a search by:

Navigating to a website's landing page.
Accepting a cookie banner.
Typing a search term into an input box.
Clicking a button to submit the query.
Waiting for the second navigation to complete.

While this may make sense for testing, in scraping contexts these steps can often be bypassed by adding a query parameter such as https://www.some-site.com/search?q=search+term and using page.goto(searchResultURL) directly. Skipping the intermediate page speeds up the script, requires less code, and typically improves reliability.

The same is often true for automation involving iframes. In many cases, the frame source URL can be navigated to directly, bypassing the hassle of working with the parent document. If the frame source isn't known in advance, you can extract it and strip off the outer document with a goto:

const frame = await page.waitForSelector("iframe");
const src = await frame.evaluate((el) => el.src);
await page.goto(src, { waitUntil: "domcontentloaded" });

The code simplification may be worth the cost of the extra (probably cached) load.

Sometimes clicking links causes unpredictable, fussy navigation behavior, such as unwanted popups. In such cases, consider following the pattern in the above snippet, extracting the link's href property and plugging it into page.goto.

Using `page.on` Vs. `page.waitForRequest` or `page.waitForResponse`

page.on("request", handler) and page.on("response", handler) are callback-based event listeners. These are useful for intercepting and processing all requests or responses, but can be awkward to use with asynchronous control flow.

In cases when you're waiting for one or more specific responses to arrive, instead of chaining your dependent code from the callback or promisifying page.on, consider using page.waitForRequest or page.waitForResponse. These handy methods are essentially promisifed page.on() handlers.

For dialogs, promisification is unavoidable, as Puppeteer doesn't offer a page.waitForDialog wrapper at the present time. However, page.once is a handy way to avoid having to remove the listener once you've intercepted the dialog:

const dialogDismissed = new Promise((resolve, reject) => {
  page.once("dialog", async (dialog) => {
    await dialog.dismiss();
    resolve(dialog.message());
  });
});

/* take action to trigger the dialog */

const msg = await dialogDismissed;

The above code has a subtle issue: there's no timeout, so your script can silently hang forever. You can add a timeout as follows:

const timeout = 30_000;
const dialogDismissed = new Promise((resolve, reject) => {
  const timeoutId = setTimeout(reject, timeout);
  page.once("dialog", async (dialog) => {
    clearTimeout(timeoutId);
    await dialog.dismiss();
    resolve(dialog.message());
  });
});

/* take action to trigger the dialog */

const msg = await dialogDismissed;

Since requests and responses are liable to occur in any order at any time, page.once is less useful for those events than it is for dialogs, which are usually predictable.

Note that, in general, you won't need to use new Promise much in Puppeteer. Promisifying a promise-based API is known as the explicit promise constructor antipattern and usually appears when programmers aren't accustomed to working with promises.

Not Using Specific Wait or Evaluate Methods

As with request and response handlers, many Puppeteer methods have a hierarchy of generality. Here are the evaluate-family calls (roughly), from general to specific:

page.evaluate() can do just about anything any other Puppeteer API call can do in the browser. It's powerful but not specific.
page.$eval() and page.$$eval() are shorthands for common-case page.evaluate() calls that immediately run a document.querySelector() or document.querySelectorAll() as their callback's first step.
page.waitForFunction() is shorthand for a page.evaluate() that registers a MutationObserver or requestAnimationFrame loop that repeatedly checks a condition, then returns when the condition becomes true.
page.waitForSelector() is shorthand for a page.waitForFunction() that blocks until a specific selector matches an element in the DOM.

It's an antipattern to use a general method when a specific one exists that's tailored for the job. For example:

await page.evaluate(() => {
  const elements = document.querySelectorAll(".foo-bar");
  return [...elements].map((el) => el.textContent.trim());
});

Versus:

await page.$$eval(".foo-bar", (elements) => {
  return elements.map((el) => el.textContent.trim());
});

Not Reusing Browsers

Launching browsers is a heavy undertaking. It's a good idea to clear browser state after each run to maintain idempotency when testing, and in web applications that use Puppeteer with Express to perform tasks. But, in many cases, a browser can be reused safely, relying on pages to encapsulate tasks.

When business logic and safety can accommodate it, browser (or even page reuse) can provide dramatic efficiency gains.

Scraping the DOM Rather than Responses

Many web applications rely on information from JSON data, either embedded inside a <script> element or as XHR response payloads. Instead of figuring out how to extract the data from the DOM, it's often useful to intercept the responses or pull the information out of a <script>. Removing the fickle presentation layer from the process can make your code more reliable.

While raw data is likely more stable than the DOM, in some cases JSON payload structures are subject to change or may be harder to identify and parse than the DOM. Although responses aren't always a useful way to scrape data, it's worth popping open the network tab to try to find the response that has the data in it. Occasionally striking gold makes it worth the effort.

Using XPath Instead of CSS Selectors

Since XPath tends to be more verbose and trickier to write correctly than CSS selectors, CSS selectors should be preferred over XPath when possible.

Puppeteer 19.7.1 introduced a ::-p-text selector, which covers a good deal of XPath's common use case in Puppeteer, selecting elements by text.

Using Attribute CSS Syntax for Classes

CSS selectors have special and useful syntax for selecting elements by their attributes. For example:

<label for="username"></label>

You can select this with page.$('[for="username"]'), which is fine, but problems arise when applying this syntax to classes:

<div class="row align-items-center"></div>

Here, there's good reason to prefer .row.align-items-center over [class="row align-items-center"]. The dot syntax is easier to read and write, and is agnostic of ordering and additional attributes. If the class list changes to:

<div class="align-items-center row"></div>

Or:

<div class="row align-items-center p-2"></div>

Then the attribute selector fails. To make the two approaches interchangeable, the attribute selector ~ could be used: [class~="row"][class~="align-items-center"]. The verbosity makes the antipattern obvious.

There are situations suited to the [class="..."] pattern — for example, selecting elements that have a consistent prefix attribute name with a generated postfix: [class^="p-"].

Adding Premature Abstractions

In scraping and testing scenarios, I often see programmers writing classes and functions before they've correctly written their scraping or testing logic. In many cases, these abstractions make their scripts harder to debug than they would otherwise, introducing promise-management issues, memory leaks, and other sources of confusion.

I typically follow the Grug-Brained Dev's advice:

...one thing grug come to believe: not factor your application too early!

Once the scraper or testing block has been written correctly, then consider breaking the code into logical chunks.

Even if this is done correctly, unnecessary abstractions can still be problematic. In the case of tests, seeing the imperative steps in a longer test can make for easier maintenance than a series of nested helper functions (at the expense of a bit of repetition).

Not Cleaning Up Browser and Page Handles with `finally`

Even if you take care to ensure browser.close() is called, it's common to forget errors, which can prevent the browser from closing.

Add a finally block that calls browser.close() for every browser.launch() call to ensure proper resource cleanup.

Not Using Built-in Selectors

As mentioned above, Puppeteer offers a ::-p-text p selector, along with ::-p-aria and ::-p-xpath. Prefer using these over hand-rolled alternatives.

Here's an example of clicking a button based on its text content:

await page.setContent(`<button>Click me</button>`);
const btn = await page.waitForSelector("button::-p-text(Click)");
await btn.click();

The above code determines a match based on a substring of the text content, case-sensitively.

The >>> and >>>> combinators let you traverse shadow roots, with >>> traversing deep shadow roots and >>>> exploring one root deep.

Usual selection methods like page.waitForSelector, page.$eval, page.$$eval, and page.evaluate all work with built-in selectors. Additionally, Puppeteer has deprecated page.waitForXPath and page.$x, unifying the selector API.

Not Using `userDataDir`

Logins can be tricky to automate. Two-factor auth, input fields with complex asynchronous validation and masks, redirects, and iframes abound.

Instead of the hassle, you can set a userDataDir and log in manually in an idling Chromium browser launched by Puppeteer. As long as the session persists, you can go about your automation task directly from the site's dashboard.

Even in cases when you decide to automate login, persisting the session offers performance improvements.

Not Using Playwright for User-Facing Testing in Node.js

Microsoft's newer Playwright library offers a different element selection philosophy than Puppeteer. Puppeteer scripts tend to rely on CSS selectors and XPath. In contrast, Playwright's approach prioritizes user-facing attributes such as accessible roles, text, and titles.

Third-party testing packages like expect-puppeteer and pptr-testing-library attempt to bring the user-facing philosophy to Puppeteer. However, Playwright offers this style of testing out of the box. Playwright is opinionated and discourages non-user-facing selection methods as well as its inherited Puppeteer-style API, which it has mostly deprecated.

For most web scraping tasks, however, it's natural to use Puppeteer-style CSS selectors. I haven't seen evidence of any benefits in adhering to user-facing principles when web scraping. Puppeteer's simpler API gets out of the way a bit more. I appreciate using it as a thin, unopinionated wrapper on already-working browser code, adding controlled events and waits when necessary.

Wrapping Up

In this article, we covered a variety of antipatterns that can degrade the quality of Puppeteer automation scripts.

A central theme that helps avoid these antipatterns is to use the most precise tool for the job. Choose focused, high-level API methods to avoid the complexities of more powerful, low-level methods that are less idiomatic and expose unnecessary details.

Additionally, I've advocated for treating Puppeteer's evaluation and wait APIs as simple wrappers on the browser console, leaving user-facing testing principles to Playwright.

Happy coding!

P.S. If you liked this post, subscribe to our JavaScript Sorcery list for a monthly deep dive into more magical JavaScript tips and tricks.

P.P.S. If you need an APM for your Node.js app, go and check out the AppSignal APM for Node.js.

Puppeteer in Node.js: Common Mistakes to Avoid

Greg Gorlen — Wed, 15 Feb 2023 15:20:19 +0000

Puppeteer is a powerful Node.js browser automation library for integration testing and web scraping. However, like any complex software, it comes with plenty of potential pitfalls.

In this article, I'll discuss a variety of common Puppeteer mistakes I've encountered in personal and consulting projects, as well as when monitoring the Puppeteer tag on Stack Overflow. Once you're aware of these problematic patterns, you can write more robust scraping and testing code, while spending less time debugging and wading through arcane Puppeteer errors.

Pre-requisites

The article was written using Node 18, Puppeteer 19.4.1, Chrome 108.0.5359.125, and Firefox 108.0.1.

We will assume you are familiar with ES6 JavaScript syntax, browser development tools, the browser DOM, and Node, and have previously written some Puppeteer scripts.

Now let's examine the pitfalls of Puppeteer.

Common Pitfalls in Puppeteer for Node.js

Attempting to Return Objects and DOM Elements from `evaluate` Callbacks

The following snippet should be a familiar pattern to those who've previously used Puppeteer:

const puppeteer = require("puppeteer");

let browser;
(async () => {
  browser = await puppeteer.launch();
  const [page] = await browser.pages();
  const url = "https://www.example.com";
  await page.goto(url, { waitUntil: "domcontentloaded" });

  const element = await page.evaluate(() => {
    // executed in the browser
    return document.querySelector("h1");
  });
  console.log(element); // => always an empty object, {}
})()
  .catch((err) => console.error(err))
  .finally(() => browser?.close());

Note: I'll skip the IIFE, import, and error-handling boilerplate in the remainder of this article.

The code above attempts to return a DOM element from the browser context back to Node for further processing (clicking it, typing into it, extracting its text content, etc.). However, the evaluate call, which runs code in the browser context, resolves to an empty object in Node. DOM elements are complex structures with circular references and cannot be readily serialized and deserialized. These elements can't be decoupled from the browser environment in a meaningful way.

This behavior isn't specific to Puppeteer. Running JSON.stringify(document.querySelector("h1")) on a page with a header element should return '{}' on a Chromium-based browser. Firefox gives 'null', also indicating a serialization failure.

One solution is to use page.$("h1") (or the more general page.evaluateHandle) to create a Puppeteer ElementHandle that exposes an interface to the DOM element. Alternatively, you can use JSHandle which exposes an interface to the JS object. These interfaces enable you to run code in the browser context on the element, possibly to extract serializable data such as text content or element properties, or issue trusted events. Be sure to dispose of these handles when you no longer need them to avoid memory leaks.

I've used evaluate here as it's the most general way to run code in the browser, but the same behavior applies to $eval and $$eval as well. These methods are shorthands for the common case when document.querySelector or document.querySelectorAll is the first line in the evaluate callback.

Trying to Access Variables from an `evaluate` Callback

At times, variables may appear to be in scope from an evaluate callback in Puppeteer when they aren't. The following example is a bit contrived because we could use $eval, but it nevertheless illustrates the pattern succinctly:

const selector = "h1";
const text = await page.evaluate(() => {
  return document.querySelector(selector).textContent;
});

Here, selector seems like it should be in scope of the evaluate callback, but it doesn't exist when the callback runs in the browser, throwing Evaluation failed: ReferenceError: selector is not defined. Yet again, serialization is the culprit: the browser is a completely separate process from Node that doesn't have your Node variables in scope.

Using the string version of evaluate makes the situation clearer:

const selector = "h1";
const text = await page.evaluate(`
  document.querySelector("${selector}").textContent
`);
console.log(text); // => Example Domain

However, building a string can lead to quoting problems. This motivates the more general approach — passing data to the browser by including extra arguments to evaluate:

const selector = "h1";
const text = await page.evaluate((selector) => {
  return document.querySelector(selector).textContent;
}, selector);

This parameter-passing pattern also applies to other important Puppeteer calls, such as page.waitForFunction. A subtle difference is that waitForFunction's second argument is a configuration options object, followed by the variable parameter arguments:

page.waitForFunction((arg1, arg2) => {...}, options, arg1, arg2);

This Stack Overflow post offers tips on passing complex arguments to evaluate calls.

Assuming Browser DevTools Is the Same as Node

Puppeteer programmers stuck on a bug often claim their selectors work in browser developer tools, but fail in Puppeteer. Unfortunately, there's no guarantee that code that works in browser developer tools will also work in Puppeteer. Here are some reasons why:

Developer tools exposes iframe and shadow root subtrees. Puppeteer requires these trees to be explicitly expanded. It’s worth noting that Microsoft's Playwright library has locators that expand the shadow DOM by default.
By the time you get around to interacting with developer tools, the page has typically loaded its resources and executed its JS scripts. In Puppeteer, you can use waitForSelector calls to ensure JS-injected elements are available before interaction.
When working in an unautomated browser's developer tools, the website's server trusts you and delivers a full experience. In Node, Puppeteer scripts are often detected as bots, so they are blocked outright or served a restricted version of a page. Adding console.log(await page.content()) is an easy way to verify that your HTML structure in Puppeteer is what you expect.

Recognizing developer tools and Node as distinct environments goes a long way to ensuring smooth translations from your developer tools exploration code to the final Puppeteer script.

Assuming Puppeteer's Headless Mode Works the Same as Headful

Just as DevTools is distinct from Node, it's a mistake to assume that Puppeteer's headless mode works the same as headful mode. Websites have a much easier time detecting scripts as bots when in headless mode than in headful mode.

As with the above tip, console.log(await page.content()) is a great way to ensure a document is what you expect. If a selector you see in the developer tools or in headful mode isn't in the headless log, there's a good chance you've been blocked.

Not Using `Promise.all` When Triggering Navigation with a Click

Navigation is a common point of failure in Puppeteer scripts. The following code is unsafe:

await page.click("#submit"); // trigger a navigation
await page.waitForNavigation();

In fact, this is a race condition. If the navigation resolves before waitForNavigation has the chance to run, the script may throw a timeout error. The correct pattern is:

await Promise.all([
  page.waitForNavigation(),
  page.click("#submit"), // trigger a navigation
]);

Or:

const navigationPromise = page.waitForNavigation();
await page.click("#submit"); // trigger a navigation
await navigationPromise;

In these examples, the navigation wait promise is set before the navigation is triggered, ensuring that it will resolve as intended.

Making Unnecessary Calls to `waitForNetworkIdle` or `waitForNavigation`

Another navigation-related mistake is making spurious calls to waitForNetworkIdle or waitForNavigation. For example:

await page.goto("some url");
await page.waitForNavigation();

This is logical: we want to trigger a navigation with goto, then wait for that navigation to settle. But goto already waits for navigation, so the second waitForNavigation is waiting for a navigation that's already occurred, causing a timeout.

It's a similar story when waiting for an idle network state, either with a waitForNetworkIdle call or page.goto(url, {waitUntil: "networkidle0"}). In fact, waiting for an idle network can make a script hang forever if the automated page keeps enough long-running connections open.

networkidle2 is usually safer since it tolerates two long-running connections. However, it is often used out of laziness or lack of awareness in place of the clear-cut waitForSelector and waitForFunction predicates.

Using Infinite Timeouts

It's a mistake to set any timeout to 0 — for example, with page.setDefaultTimeout(0). Infinite timeouts introduce the potential for the script to block forever when encountering an unexpected state, without giving a clear error message. Under most circumstances, when a script hangs on a selector or navigation for more than a few minutes, it should log an error and either exit so its maintainer can fix the problem, or restart itself if it should keep attempting to do something.

Usually, when I come across infinite timeouts in Puppeteer scripts, it's an artifact of attempting to fix a deeper issue, like the script being detected as a bot and blocked. But the infinite timeout makes these errors harder to detect and resolve by stifling them and causing a silent hang.

Forgetting to Await a Puppeteer Call

Almost all Puppeteer API calls are asynchronous. The reason for the asynchronous interface is that the browser runs in a separate process from Node. Puppeteer's methods send and receive data and wait for the browser process to respond, much like networking or file system operations. The Node process can use this time to perform CPU-bound work.

A common mistake is forgetting to await a promise returned by a Puppeteer call. This can lead to confusing and non-deterministic errors and race conditions.

For example, omitting await on a page.goto call may result in a Protocol error (Page.navigate): Target closed or Execution context was destroyed, most likely because of a navigation.

Awaiting Synchronous Puppeteer Calls

One solution to the missing await problem described above is to await everything, but this can lead to confusion as well.

For example, I see the following pattern often:

await page.on("request", (request) => {
  /* handle the request */
});
// do stuff after the request has been handled

Since page.on doesn't return a promise, it's easy to forget that // do stuff after the request has been handled runs before the request handler callback. The callback is in a different promise chain.

To solve this, use Puppeteer's page.waitForRequest (or page.waitForResponse) instead of page.on, which acts as a shorthand for manually promisifying the page.on callback.

Creating Node.js Memory Leaks with `page.on` Listeners

Consider the following code:

for (;;) {
  page.on("request", () => {});
  await setTimeout(10); // from "timers/promises"
}

This code reinstalls an event listener over and over again, slowly eating memory. The problem seems obvious in this minimal example, but I've seen it buried in the midst of moderately-complex, long-running jobs that eventually crash.

Misunderstanding `page.exposeFunction`

page.exposeFunction enables your browser code to trigger Node code. As with evaluate, you can't pass DOM elements or other non-serializable structures as parameters, so a typical use case is passing serialized data like JSON or text for periodic processing. In the common case, you'll use evaluate to extract data rather than exposeFunction.

Doing Too Much Work in Parallel

Another obvious pattern when seen in isolation is:

const urlsToScrape = [
  /* large array */
];

const browser = await puppeteer.launch();
const results = await Promise.all(
  urlsToScrape.map(async (url) => {
    await page.goto(url, { waitUntil: "domcontentloaded" });
    return page.$eval("h1", (element) => element.textContent);
  })
);

If urlsToScrape happens to be large enough, the memory and processor load from spawning dozens or hundreds of pages can bring a system down quickly. Consider puppeteer-cluster.

Wrapping Up

In this article, we've seen a variety of mistakes and gotchas that every Puppeteer programmer should be aware of.

I hope these tips will save you from making the same mistakes I've made over the years, so you can keep your tests and scripts running smoothly.

P.S. If you liked this post, subscribe to our JavaScript Sorcery list for a monthly deep dive into more magical JavaScript tips and tricks.

P.P.S. If you need an APM for your Node.js app, go and check out the AppSignal APM for Node.js.

DEV Community: Greg Gorlen

Avoiding False Positives in Node.js Tests

Prerequisites

Common False Positive Test Patterns in Node.js

Using equal() Rather Than strictEqual()

Using Overly General Assertions

Using Shallow Equality for Deep Comparisons

Misunderstanding Assertion Behavior

Example: Playwright

Example: React Testing Library

Forgetting to Call a Matcher

Misusing Mocks

Using Regex Matching Incorrectly

Copy-Paste Errors

Using Incorrect Properties in Configuration Objects

Misusing Snapshot Tests

Wrapping Up

Puppeteer in Node.js: More Antipatterns to Avoid

Prerequisites

Antipatterns to Avoid in Puppeteer for Node.js

Underusing page.goto

Using page.on Vs. page.waitForRequest or page.waitForResponse

Not Using Specific Wait or Evaluate Methods

Not Reusing Browsers

Scraping the DOM Rather than Responses

Using XPath Instead of CSS Selectors

Using Attribute CSS Syntax for Classes

Adding Premature Abstractions

Not Cleaning Up Browser and Page Handles with finally

Not Using Built-in Selectors

Not Using userDataDir

Not Using Playwright for User-Facing Testing in Node.js

Wrapping Up

Puppeteer in Node.js: Common Mistakes to Avoid

Pre-requisites

Common Pitfalls in Puppeteer for Node.js

Attempting to Return Objects and DOM Elements from evaluate Callbacks

Trying to Access Variables from an evaluate Callback

Assuming Browser DevTools Is the Same as Node

Assuming Puppeteer's Headless Mode Works the Same as Headful

Not Using Promise.all When Triggering Navigation with a Click

Making Unnecessary Calls to waitForNetworkIdle or waitForNavigation

Using Infinite Timeouts

Forgetting to Await a Puppeteer Call

Awaiting Synchronous Puppeteer Calls

Creating Node.js Memory Leaks with page.on Listeners

Misunderstanding page.exposeFunction

Doing Too Much Work in Parallel

Wrapping Up

Using `equal()` Rather Than `strictEqual()`

Underusing `page.goto`

Using `page.on` Vs. `page.waitForRequest` or `page.waitForResponse`

Not Cleaning Up Browser and Page Handles with `finally`

Not Using `userDataDir`

Attempting to Return Objects and DOM Elements from `evaluate` Callbacks

Trying to Access Variables from an `evaluate` Callback

Not Using `Promise.all` When Triggering Navigation with a Click

Making Unnecessary Calls to `waitForNetworkIdle` or `waitForNavigation`

Creating Node.js Memory Leaks with `page.on` Listeners

Misunderstanding `page.exposeFunction`