Our website has 111 tests in four files, no test framework, and every one guards a silent failure

Here is the entire test setup for Notifio's website, a Next.js app with 38 pages, Stripe checkout, two webhooks and a Postgres table behind it:

"scripts": {
  "typecheck": "tsc --noEmit",
  "test": "node --test"
}

And the devDependencies, in full:

"devDependencies": {
  "@tailwindcss/postcss": "^4.3.3",
  "@types/mailparser": "^3.4.6",
  "@types/node": "22.20.1",
  "@types/react": "19.1.8",
  "@types/react-dom": "19.1.6",
  "drizzle-kit": "0.31.10",
  "postcss": "^8.5.20",
  "tailwindcss": "^4.3.3",
  "typescript": "5.9.3"
}

No Jest, no Vitest, no ts-node, no transform config, no jest.config.js, no setupFilesAfterEnv. node --test discovers the files, and Node strips the TypeScript itself, so the test files import .ts modules by path with the extension on:

import { describe, it } from "node:test";
import assert from "node:assert/strict";
import { convertForDisplay, formatForDisplay } from "../../src/lib/pricing/display-currency.ts";

What that buys, measured just now:

ℹ tests 111
ℹ suites 25
ℹ pass 111
ℹ fail 0
ℹ duration_ms 1697.48925

111 tests in 1.7 seconds across four files and 1,035 lines. The interesting thing is not the tooling, though, it is which four files they are.

The four

__tests__/pricing/display-currency.test.ts    353 lines
__tests__/seo/indexnow.test.ts                292 lines
__tests__/seo/metadata-length.test.ts         158 lines
__tests__/stripe/managed-payments.test.ts     232 lines

Each one opens with a comment saying why it exists, and I did not notice until I lined them up that all four say the same thing in different words.

Currency conversion:

/**
 * These prices are advertised next to a checkout that charges in GBP and lets
 * Stripe or the customer's card issuer do the conversion, both of which add a
 * fee. The property this module exists to guarantee is that our displayed number
 * is never the lower of the two, so most of these tests are about that direction
 * rather than about exact values.
 */

IndexNow submission:

/**
 * IndexNow fails quietly when it fails at all: a wrong host or a stale key
 * comes back as a bare 403 or 422 with no body, and nothing on the site looks
 * broken. The batching and filtering rules are therefore pinned here so that a
 * regression shows up as a failing test rather than as pages that stopped being
 * re-crawled by everything except Google.
 */

Metadata lengths:

/**
 * Search engines truncate a title past roughly 60 characters and a description
 * past roughly 160, so every page has to fit inside those budgets. The budget
 * that matters is the *rendered* one: the root layout applies a `%s | Notifio`
 * template, which silently spends ten of the sixty on every page but the home
 * page. Hand-counting that is what let three titles drift to exactly 60, so it
 * is counted here instead.
 */

Merchant-of-record routing:

/**
 * Managed Payments decides whether Stripe becomes the seller of record and
 * charges the customer's local VAT. Getting the routing wrong is expensive in
 * both directions, under-collecting tax we owe, or charging tax we don't, so
 * these tests pin the allowlist rather than the mechanism.
 */

Quietly. Nothing looks broken. Silently spends. Expensive in both directions.

Every one of these four is a piece of code whose failure mode is being wrong rather than being broken. A price that is three percent under what the card will be charged. A submission endpoint returning 403 to a key it never read. A title cut off mid-word in a search result. VAT not collected on a sale in a country that wanted it. None of those throws. None reaches Sentry. None produces a support email, because the customer does not know either. The first time you find out is when you go and look, and the reason you go and look is usually that something else went wrong first.

That is the selection rule, written down after the fact: the suite covers code where nobody will tell me it is broken.

What is not tested, which is most of it

Being specific about this is the only way the first half of the post means anything.

There are no component tests. No test renders a page. Nothing covers the Stripe webhook, the Resend inbound webhook, the contact form endpoint, the download redirect, the licence validation endpoint, the rate limiter, or the database layer. There is no end-to-end suite and no browser in CI. tsc --noEmit and next build are the only things between a broken page and production.

For most of that list the argument holds up. A route handler that throws returns a 500, which is a page with an error on it, which somebody notices, and there is a reporter wired up to catch it:

export async function register() {
  if (process.env.NEXT_RUNTIME === "nodejs") {
    await import("./sentry.server.config");
  }
  if (process.env.NEXT_RUNTIME === "edge") {
    await import("./sentry.edge.config");
  }
}

export const onRequestError = Sentry.captureRequestError;

A page that fails to build fails the deploy. A webhook that 500s gets retried by Stripe and shows up in Stripe's own dashboard as a failing endpoint. Those are loud, and loud failures have a different owner: monitoring, not tests.

Where the argument gets thin is the Stripe webhook, and I would rather say so than let the tidy principle carry it. A webhook can fail silently in exactly the way this suite exists to prevent: take a payment, return 200, and write no licence row, because the branch that decides which of two products was bought went the wrong way. That is money in and nothing delivered, with no error anywhere. It is the single clearest candidate for the fifth file, and the reason it does not exist yet is that the branch is covered by a different mechanism, the idempotency work I wrote about in Two products, one Stripe event, and the branch that must come first, rather than by a test asserting it.

"Loud" also assumes somebody is listening. With a 10 percent trace sample and a one-person team, that is an assumption and not a guarantee.

The one trick in here worth stealing

/** The conversion-fee buffer the module applies. Restated here on purpose: a */
/** test that imported the constant would pass if the constant were deleted. */
const FEE_BUFFER = 1.04;

This is the most useful line in the suite. The module applies a 4 percent buffer so the displayed figure sits above what the card will actually be charged. The natural way to test it is to import the buffer and multiply by it, and that test passes whatever the buffer is, including zero, including deleted. It verifies that the code agrees with itself.

Writing 1.04 in the test turns it into a second, independent statement of the requirement. Change the buffer in the module and the test fails, which is correct, because changing the amount by which every advertised price is inflated is a decision that should require touching two files and reading a failure message.

The same reasoning puts the search-engine budgets in the test file rather than in a shared constants module:

const TITLE_MAX = 60;
const DESCRIPTION_MAX = 160;

And it is why one of these tests reads application source code as text:

/** The suffix the root layout's title template appends to every child page. */
function titleTemplateSuffix(): string {
  const layout = readFileSync(LAYOUT, "utf8");
  const match = layout.match(/template:\s*"(.*?)"/);
  assert.ok(match, "root layout no longer declares a title template");
  const [, template] = match;
  assert.ok(template.includes("%s"), `unexpected title template: ${template}`);
  return template.replace("%s", "");
}

Reading your own source with a regex is normally a bad sign. Here it is the only honest option: the thing being tested is a number of characters that a Next.js layout spends on every page in the site, and that number is not exported, not importable, and not visible from any page's own metadata. Either the test knows what the template is or it is checking the wrong budget, which was the original bug. I wrote that one up on its own in Our title budget is 60 characters, the layout quietly spends ten of them, and a test reads our own source files to prove it.

Note the two assertions inside the helper. If the layout stops declaring a template, the test does not quietly fall back to counting bare titles and passing. It fails, and the message says the layout changed. A helper that returns a wrong value on a wrong assumption is how a suite becomes decorative.

The one place a glob earns its keep

/**
 * The alerts barrel uses extensionless imports that Node cannot resolve, so the
 * site files are loaded straight from the directory. Globbing rather than
 * listing them also means a new market file is covered the day it is added.
 */
const SITES_DIR = path.join(import.meta.dirname, "../../src/lib/alerts/sites");
const ALERT_SITES = (
  await Promise.all(
    readdirSync(SITES_DIR)
      .filter((name) => name.endsWith(".ts"))
      .map((name) => import(path.join(SITES_DIR, name))),
  )
).flatMap((module) => Object.values(module).flat());

That started as a workaround. Node's type stripping does not do the extensionless resolution that a bundler does, so importing the barrel that collects the alert site definitions does not work in a test. Reading the directory instead was the fix.

The accidental benefit is the better reason to keep it. There are three market files today, and the day a fourth is added it is inside the metadata budget check without anyone remembering to add it. A hand-written list of imports in a test file is a second place to update, and the failure mode of forgetting is a new page that nothing checks, which is the whole category this suite exists for.

Why no framework, specifically

Two reasons, and neither is purity.

The first is that 1.7 seconds and zero dependencies means the suite never becomes a thing you skip. There is no watch mode to learn, no config to get wrong, and no version matrix between a test framework, a transform, TypeScript and Next.js, which in my experience is where a day goes when a framework upgrade lands.

The second is that nothing in these four files needs a framework. There are no mocks except process.env, no snapshots, no DOM, no module interception. describe, it, and assert.ok with a decent message cover all 111 cases. node:test has had that since Node 18, and Node runs the TypeScript directly now, so the only honest argument left for pulling in a runner is features this suite does not use.

Where that stops being true is the day something here needs to render a React component or fake a network call. At that point a runner is the right answer and I will add one. Starting with it, for four files of pure functions, would have meant a config file older than any of the tests.

The summary I would put on a wall

Coverage is the wrong question when the number is going to be low anyway. The useful question is: for each piece of this system, what happens when it goes wrong, and who finds out? Code whose failures announce themselves needs monitoring. Code whose failures look exactly like success needs tests, and that set is usually much smaller than it feels, and almost always includes anything touching money or search engines.

For this site that question picked four files. You can see what three of them protect from the outside: the converted price and its footnote at notifio.app/pricing, the titles and descriptions of every page in notifio.app/sitemap.xml, and the tax treatment behind the comparison tables at notifio.app/compare. The merchant-of-record logic the fourth one pins is in Merchant of record for 27 countries, and deliberately off for our own, and the IndexNow failure that started that file is in A 200 from IndexNow does not mean it read your key.