167 tests across twelve files, and 80 of them are about money

vitest run on this repository:

 Test Files  12 passed (12)
      Tests  167 passed (167)
   Duration  18.18s

Twelve files, 167 tests, 2,286 lines. Against 201 .tsx files and 101 route files. This is the census I had never done, file by file, because "do we have tests" is a question with a yes or no answer and "what do they cover" is a question with a shape.

File	Tests	Lines
pricing/display-currency.test.ts	38	406
billing/managed-payments.test.ts	27	240
content/registry.test.ts	21	232
scoring.test.ts	16	112
billing/checkout-session.test.ts	15	246
gameplay.integration.test.ts	13	543
indexnow.test.ts	10	87
maintenance.test.ts	9	126
routes.test.ts	8	77
server-clock.test.ts	5	76
content/page-metadata.test.ts	3	103
shared-constants.test.ts	2	38

Nearly half the suite is about charging people correctly

Add the three money files: 38 plus 27 plus 15 is 80 tests, 48 per cent of the suite, for a product whose entire billing surface is a checkout session and a plan table.

That is not an accident and I would not reduce it. Currency and tax code is the only code in the product where being wrong is invisible until it is expensive, and where the inputs are hostile: a country code I did not choose, a plan that may be free, a price in a currency with no decimal places. 38 of those 80 tests are over one pure function that decides which currency to show you, which you can see the output of on the pricing page from wherever you happen to be reading this.

The other 42 got their own post a couple of days ago, so I will leave them there. What I had not noticed until I lined the files up is how the remaining 87 tests split.

• 42 tests on content, routing and indexing: the content registry, the metadata on every route file, the route lists, the indexing submitter.
• 34 tests on the quiz itself: the integration test, the scoring rules, the server clock. Those last two are the only files in this table I do not write about in any detail, because how the game decides things is the one part of this codebase that stays unpublished. The count is the point: a fifth of the suite.
• 11 tests on infrastructure: the maintenance switch and the shared constants.

Which means the half of the suite that is not about money is mostly about whether pages exist and say the right thing about themselves. For a product whose growth plan is 72 URLs, that is defensible. It is still a strange sentence to write: a quiz app whose test suite is more concerned with canonical tags than with quizzes.

The global setup file exists for one file out of twelve

// __tests__/setup.ts
import { config } from 'dotenv'

config({ path: '.env.local' })

Seven lines, loaded before every test run by vitest.config.ts. Exactly one of the twelve files needs it: gameplay.integration.test.ts, the only one that imports the database client and talks to a real Postgres. It is also the largest file in the suite, 543 lines, and it carries the fewest tests per line of any of them, which is what integration tests cost.

The other eleven files import project modules and vitest, and nothing else. Three of them read the filesystem, and that is the most interesting pattern in the suite: page-metadata.test.ts opens every route file and regexes export const metadata out of it, indexnow.test.ts reads the key file that has to exist in the right place, and shared-constants.test.ts reads the other package's source to prove two build roots agree on a number. Those are not unit tests of functions, they are assertions about the shape of the repository, and they catch the class of mistake no function-level test can: a new page that forgot to declare anything about itself.

The practical consequence of the setup file is that a fresh clone with no .env.local cannot run that one file at all, while the other eleven are completely unaffected by its absence. I would rather that file skipped itself with a clear message than failed, and the honest reason it does not is that the failure has never cost me anything.

Not one test renders a component

export default defineConfig({
    test: {
        environment: 'node',
        include: ['**/__tests__/**/*.test.ts'],
        setupFiles: ['./__tests__/setup.ts'],
    },
})

environment: 'node'. No jsdom, no happy-dom, and no testing library in package.json. The include pattern is *.test.ts, not .tsx.

So the configuration could not run a component test if one existed. 201 .tsx files, 167 tests, zero renders. Everything that is tested is a function that takes data and returns data, or a file that is read off disk.

This is the gap I would defend least comfortably. It is partly reasonable: the public pages are prerendered HTML built from one component set and a registry, so an assertion about the output of that pipeline is more useful than an assertion about one component in isolation, and 70 of the 72 pages in the sitemap can be checked by fetching them. It is partly just the path of least resistance, because adding jsdom to this config is twenty minutes and I have not spent them.

The clearest hole is the one the user touches most freely: the four free tools are client components with real logic in them, a PDF generator, a QR encoder, a planner that does arithmetic, and the suite has nothing to say about any of them. They are also the only part of the product you can use without an account, which is the opposite of the order you would choose.

What the census actually changed

Three things came out of lining the files up, none of which I would have got from a coverage percentage:

1. The suite's priorities are legible and roughly right. Money first, then whether the site can be found, then the game, then the switches.
2. The one file that needs the environment is the one file that is slow, and it taxes every other run.
3. The untested surface is not the surface I would have guessed. It is not the dashboard, which is at least covered indirectly by the integration test. It is the public, no-account, client-side tools.

A coverage number would have told me 167 tests pass, which I already knew. Twelve rows and a sum told me which 48 per cent of the effort went where the risk is priced in currency, and which part of the product a stranger can reach without a single test standing behind it.

Comments

[Jessica Williams]

The census format is more useful than a coverage number. On the integration file, Vitest has describe.skipIf, so you can skip that file with a clear message when the database variable is missing and keep fresh clones from failing. For the free tools, a few Playwright smoke tests (open the page, run the tool once, check the output) would cover the untested client-side surface without a jsdom setup.