DEV Community

Daniel Pertu
Daniel Pertu

Posted on

344 tests run in 675ms, no CI runs them, and the 24th file is the only one that needs a database

I counted our test suite rather than describing it, because every project's story about its own testing is more flattering than the folder.

The census

24 files matching **/__tests__/**/*.test.ts, 4,208 lines, 308 literal it( or test( calls. Running everything except one file:

Test Files  23 passed (23)
     Tests  344 passed (344)
  Duration  675ms (transform 1.14s, setup 273ms, import 2.23s, tests 465ms, environment 1ms)
Enter fullscreen mode Exit fullscreen mode

344 at runtime against 308 in the source, because two files use it.each and a table expands into one test per row. If you ever audit a suite by grepping for it(, that is the size of the error you should expect: about 12 per cent, all of it in the files that test a formatter against a list of inputs.

The whole thing by area, counting source-level it( calls rather than expanded ones, which is why the tests column sums to 308 and not 344:

area files lines tests
pricing and currency display 2 607 50
billing and Stripe 4 1,141 75
gameplay, against a real database 1 544 13
content registry and page metadata 2 337 24
admin session health 1 241 16
free tools 2 275 24
everything else 12 1,063 106

465 milliseconds of actual test execution. Nothing in here renders a component, starts a server, launches a browser or opens a socket, which is why.

Zero component tests, and the config is why

export default defineConfig({
    test: {
        environment: 'node',
        include: ['**/__tests__/**/*.test.ts'],
        setupFiles: ['./__tests__/setup.ts'],
    },
    resolve: { alias: { '@': path.resolve(__dirname, '.') } },
})
Enter fullscreen mode Exit fullscreen mode

Two lines there decide the shape of everything else. environment: 'node' means no DOM, and include ends in .test.ts, so a .test.tsx file would not even be collected. There is no jsdom, no happy-dom and no testing library in our dependencies: the only test packages installed are vitest and @vitest/coverage-v8.

We have 118 component files. Zero of them are rendered by a test.

I am not going to defend that as a strategy, but I will describe it accurately, because "we have no component tests" and "we have no tests of the behaviour those components implement" are different statements. What the suite does instead is test the pure functions the components call and the data they render: currency resolution, price copy, the content registry's invariants, the poster layout maths, the profit calculator's arithmetic, route lists, a label formatter, a clock. The parts that are genuinely untested are the parts where a user's click turns into a state change.

That is a real gap and the honest reason for it is cost. A React render test needs a DOM, a library, and a set of decisions about what counts as the user-visible contract, and it pays off most in a codebase with many people in it. The 344 tests that exist took the cheap half of the value and stopped.

12 mocks, in 5 files

__tests__/billing/checkout-route.test.ts    4
__tests__/billing/stripe-webhook.test.ts    3
__tests__/maintenance.test.ts               3
__tests__/billing/checkout-session.test.ts  1
__tests__/game/abandoned.test.ts            1
Enter fullscreen mode Exit fullscreen mode

19 of the 24 files mock nothing at all. That is not virtue, it is a consequence of the same shape: a function that takes a country code and returns a currency has nothing to mock. Where the mocks cluster is exactly where the code touches something it does not own. The eight mocks in the billing files replace the Stripe client, the Supabase server client, the database handle and the rate limiter, so 75 billing tests run without a single outbound request.

The most interesting of the five is abandoned.test.ts, which mocks a single module so it can test the internal route handler that sweeps up quizzes nobody came back to, without that handler reaching a database:

const endAbandonedSessions = vi.hoisted(() => vi.fn())
vi.mock('@/utils/game/end-abandoned', () => ({ endAbandonedSessions }))
Enter fullscreen mode Exit fullscreen mode

One vi.hoisted plus one vi.mock is the whole seam, and it is the difference between a 17-test file that runs in milliseconds and a file that needs Postgres.

The 24th file

The excluded file is gameplay.integration.test.ts: 544 lines, 13 tests, no mocks whatsoever, and a database connection. It exercises the real server actions rather than imitations of them, which is the point of having it, and it writes rows to do that: a venue, a table, a session, the lot. Every row it creates is tracked and deleted in afterAll.

Here is the part worth publishing, because it is the part that bit us. Our test setup file is seven lines:

import { config } from 'dotenv'
config({ path: '.env.local' })
Enter fullscreen mode Exit fullscreen mode

And .env.local points at the same Supabase database as production. So pnpm test is a command that writes to the production database, and the cleanup is an afterAll, which does not run if you interrupt the process with Ctrl+C. We have had exactly that: stale sessions sitting in the real venue after a cancelled test run, confusing a later production check.

The right fix is a second database and a DATABASE_URL in a .env.test that cannot be the live one, enforced by the test file refusing to run against a host it does not recognise. The cheap intermediate fix is the one in the file name: the integration test is the only file that needs the database, so it is the only file you have to think about, and --exclude makes the other 23 safe to run anywhere, which is the command above.

There is also no describe.skip, no it.skipIf, and no environment guard anywhere in the suite. If the database is unreachable, that file fails rather than skipping, which I prefer: a silently skipped integration test is worse than a red one.

Nobody runs any of this automatically

There is no .github/workflows directory. Nothing in the build command runs vitest. The deploy platform builds and ships on push without consulting the suite at all.

So 344 tests that take 675 milliseconds exist, and the trigger for them is a human remembering. @vitest/coverage-v8 is installed and there is no coverage script, which means it was added in a moment of good intentions and never used again.

For a solo project this is less reckless than it sounds, because a 675 millisecond suite gets run constantly during development. It is still the single highest-value missing piece of infrastructure here: the tests are fast, hermetic for 23 of 24 files, and already written. A CI job would be ten lines of YAML and an --exclude.

The second gap is the WebSocket server, which is a separate package in the same repo and has zero test files. Its logic is the hardest thing in the project to test and the easiest thing to break.

Check it yourself

The suite's subjects are mostly public pages, so you can look at what 344 tests are actually defending:

  • 50 tests cover currency display and price copy, which is what makes pub-trivia.app/pricing show you prices in your own currency. Load it through a VPN exit node in another country and watch the symbols change.
  • 24 tests cover the two tools' maths: the poster layout at pub-trivia.app/tools/quiz-night-poster-maker and the arithmetic at pub-trivia.app/tools/quiz-night-profit-calculator. Both run entirely in your browser, so the Network tab stays empty while you use them.
  • 24 tests cover the content registry and page metadata, which is why every page under pub-trivia.app/guides has a title, a breadcrumb up to its hub, and links to its siblings. A page that forgot one fails a test rather than shipping.

And if you want the shape of your own suite in one command:

npx vitest run --exclude "**/*integration*" --reporter=basic
grep -rc "vi.mock(" __tests__ | grep -v ':0'
Enter fullscreen mode Exit fullscreen mode

The second line is the more revealing of the two. A file with no mocks is usually testing arithmetic; a file with four is usually testing a boundary. The ratio tells you what kind of suite you have long before the coverage number does.

Top comments (0)