We merged 25 games with no pages, and exactly two tests were allowed to fail

A provider on CogniPrep is a string. tsa, asvab, train-driver. There are 56 of them, and they live in one file that does nothing else:

const PROVIDER_IDS: Record<AssessmentProvider, true> = {
  'arctic-shores': true,
  hirevue: true,
  usps: true,
  tsa: true,
  // ... 52 more
};

A Record keyed on the union type rather than an array literal, so adding a provider to the type without adding it here is a compile error instead of a slug that quietly stops being recognised. The key order is load bearing too: it is the order providers are listed and numbered on the landing page, and a test fails if it ever diverges from the order of the metadata map it used to be derived from.

That array then fans out. The sitemap maps over it. The landing page section maps over it. The in-app provider search maps over it. The theme's forced-dark path list is checked against it. A test asserts that each slug has a page file on disk, and the comment on that test says why in one line:

it('has a public page on disk for every provider', () => {
  // Reached at /games/<provider> from the hub, landing page and sitemap - a
  // missing file is a 404 from three places at once.
  for (const provider of ALL_PROVIDERS) {
    const page = join(process.cwd(), 'app', 'games', provider, 'page.tsx');
    expect(existsSync(page), `missing app/games/${provider}/page.tsx`).toBe(true);
  }
});

Which is a good test, and it is guaranteed to fail every time we add a provider, because of how the work is split.

Games and pages are two different jobs

Building a provider is two unrelated kinds of work. One is the tests themselves: items, scoring, feedback, state. The other is the public surface: a hub page, a cheating guide, an article, rows in the disambiguation table, employer entries, structured data, metadata inside the SEO length limits.

Those run as separate steps, each in its own git worktree, each merged into an integration branch when it passes review. The last round merged four providers and 25 games in the first step. For a day, the site's data knew about four providers that had no pages at all.

The brief for the round says what that is allowed to look like:

Known failures until the public surface step, accepted: "has a public page on disk for every provider" and "covers every provider hub page" fail for each merged round 3 provider until its hub page and FORCE_DARK_PATHS entry exist. After every merge, nothing else may fail.

Two named tests, and a hard rule about everything else. That second sentence is the part that makes the first one safe. "The suite is red, we know, it is the provider thing" is how a real failure gets through; an allowlist of exactly two assertions keeps the other 7,000 meaningful. The run that closed the step reported 7,226 passing, one skipped by design, and those two.

The same reasoning kept the branch off main:

The integration branch does not go to main until the public surface step has landed. app/sitemap.ts, the landing providers section and the search dropdown derive from ALL_PROVIDERS, so a provider with no hub page would publish links to a 404.

Deriving your public surface from one list is what makes adding a provider cheap. It is also what makes shipping half of one expensive.

The list that is deliberately not derived

The second allowed failure is more interesting, because at first glance it should not exist. Every provider hub renders dark regardless of the visitor's theme, so why is there a hand maintained list?

Because it is matched exactly, and it is read by a blocking inline script before the first paint. /games/shl is forced dark; /games/shl-verbal is not. Deriving it from ALL_PROVIDERS would mean shipping the derivation into that script, and the list holds 21 non-provider routes as well, so a prefix rule would not do either.

The file's own comment records what happens without a test:

It drifted again when Thomas and TestGroup were added, so
__tests__/theme/force-dark-paths.test.ts now asserts that every provider in
ALL_PROVIDERS has an entry here. Adding a provider without one is a test
failure rather than a page someone has to notice is the wrong colour.

The first drift left Aon and Korn Ferry rendering light beside nine dark siblings. The second was Thomas and TestGroup, each added by a different provider worker that had no reason to know the file existed. That is the real argument for the test: the person adding a provider is not the person who wrote the theme script, and on a parallel round they are four different agents in four different worktrees.

Why the test checks the disk and not a URL

Here is the part I had not thought about until this round. A provider merged without its page does not produce a loud 404.

/games/<id> sits under a catch-all route that resolves a game by slug. Hand it something it does not know and it answers 200 with a page titled "Game Not Found". A crawler is free to index that, and a monitor watching for 404s sees nothing at all. The only reliable signal is whether the file exists in the repository, which is exactly what the assertion checks.

See it

• Fetch our sitemap and count. 375 URLs, of which 56 are /games/<provider> hubs, 57 are cheating guides and 133 are employer pages. Every one of those provider rows is generated by mapping over that array, which is why the test about files on disk is worth having.
• Open any page, view source, and find forceDarkPaths in the blocking script in the head. 77 entries, 56 of them provider hubs, inline in the HTML of every page because it has to run before the first paint. Comparing that count against the 56 hubs in the sitemap is the test, run by hand.
• Then open cogniprep.app/games/cfat, a slug we deliberately never built. 200, and a page that says it does not exist. That is what a merged provider with no hub page would have served to a crawler, and the reason the red test was worth accepting for a day rather than skipping.