An onboarding answer is a fact in a column until you join it to behaviour

CogniPrep asks two questions after signup: which company are you applying to, and which role. Almost every product asks something like this, and in most of them the answers go into two columns and are never read again, because on their own they are inert. "Applying to PwC" tells you nothing you can act on.

They become useful the moment you join them to what the user has actually done. Then the same two answers produce sentences:

• "PwC uses Arctic Shores, SHL and Aon. You have practised SHL. Two to go."
• "You unlocked Arctic Shores three weeks ago and have not played it."
• "You have been practising Watson Glaser, and the employer you named does not use it."

Those are three genuinely different things worth saying to someone: a gap to close, a purchase going to waste, and effort pointed at the wrong test. This post is about the module that derives them, and the three decisions that made it possible to write at all.

Decision one: store an id, not a string

The role answer is an id from a taxonomy, never free text:

/**
 * The answer is normally an id from this list, never free text, because a
 * stored taxonomy id can be aggregated ("38% of new users target finance")
 * and re-labelled later without a data migration. The one exception is the
 * ROLE_OTHER_ID escape hatch, which exists because no closed list of job
 * titles is complete and a forced wrong answer is worse than a typed one.
 */

Free text is a one way door. You can always render an id as a label; you can never reliably turn "IB analyst (summer)" back into a category. And the escape hatch matters as much as the list: a closed list with no exit makes people pick something false, which is worse than storing nothing.

The employer directory takes the same idea further by refusing to be a list at all:

/**
 * DERIVED, not hand-maintained. The app already ships two curated employer lists:
 *   - EMPLOYER_ASSESSMENTS - employers with a sourced assessment guide
 *   - PROVIDER_COMPANIES   - employers named on each provider's public page
 * Re-typing those names here would create a third list to keep in sync, and the
 * first employer added to a guide would silently be missing from onboarding.
 */

The ids are the same slugs the public employer guides use, which means a stored answer joins straight back to the guide and to the provider list without a mapping table.

See it: cogniprep.app/employers is the public face of that same data. Open any guide and the assessment it documents is the provider id the join uses. Some guides deliberately name no vendor at all, like Deloitte, where the page says the vendor is not published rather than guessing. An employer with no provider claim produces no gap signal downstream, which is the correct behaviour and falls out of the data rather than needing a special case.

Decision two: the derivation is pure and knows nothing about the database

/**
 * This module is deliberately PURE and db-free: it takes the answers and the two
 * provider lists as arguments. The queries that produce them live in
 * lib/onboarding/insights-repository.ts, which keeps this file testable without
 * a database and usable from a batch email job that has already loaded the rows.
 */
export function deriveApplicationInsights(input: ApplicationInsightsInput): ApplicationInsights

The second half of that sentence is the real reason. The main consumer is a batch email job that has already loaded thousands of users' sessions and entitlements in a handful of queries. If the derivation did its own fetching, the batch job would become N queries per user and nobody would run it. Taking the rows as arguments means the same function serves a single dashboard request and a bulk send without either of them being a special case.

The set arithmetic is unremarkable, which is the point:

const practisedTargetProviders = targetProviders.filter((p) => played.has(p));
const gapProviders = targetProviders.filter((p) => !played.has(p) && !owned.has(p));
const ownedNotPractisedProviders = canonical(owned).filter((p) => !played.has(p));
const offTargetProviders = canonical(played).filter((p) => !targetProviders.includes(p));

canonical() exists so every list comes out in one fixed order and free of duplicates:

function canonical(providers: Iterable<string>): AssessmentProvider[] {
  const wanted = new Set(providers);
  return ALL_PROVIDERS.filter((provider) => wanted.has(provider));
}

Filtering the master list by a set, rather than sorting the input, means the ordering is defined in exactly one place. Two users with the same providers get the same sentence in the same order, and a rendered email never depends on the order rows came back from Postgres.

One filter carries a product rule rather than a data rule:

/** Providers we actually have games for, so we never point at an empty shelf. */
const PLAYABLE_PROVIDERS: ReadonlySet<string> = new Set(GAME_LIBRARY.map((g) => g.provider));

We know more employer-to-provider relationships than we have practice content for. Recommending practice for a provider we do not cover is an email the reader cannot act on, and worse, it is an email that makes the product look bigger than it is right up until they click.

Decision three: emit statements, not prose

The output is a discriminated union, and every variant carries ids and counts only:

export type InsightSignal =
  | { kind: 'target_provider_gap'; providers: AssessmentProvider[] }
  | { kind: 'owned_not_practised'; providers: AssessmentProvider[] }
  | { kind: 'off_target_practice'; providers: AssessmentProvider[]; insteadProviders: AssessmentProvider[] }
  | { kind: 'target_fully_practised'; providers: AssessmentProvider[] }
  | { kind: 'employer_unknown'; typedByUser: boolean };

The reasoning lives in this module; the words live in the email templates that switch on kind. That division is what lets marketing rewrite a subject line without anyone touching the logic that decides who receives it, and it is why the only free text anywhere in the payload is the employer name the user typed themselves.

The ordering of the array is a product decision written as code:

// Ordered by what we would actually lead with. A gap in the tests they are about
// to sit beats an unused purchase, which beats a general observation, and the
// "we have no data for this employer" case is last because it is about us.

A consumer that wants one thing to say takes signals[0] and is right. That is worth more than any amount of scoring machinery, because there are five kinds and the priority between them genuinely does not change per user.

Two of the guards are the interesting ones. The off-target signal only fires when there is effort to be off-target with:

if (targetProviders.length > 0 && offTargetProviders.length > 0) {

Without the first clause, a brand new user who has played one game gets told they are practising the wrong thing, when what is actually true is that we do not know what their employer uses. Same output, completely different meaning, and only one of them is honest.

And the last signal exists so the absence of data is itself a case, rather than a branch that silently emits nothing:

if (targetProviders.length === 0) {
  signals.push({ kind: 'employer_unknown', typedByUser: input.employerId === 'other' });
}

typedByUser distinguishes "they picked an employer we have no provider data for" from "they typed a name we have never heard of". Those need different follow-ups, and the difference is one boolean rather than a second code path.

The coverage number

targetCoverage: targetProviders.length > 0
  ? practisedTargetProviders.length / targetProviders.length
  : null,

Null rather than zero when we do not know the employer's providers. Zero means "you have practised none of them", null means "there is no denominator". Collapsing those two into one number is how a progress bar ends up telling someone they are 0% ready for an assessment nobody has established they are sitting.

If you are adding an onboarding step

Ask what statement the answer will let you make. If you cannot write the sentence before you write the question, the column will be inert, and you will have spent one of the few moments of attention a new user gives you collecting it.

You can see the public half of this data at cogniprep.app/employers and cogniprep.app/games; the join happens after signup.