Three search rankers in one app, sharing one function and nothing else

CogniPrep has three places where you type into a box and get a ranked list back: a header search over providers and tests, an employer picker, and a job role picker. Three ranked lists over small in-memory datasets.

There are three separate ranking implementations, and that is on purpose. Here is the reasoning, because "you have the same thing in three places" is the kind of observation that is usually correct and here is not.

What is actually shared

One function:

export function normalise(input) {
  return input
    .normalize('NFKD')
    .replace(/[̀-ͯ]/g, '')   // strip accents
    .toLowerCase()
    .replace(/[^a-z0-9]+/g, ' ')       // punctuation becomes space
    .trim();
}

Two of the three modules had byte-identical copies of this before it was pulled out. That duplication was real and worth removing, because the rankers do have to agree on what a query is before they can disagree about how to rank it. Sharing the primitive and not the strategy is the whole idea.

Ranker one: every token must appear

The header search covers providers and test formats. It is a plain substring matcher with a hard requirement:

for (const token of tokens) {
  if (!haystack.includes(token)) return -1;   // all tokens or nothing
}

Requiring every token is what makes a second word narrow the list instead of widening it. People type a provider name followed by a test name and expect fewer results, not more. Scoring then favours the title over the description, and an exact title over a prefix over a mid-string hit.

No fuzziness at all. The index is small, static and made of names the user is reading off a job application email, so they are typing something they can see. Fuzzy matching here would only add wrong answers.

Ranker two: prefix and substring over a derived directory

The employer picker asks which company you are applying to. The directory behind it is not hand maintained: it merges two lists the app already ships, the employers with a sourced assessment guide and the employers named on each provider's page, and de-duplicates them.

That derivation matters more than the ranking does. A hand-typed third list means the next employer added to a guide is silently missing from onboarding, and nothing fails to tell you. Ids are the existing guide slugs, so a stored answer joins straight back to the guide and to the provider list with no mapping table.

Ranking is prefix then substring, no fuzziness. Company names are proper nouns that people spell correctly, and a fuzzy match on a company name produces confidently wrong suggestions.

Ranker three: the one that needs to be forgiving

The role picker is different in kind, because there is no canonical spelling of a job in a user's head. They type what they call their job, and there is no list in front of them.

So this one is tiered, and the tiers are ordered so a clearly expressed intent always beats a coincidence:

exact term                         1000
term starts with the query          800
a WORD in the term starts with it   700 minus 15 per word of depth
term contains the query             600
all tokens present, not contiguous  500   "bank invest" → investment banking
subsequence match, ratio >= 0.55    200 + ratio

Every literal match beats every fuzzy match. That single rule is what stops a fuzzy matcher feeling random.

Two of those numbers are scar tissue.

The 15 per word depth penalty exists because "technology" scored identically on "Technology Consulting" and on the "garment technology" alias of a fashion role. Matching the first word of a label is a much stronger signal than matching its fourth, and without that penalty incidental late-word matches outranked the sector the user obviously meant.

The 0.55 subsequence ratio is the line between helpful and unhinged. The subsequence test also carries a maximum gap of 6 characters between matched letters:

const MAX_GAP = 6;
// ...
if (found - textIndex > MAX_GAP) return 0;

Without a gap bound, subsequence matching makes almost everything match almost everything: "sea" happily matches "supply chain analyst" through three letters scattered across twenty. With the bound, "invstment" still finds investment banking and "civeng" still finds civil engineering.

There is one more layer that no amount of matcher tuning would have produced: a synonym map from everyday job titles to terms already in the taxonomy. "Train driver" is not a substring, prefix or plausible subsequence of anything in a graduate role taxonomy, and neither is "air hostess" or "firefighter". Each key expands into terms that already exist and is then scored through the same matcher, so there is no per-role keyword sprawl and one obvious place to add the next reported miss. In a flow with no skip button, a dead-end search is not a search quality problem, it is a stuck user.

Why not one ranker with options

The honest test is to imagine the unified version. It takes a flag for whether fuzziness is allowed, a flag for whether all tokens are required, a weight table, an optional synonym map and an optional position penalty. Every call site passes a different combination, so every change to the shared code has to be reasoned about against three sets of expectations.

At that point the "shared" function is three functions in a trench coat, plus the risk that tuning role search quietly changes what the header search does. These three rank different things for different users with different information in front of them. They are not a duplication, they are a divergence, and divergence is cheaper to maintain in separate files.

See it

The employer directory is public: cogniprep.app/employers lists the guides, and they carry the same slugs the picker stores as ids. cogniprep.app/games is the provider side of the header index.

To try the forgiving one, sign up free and type into the role question in onboarding. Running those queries through the ranker right now gives:

invstment    Asset & Investment Management (300), Investment Banking (300)
civeng       Civil & Structural Engineering (276)
bank invest  Investment Banking (510)
air hostess  Aviation & Airlines (1007)
train driver Rail & Networks (1007)
firefighter  Police & Emergency Services (1006)
sea          Recruitment & Talent Acquisition (690), Academic Research & PhD (610)

The scores tell you which tier each one landed in. The typo and the abbreviation come back in the 200s, which is the subsequence tier. bank invest at 510 is the all-tokens-present tier. The synonyms score above 1000 because an expansion resolves to an exact term. And sea returns literal matches through the word "research" rather than a scattered subsequence through "supply chain analyst", which is the gap bound doing its job.

The takeaway

Before merging two similar implementations, check whether they are similar in mechanics or in purpose. Ours were similar in mechanics only. What they genuinely had in common was one 8 line normaliser, and that is exactly what got shared.