DEV Community

Daniel Pertu
Daniel Pertu

Posted on

58 component prefixes for 56 providers, and includes('tsa') is a bug waiting to happen

CogniPrep has 56 assessment providers and 369 practice games. Each provider owns a folder of data and scoring, and each playable game lives in a component folder named by the prefix of its game id, not by its provider. SHL's verbal test is shl-verbal in components/games/shl-verbal/. The FAA's air traffic battery is atsa-spatial, atsa-logic and five more.

The registry for that namespace is a markdown table in lib/games/providers/README.md. 58 rows for 56 providers, because the mapping is not one to one:

Prefix Provider
none arctic-shores
cog revelian (Cognify)
rev revelian (ability tests)
rr shl, except rr-scales-cls, which is aon
knx kenexa (game ids only)
tsa tsa (not atsa: test ids with startsWith('tsa-'), never includes('tsa'))

Revelian has two prefixes because its gamified battery and its conventional ability tests are different families of component. SHL has two because of a historical rename, and one id under the second prefix belongs to a different vendor entirely. Arctic Shores, the oldest provider on the site, has bare names and no prefix at all. Kenexa has game ids but no component folders: its tests reuse three other providers' components.

None of that is elegant. All of it is load bearing, which is why the table exists.

The table is there because branches cannot ask each other

Providers arrive in rounds. The last one researched six batteries in parallel, built four of them, and each build ran in its own git worktree off a shared integration branch. Nothing in a worker's prompt makes it aware of what the other three are doing.

That works because the aggregation points are append only. Every hub in lib/games merges one import and one line per provider:

import { games as tsaGames, meta as tsaMeta } from './providers/tsa/games';
Enter fullscreen mode Exit fullscreen mode

Two branches adding different providers do not touch the same line, so the merge is clean without anybody arbitrating. The prefix table is the one piece of shared state that a worker has to read before it picks a name, and the round brief says so explicitly: prefixes were checked against every existing prefix and every existing game id before the work started.

At least one provider's registration test asserts its own row is still in the table, which is a strange looking test until you realise the table is the only thing standing between two providers and the same folder name:

it('has a component prefix row in the providers README', () => {
  expect(read('lib', 'games', 'providers', 'README.md')).toContain('| `ukff` | `uk-firefighter` |');
});
Enter fullscreen mode Exit fullscreen mode

The collision that is not a collision

The new US provider in that round is the TSA's own Computer Based Test, so its prefix is tsa-. We already had atsa- for the FAA's Air Traffic Skills Assessment.

'atsa-spatial'.includes('tsa'); // true
'atsa-spatial'.startsWith('tsa-'); // false
Enter fullscreen mode Exit fullscreen mode

Prefixes are a namespace with no delimiter at the front, so substring matching silently joins them. The rule in the brief is a sentence long: never filter a game id with a bare includes('tsa'). The rule in the code is a test:

it('never confuses its games with FAA ATSA', () => {
  const ids = getGamesByProvider('tsa').map((g) => g.id);
  expect(ids.every((id) => id.startsWith('tsa-'))).toBe(true);
  expect(getGamesByProvider('faa-atsa').some((g) => g.id.startsWith('tsa-'))).toBe(false);
});
Enter fullscreen mode Exit fullscreen mode

The near misses are a list by now. tg is TestGroup and tgor is TestGorilla. crc is CritiCall and crt is Criteria. cs is the UK Civil Service and cpsc is the Public Service Commission of Canada. pel, pol and pod are PELLETB, the Dutch police and Podium.

The harder collision is in search

Three different things on the site are correctly called TSA.

The provider is the TSO Assessment Battery, which is what an applicant for a screening officer job sits. The FAA ATSA is an unrelated air traffic test that happens to carry the letters. And the TSA is also an employer, whose office and support staff sit a completely different federal battery, USA Hire.

A candidate searching the site for "tsa" wants exactly one of those three, and the token matches the FAA provider's description as well as the new provider's label. So the ranking is asserted rather than hoped for:

it('ranks above FAA ATSA when someone types tsa into provider search', () => {
  const tokens = tokenize('tsa');
  const tsa = scoreMatch(PROVIDER_META.tsa.label, `${PROVIDER_META.tsa.label} ${PROVIDER_META.tsa.description}`, tokens, 'tsa');
  const atsa = scoreMatch(PROVIDER_META['faa-atsa'].label, `${PROVIDER_META['faa-atsa'].label} ${PROVIDER_META['faa-atsa'].description}`, tokens, 'tsa');
  expect(tsa).toBeGreaterThan(atsa);
});
Enter fullscreen mode Exit fullscreen mode

What makes it pass is not a weight in the ranker. It is the display label, TSA CBT (Officer Test), chosen so it starts with the letters somebody is typing, against FAA ATSA (Air Traffic Controller Test) for the older one.

See it

Open which assessment am I taking and search the page for TSA. Five rows come back, pointing at three different destinations:

  • TSA CBT, TSA test and TAB go to the officer test.
  • The TSA X-ray and English sections go to the same place with a label saying applicants have not reported that version since early 2023.
  • USA Hire assessments for a TSA office job go somewhere else entirely, with the label "USA Hire, not the TSA CBT".
  • ATSA and AT-SA go to the FAA.
  • AT-SAT, the FAA's test before 2016, goes to the FAA page with "retired in November 2016" in the label.

Then open the TSA hub and the FAA ATSA hub side by side. Different tests, different employers, four letters apart, and one substring check away from each other's data.

A prefix namespace with no separator is fine until it is not. Ours is a table in a readme, a sentence in a build brief, and three assertions, and that is roughly the right amount of ceremony for a decision nobody will remember making.

Top comments (0)