Our site explains assessments that other companies sell. That means most of our content is a claim about somebody else's product: how long their test runs, how many items it has, whether you can retake it. Every one of those claims can go stale without us touching a line of code, because the vendor ships a new version and says so on their own help centre.
Last weekend we read the vendors' own documents again and fixed what they contradicted. Thirteen commits, 160 distinct files, 1,525 insertions against 1,371 deletions, and not one line of behaviour changed. It was all prose.
Here is what the sweep actually looked like.
The one that spread the furthest
We said that an AssessFirst result stays valid for twelve months and that an early reset needs the recruiter to ask. The vendor's current documentation says something different: a recruiter can invite you to retake at any time, and results no longer expire after a year. One of their older help articles still says they do, which is presumably where ours came from.
One sentence, and the diff touched eleven files:
app/blogs/assessfirst-brain-swipe-drive-explained/page.tsx
app/cheating/assessfirst/page.tsx
app/employers/lcl/page.tsx
app/employers/sncf-reseau/page.tsx
app/games/assessfirst/page.tsx
components/games/af-swipe/AFSwipeGame.tsx
docs/providers/assessfirst.md
lib/cheating/registry.ts
lib/games/providers/assessfirst/insights.ts
__tests__/assessfirst/af-registration.test.ts
__tests__/assessfirst/af-scorers-feedback.test.ts
A hub page, two employer guides, a blog article, a strategy guide, the instruction screen a candidate reads before a practice run, the registry that feeds several of those, the research dossier it all came from, and two tests. Two test assertions moved with it. One of them used to check only that the copy mentioned a year, which is exactly the assertion that would have let the wrong sentence sit there forever:
- expect(text).toMatch(/year/);
+ expect(text).toMatch(/recruiter invites you to retake/);
The one we invented
This is the one I would rather not publish. On the Cappfinity strengths assessment we had written a confident, actionable mechanism:
Cappfinity records not only what you choose but how quickly and confidently you choose it. The reasoning is that fast, decisive answers to options fitting your genuine strengths signal authentic preference, whereas a slow answer suggests you are constructing a response rather than reporting one, so it can carry less weight.
The vendor's own candidate help says its behavioural questions are untimed, apart from video and written responses, and that where an assessment is time recorded you are told in advance. So our paragraph was not a stale figure. It was a theory, written with the cadence of a fact, that told readers to hurry when the vendor tells them not to overthink. The section heading went from "How you answer is part of the answer" to "Check whether you are being timed", and the advice is now: read the instructions, and if a section says it is time recorded then work quickly, otherwise take the time you need.
A wrong number is embarrassing. A wrong mechanism changes what somebody does in a real assessment.
The one where our numbers described a product generation that had moved on
Our SHL figures were not invented. They were right about a previous generation. "Around eighteen questions, roughly seventeen to twenty-five minutes, roughly 75 seconds a question" described the older multiple-choice Verify tests; the current Interactive versions allow 18 minutes for up to ten questions and have you build charts and fill in spreadsheets. "Inductive Reasoning is 24 shape-sequence items in about 24 minutes" became "up to 15 items in 18 minutes on shape and symbol sequences, and the older multiple-choice version up to 18 in 24 minutes".
Two habits came out of that one. Name the generation, because both are in the field and a candidate's invitation decides which they get. And write "up to", because the tests are adaptive and a fixed count is a promise we are not in a position to make. The page now says so in plain words: "These are SHL's maximums. The tests adapt to your answers, so you may see fewer questions."
The one that went the other way
Not every correction removed a claim. One added one back.
For the LTP G-test our copy was carefully hedged: "LTP does not publish its time limit; preparation sites report 20 minutes, which is what our version uses." The hedge was wrong. LTP's own candidate portal does publish it, in Polish, and the figure is the same 20 minutes. The code comment that used to warn the next reader off now carries the quote instead:
// LTP's portal instruction: "50 zadan, na ktorych rozwiazanie masz 20
// minut" (see lib/games/test-pages/ltp.ts for the source).
The reverse case also turned up: on one Arctic Shores task we had "Rounds: reported as 10, 20 or 30". Those three numbers are a prep site's own practice settings, not a fact about the real task. They are now "Not published", which is less satisfying and true. Elsewhere a single word went: an unsourced "free" came off a card, because nobody had checked whether that one is free.
Why it was 160 files and not one constant
The obvious engineering reaction is that a figure restated in eleven places is a missing constant. We thought about it and left it alone.
The 160 files break down as 55 instruction components, 36 modules under lib/games, 19 files under app/games, 19 blog articles, 14 employer guides, 7 strategy guides, a research dossier and 4 test files, plus five singletons. None of those surfaces wants the same sentence. A hub needs one line of orientation, a per-test page needs the format in detail, an employer guide needs it in the context of that employer's process, an instruction screen needs the one number that matters in the next thirty seconds. A shared SHL_VERIFY_NUMERICAL_DURATION would be interpolated into eleven sentences whose framing is the thing that actually goes wrong.
What is missing is not a constant. It is the review date, which these pages now carry per entry, and the dossier that every line traces back to. The sweep is the cost of writing prose about somebody else's product, and it is a cost that recurs.
See it
Open the AssessFirst strategy guide and search the page for "twelve months". Nothing. Search for "recruiter invites you" and you get nine hits, which is the new rule restated in every place the page needs it.
Then open the SHL hub and search for "24 questions in 24 minutes": gone. "Verify Interactive" appears eleven times. On the Cappfinity tips article, "time recorded" appears five times and "how quickly and confidently" appears zero. On the LTP G-test page, "preparation sites report" is gone and the 20 minutes stayed.
The practice itself is at cogniprep.app. The thing I would take from this if you run a content site about other people's products: the vendors publish more than you think, they change it without telling you, and the copy you are proudest of is the copy most likely to be a theory.
Top comments (0)