1.500 is not 1.5: what broke while re-reading 131 banks' rules every Monday

Banxta is a free check that tells a founder which business-account providers take a company like theirs. It covers 131 banks and EMIs (Wise, Revolut Business, Mercury, Qonto, Payoneer and so on), and every rule it shows is quoted word for word from the provider's own page, with the link and the date it was read.

A quote is a promise that those words are still on that page. So every Monday at 05:00 UTC a GitHub Actions job re-reads all of it. This is the run of 19 October:

What it re-reads	Verified	Changed	Could not read
Licences, on the regulators' registers	130	0	1
Policy quotes	735	9	6
Owner-residency rules	32	1	1
Product facts	753	11	9
Currency quotes	915	10	4
Fee lines	364	5	6

That's 1,174 of 1,186 source pages, one request per host every two seconds, four hosts at a time, inside an 80-minute job. Here's what broke on the way.

Three outcomes, and the date moves on one of them

Every check returns verified, changed or unverifiable. The third one exists so the job never has to guess. A bot wall, a timeout or a layout the parser doesn't know is unverifiable, never verified.

The date printed next to a quote moves only on a positive result, and only for the same fact:

// A fact is its quote plus its URL. Edit one character and old dates stop counting.
const cellFact = (c: Cell) => fingerprint(["cell", c.id, c.quote, c.source_url]);

function entryFor(prev: Entry | undefined, fact: string, outcome: Outcome, today: string) {
  const carried = prev?.fact === fact ? prev.checked_at : null;
  return { checked_at: outcome === "verified" ? today : carried, outcome, last_run: today, fact };
}

Those dates gate the product. A provider whose licence check is more than 30 days old drops out of every result, silently. So a second workflow runs daily and goes red once the newest merged run is more than 8 days old, which is one missed Monday.

The job never edits the knowledge base. It commits dates only and opens one issue with a "needs a human" list, and a person fixes a changed fact by hand. It used to open a pull request instead. The run of 5 October re-read everything, passed its tests, then failed at gh pr create because Actions weren't allowed to open pull requests in the repo, and nobody heard about it.

Bug 1: our own quotes failed first

The first run, on 23 September, verified 39 of 39 licences and only 42 of 81 policy quotes. Of the 40 quotes that failed (policy quotes and owner rules together), most were our fault. 16 were summaries or third-party wording stored where a provider's own words should be. 10 pointed at dead URLs. 7 had the right words with different punctuation or layout. Only 7 were plainly not on the page.

That run is why both sides now go through the same normaliser before any comparison:

export function normalise(s: string): string {
  return s
    .normalize("NFKC")
    .replace(/[\u2018\u2019\u201a\u201b\u2032\u2035\u02bc`\u00b4]/g, "'")
    .replace(/[\u201c\u201d\u201e\u201f\u2033\u2036\u00ab\u00bb]/g, '"')
    .replace(/[\u2010-\u2015\u2212\ufe58\ufe63\uff0d]/g, "-")
    .replace(/[\u200b-\u200d\u2060\ufeff\u00ad]/g, "")
    .replace(/\s+/g, " ")
    .trim();
}

A quote with an ellipsis is split into fragments that must appear in order. Case and spacing are forgiven, and the level that matched is recorded. If every word is there in the same order and only punctuation differs (a list the page joins with commas), it still fails, with a hint telling the reviewer to copy the page's own text into the quote. Hidden text such as a collapsed FAQ answer counts, because it's on the page.

Bug 2: 1.500 is not 1.5

FinXP's terms print an opening fee as € 1.500 00. Fio's price list says 5.000 Kč. The fee check read those as 1.5 and 5, so a fee line saying EUR 1,500 could never verify.

A lone 1.500 really is ambiguous, so the parser now offers both readings and an amount matches if either does:

const values: number[] = [];
if (/^\d{1,3}(?:\.\d{3}){2,}$/.test(raw)) values.push(Number(raw.replace(/\./g, ""))); // 1.500.000
else if (/^\d+,\d{1,2}$/.test(raw)) values.push(Number(raw.replace(",", ".")));       // 12,99
else values.push(Number(raw.replace(/[, ]/g, "")));                                    // 1,500 or 1.5
if (/^\d{1,3}\.\d{3}$/.test(raw)) values.push(Number(raw.replace(".", "")));          // 1.500 is also 1500

The change only adds readings, so nothing that verified before can stop verifying. The test that matters most is the negative one: €1.50 per transfer must still not match a line that says EUR 150. Every amount also has to sit next to its currency. A bare 99 on the page doesn't verify AED 99.

Bug 3: sites that let you read one page

Some sites let a browser session read one page, then answer 403 or a bot check on the next. The job used one shared Chromium profile, and that shared session was what got refused. On the seven providers we were adding that week, 37 of 48 pages read.

The fix is one retry in a fresh context, from a second browser launched only when needed:

const res = await load(await shared.newPage(), url);
if (res.status !== 403 && res.status !== 429 && !res.challenged) return res;

const fresh = await (await isolatedBrowser()).newContext({ locale: "en-GB", userAgent: USER_AGENT });
try {
  const again = await load(await fresh.newPage(), url);
  return again.status >= 200 && again.status < 300 && !again.challenged ? again : res;
} finally {
  await fresh.close();
}

After it, 48 of 48. The fresh context was used 13 times, all on two hosts. The unit tests use fake fetchers, so this path is proven by the live run, and the pull request says so.

The order of readers matters as much: plain HTTP first, then the public JSON API for a Zendesk help-centre article that answers 403, and Chromium last. Launching that Chromium caused its own early bug: two hosts asked for it at once, two browsers started, one was never closed, and the process wrote its files and never exited. The launch is now one shared promise.

Bug 4: a register that blocks CI

On 19 October Gibraltar's FSC register answered the GitHub runner with 403. The licence came back unverifiable, its date didn't move, and that provider was set to drop out of every result on 9 November. The daily check went red on every open pull request, which is how we noticed. A re-read the next day from a machine the register does answer verified it.

There's no clever fix here. unverifiable is the honest outcome, the red check makes a person look, and a person re-reads it.

Bug 5: "Authorised" was not enough

The FCA register is a Salesforce app, so only a browser renders it. For the first three weeks the adapter read the status line. Then we found a UK bank listed as "Authorised" while a requirement on the same page barred it from onboarding any new clients. The adapter now opens each requirement's "More information" panel and treats one that stops new clients or the account business as changed, whatever the status says. If that block doesn't render, the read is unverifiable. All 49 FCA reads of 18 October rendered it.

Bug 6: a 202 with nothing in it

One provider's site answered the runner with HTTP 202 and an empty body. The reader took it as a readable page, found none of 20 quotes, and marked every fact from that provider changed, which the public site reads as "don't show". A browser read the same pages word for word. The fix, still in review as this goes out: a 2xx with no text is unreadable, not changed.

Page hashes are noise, quotes are signal

The job also keeps a SHA-256 of each page's main text (<main>, else the first <article>) and a line diff when it changes. On 19 October, 106 pages had changed since the week before. Only 9 policy quotes had actually gone missing. Menus, cookie banners and marketing copy move every week. The hash tells a reviewer where to look, and the verbatim quote decides.

The rule all of this protects

None of it would matter without one product rule: a provider is excluded only with a quote as evidence, from its own document, above a floor.

export const strongEnoughToExclude = (e: { source_type: "primary" | "secondary"; confidence: number }) =>
  e.source_type === "primary" && e.confidence > 0.6;

Silence is never an exclusion and never a match. When a provider publishes nothing about an industry, the result says exactly that. The weekly job exists so that "quoted on 19 October" still means the words are on the page today.