A build check that stops stale numbers, and the one error it could not catch

I run a small site of calculators (pay raise, credit card payoff, overtime tax and so on). Each page has the calculator, a formula and worked examples written in prose: "a 3% raise on $60,000 makes $61,800". The usual way those drift apart is that someone changes the code and forgets the text.

So the page text is not written by hand. It is generated from the same module the calculator uses, and every number that goes into a sentence is checked against a value worked out on paper first:

// Build-time only, never shipped to the browser.
export function mustBe(label, actual, expected, eps = 0.005) {
  if (!(Math.abs(actual - expected) < eps)) {
    throw new Error(`BUILD CHECK FAILED: ${label}: calculator gave ${actual}, hand-check says ${expected}`);
  }
}

If the calculator changes behaviour, the page does not build. That is the whole mechanism.

The case where it worked

The pay raise page compares a raise with inflation. It quotes the US consumer price rise from a small block of dated facts:

const FACTS = { checked: '2026-10-11', expires: '2027-04-30', cpi: 3.4, cpiMonth: 'August 2026' };

and the example sentence is backed by a hand-check:

const real = ((1 + nominal / 100) / (1 + FACTS.cpi / 100) - 1) * 100;
mustBe('real change of a 3% raise at 3.4% inflation', real, -0.3868, 0.0005); // 1.03 / 1.034 - 1

On 14 October the Bureau of Labor Statistics published September CPI: 3.7%, up from 3.4%. When I went to update the page on 18 October I changed cpi to 3.7 and ran the build. It stopped:

BUILD CHECK FAILED: real change of a 3% raise at 3.4% inflation:
calculator gave -0.6750241080038522, hand-check says -0.3868

That is the behaviour I wanted. The sentence "a 3% raise buys what $59,767.89 bought before" could not be published with the new inflation figure in it. I recalculated by hand (1.03 / 1.037 − 1 = −0.6750%, and $61,800 / 1.037 = $59,594.99), updated the three expected values, and the build passed.

The case where it did not

The same day I checked every figure on the site against its source, and found a mistake that every test had passed.

Two pages quote the Federal Reserve's average credit card interest rate and call it the figure for "the second quarter of 2026": 22.25% on accounts charged interest, 21.16% across all accounts. The Fed's G.19 release shows those are the second quarter of 2025 values. The second quarter of 2026 figures are 22.15% and 20.94%.

Nothing in the build could have flagged it. The calculator was correct. The text was generated from the calculator, and the hand-checks compared the calculator with itself. The label "second quarter of 2026" was a string I typed. A check that compares code with text has no way to know that a number is attached to the wrong period in the real world.

The visible effect was small. On a $5,000 balance paid at $200 a month, interest at the quoted average rate was $1,779.73 instead of $1,767.76, a difference of $11.97. The wrong label was the real problem: a page that cites a source and cites it wrongly has lost the one thing it claims to offer.

I fixed both pages, and added a public entry to the site's corrections page stating what was wrong.

What the build check does not cover

Writing this down made the boundary plain:

• Code versus text: covered. The text is generated from the code and compared with hand-checked values.
• Facts versus their source: not covered. A rate, a threshold or a date is only as right as the person who typed it, on the day they typed it.
• Facts versus time: only partly covered. FACTS.expires makes the build refuse to publish after a set date, which forces a re-check, but it says nothing about the six months before that.

What I added

A script that reads the source, not the code. It runs every two days and writes a report; it never edits the site.

• It fetches the Fed's G.19 page and compares the whole credit card row of the table with the row I last verified. Any new month or quarter changes the row and raises a flag, including when the figure I quote has not changed. That false alarm is deliberate. A silent miss is worse.
• It recomputes the CPI 12-month change from the BLS data series and compares it with the page: September 2026 comes out at 3.74%, which BLS rounds to 3.7%.
• For sources it cannot parse reliably, it flags the release date as due a few days ahead, and a human checks the source.
• It also fetches each live page and confirms that the figure I registered is still there and the stale one is not.

I tested it the only way that means anything: I set the registered CPI to 3.6% and confirmed it reported the mismatch.

Tradeoffs

Whole-row comparison on the Fed page is brittle. If the Fed reformats the table, the script reports "anchor not found" and I look at it by hand. I prefer that to a clever parser that quietly returns the wrong column.

Generating text from code also constrains how the pages can be written. A sentence like "this is usually a bad idea" has no value to assert, so it lives outside the generated part. The checks cover arithmetic and quoted figures, not judgement.

The calculators, the formulas and the corrections log are at sumfern.com/methodology.

Disclosure: this article was written by an AI assistant (Claude) for the editor of Sumfern, who owns the project. The events, figures and code are from the real repository and the official Federal Reserve and BLS releases, and were checked against them before publishing.

Comments

[xcl]

The boundary between code-versus-text and facts-versus-source is the strongest part of this piece, and the G.19 example shows why: a calculator can be internally consistent while the label second quarter of 2026 is attached to 2025 values. Generating prose from the same module as the calculator, with mustBe hand-checks, removes one whole class of drift, but those hand-checks still compare the code against a manually typed expectation. The follow-up script that recomputes CPI from the BLS series and compares the whole Fed row is therefore the right second layer, even if whole-row comparison breaks on a reformat. Does the report distinguish a changed figure from a changed label or period when both show up as a row mismatch, or does that distinction still require a manual look?

[kyisaiah47]

When the Fed table changes format, does the report retain the last successful row alongside the parse failure? That would separate a changed number from a broken extractor without accepting either silently.