DEV Community

Javeed Shaik
Javeed Shaik

Posted on

The bug no unit test can catch

I maintain a set of finance calculators. Every formula has a unit test and the build fails if a
number drifts more than 0.5% from a verified value. Last week a test caught me computing an
interest figure by hand as ₹5,272 when the correct answer was ₹5,326. That is exactly what the
suite is for, and it worked.

This week I hit the other kind of bug. The kind where every test passes and the answer is still
wrong.

The setup

I was building a table of India's 7th Pay Commission pay matrix — the grid that sets basic pay for
central government employees. Nineteen levels, one entry-pay figure each.

I could have typed the numbers from memory or from a blog. Instead I did the responsible thing and
went to the primary source: the Commission's own report, Table 5, published by the government.

It lists Level 13 entry pay as ₹1,18,500.

That figure is wrong today. The correct number is ₹1,23,100.

What happened

The Commission recommended Level 13 at an "Index of Rationalisation" of 2.57, which produced
₹1,18,500. The government then accepted the recommendation with a change — the CCS (Revised Pay)
(Amendment) Rules, 2017 re-based Level 13 on an IOR of 2.67, giving ₹1,23,100, backdated to
1 January 2016. The wording in the amendment is that the earlier Level 13 became non-existent
ab-initio
.

So the primary source — the actual PDF, on the actual government domain — has been describing a pay
level that legally never existed, for the better part of a decade. Every other level in that table
is still correct. Only one row moved.

Why no test catches this

Think about what a unit test asserts:

assert.equal(payMatrix(13, 1), 118500)
Enter fullscreen mode Exit fullscreen mode

If I had written that test from the same source I read the number from, it passes forever. The test
and the code agree. They are both wrong, and they are wrong together, which is the worst
possible failure mode because it looks like verification.

This is not a testing problem. Tests protect the transformation — given these inputs, do I get the
right outputs. They cannot protect the inputs. If your constant is stale, your arithmetic being
flawless just means you compute the wrong answer very reliably.

And notice the trap in the reasoning that got me there. "I checked the official source" felt like
the careful move. It is normally the right instinct. Here it was the thing that would have produced
the error, because the official source is a snapshot of a recommendation, not a statement of current
law.

What actually helps

1. Treat statutory values as data, not code. Every function that depends on a notified figure
takes it as an argument with a documented default:

gratuity(500000, 30);                        // ₹20,00,000 ceiling — most employees
gratuity(500000, 30, { ceiling: 2500000 });  // ₹25,00,000 — Central Govt civil employees

incomeTaxNewRegime(1600000);                 // FY 2026-27 slabs by default
incomeTaxNewRegime(1600000, { slabs: [...] });
Enter fullscreen mode Exit fullscreen mode

Hard-coding a slab table means shipping a release every time a Finance Act moves. Making it an
argument means whoever notices can fix it that afternoon. The value is still a default, so the
common case stays simple — but it stops being a fact baked into your logic.

2. Date every constant, in the code. Not "updated recently". A date and a source:

// Entry cell (Index 1) of the 7th CPC Pay Matrix.
// Verified 2026-08-04 against Table 5, Report of the Seventh CPC, p.75.
// ONE deliberate divergence: that table shows Level 13 as 118500 (IOR 2.57).
// Superseded — CCS (Revised Pay) (Amendment) Rules 2017 re-based it to 123100.
// 123100 below is correct; do not "fix" it back to the figure in the report.
Enter fullscreen mode Exit fullscreen mode

That comment exists because I know what will otherwise happen. Someone — possibly me, in a year —
will check the primary source, see ₹1,18,500, assume the code is wrong, and "correct" it. The
comment is not documentation. It is a tripwire for a specific future mistake.

3. Separate "the maths is right" from "the inputs are current". These are different guarantees
with different expiry dates. A formula is right forever. A ceiling is right until a notification
changes it. Publishing them with the same confidence is how stale numbers get laundered into
trustworthy-looking output.

On the calculators, that means a page says "last reviewed" only when a human actually checked the
statutory figures against a cited source on that date — otherwise it says "last updated", which is
just a file timestamp and promises nothing. The distinction is small and it is the whole point.

The part that generalises

You probably do not care about Indian pay commissions. But every domain has these:

  • tax rates, contribution ceilings, statutory limits
  • currency and country codes that get reassigned
  • API rate limits and quota tiers your client hard-codes
  • timezone rules, which change by political decision several times a year
  • postal and phone formats, sanctions lists, holiday calendars

All of them share the shape: a value that is correct, then silently is not, and no amount of test
coverage tells you. Your CI is green. Your output is wrong.

The only defence is knowing which of your constants are facts about mathematics and which are facts
about the world — and treating the second kind as perishable.


The formulas are open source if useful:
indian-finance-formulas — MIT, zero
dependencies, 20 tests, every statutory value a parameter. Source on
GitHub. The calculators they came from
are at emicalcs.com.

If you have hit a stale-constant bug that your tests happily confirmed, I would like to hear it.

Top comments (0)