Last week I wrote about validating applications when the only real test takes two weeks. Short version: CertiSetu files domicile, income, caste and EWS certificate applications to Uttar Pradesh's e-District system. A rejection comes back 7 to 20 working days later and rarely says which field caused it, so every check we can run has to run before the file goes out. When something does get rejected, a reviewer works out why, and the reason becomes a rule that runs on every application after it.
A comment on that post pointed at a hole in the loop. A rule that blocks an application stops that application from ever reaching the department, so the rule never gets tested against the department again. If the rule came from something temporary, like one tehsil's affidavit preference that month, it can go stale and keep blocking valid files with nothing ever telling you.
That's right, and it's a bigger gap than it first looks.
A blocking rule never hears back
Look at what each outcome can teach a rule. When a rule lets a file through, the department eventually accepts or rejects it, and that answer is evidence. When a rule blocks a file, the file is never submitted, so there is no answer.
The evidence is lopsided as a result. A rule that is too loose shows up in the rejection log. A rule that is too strict looks exactly like a rule doing its job. In our own records, every false positive reads as a mistake we caught.
There was a good post here this week about how a type guard can silently drift from the TypeScript type it's meant to describe. This is the same failure with a slower clock. The check still agrees with itself. It has stopped agreeing with reality, and nothing in the system is built to notice.
Rules don't all age at the same rate
The last post sorted checks into three groups, and they go stale very differently.
A check that stands on its own ("this date parses", "this file opens") doesn't really go stale.
A check across two documents mostly doesn't either. The department is not going to start accepting an Aadhaar and a ration card with two different spellings of the applicant's name. (If you're an applicant stuck on exactly that, there's a guide on fixing a name mismatch between Aadhaar and ration card.)
The third group is the risky one: rules we learned from a rejection and can't verify ourselves. One tehsil wanted a particular affidavit format. One office started asking for a more recent utility bill. Each of these came from a single rejection, in one place, at one time. They're the rules most likely to be wrong six months later, and they're also the ones we're most tempted to make blocking, because the rejection that produced them was painful.
So the first change is small: a rule has to say which kind it is.
Every rule records where it came from
{
id: "affidavit-format-tehsil",
kind: "learned", // "intrinsic" | "relational" | "learned"
source: {
rejectionId: "rej_...",
observedOn: "2026-08-14",
scope: { district: "...", tehsil: "..." },
},
reviewBy: "2026-11-14",
mode: "warn", // "warn" | "block"
applies: app => /* ... */,
check: app => /* ... */,
evidence: ["affidavit"],
message: "This tehsil rejected this affidavit format recently.",
}
Three fields matter here.
source ties the rule to the rejection that created it. Anyone reading the rule later can see why it exists and judge whether that reason still holds. This was the other half of the comment's suggestion, and it's cheap.
scope keeps a rule learned in one tehsil from applying to the whole state. Without it, one office's habit turns into a statewide requirement the moment someone writes the check.
reviewBy gives learned rules an expiry. When the date passes, the rule drops back to a warning until a reviewer looks at it again. Intrinsic and relational rules don't get one.
Start as a warning, earn the block
A learned rule starts out as a warning. The reviewer sees it, decides, and can file anyway. It becomes blocking only after a second, separate rejection points to the same cause.
This has a cost. Warnings are easier to wave through, and a reviewer who overrides a correct warning sends someone's file into a two-week wait for nothing. But the two kinds of mistake aren't the same size. A wrong warning costs a reviewer a few seconds. A wrong block can cost an applicant a certificate they were entitled to, and we would never find out it happened.
An override is the only test a rule gets
The comment's second suggestion was to let a reviewer override a rule and file anyway now and then, since that's the only way a check can learn it was wrong. I had been treating overrides as noise to minimise. They're the opposite: the only experiments the system runs on its own rules.
So each override gets logged against the rule it overrode, and the department's answer is attached to it when the file comes back. Over time a learned rule collects a small history:
- overridden and then accepted, which is evidence the rule is wrong, at least in that scope
- overridden and then rejected for the same reason, which is evidence it's right
- overridden and then rejected for something unrelated, which tells us nothing about this rule
If overrides of a rule start coming back accepted in the same scope, the rule goes back to review. The numbers are far too small for statistics. The point is only that a rule now has some way to lose an argument.
The same trap applies to document reading
Most of the tooling attention right now is on AI that reads documents, so it's worth saying this applies there too. In the first post I argued that anything reading a phone photo of a certificate should treat "I could not tell" as a normal result. Staleness is the same problem one level up. A model that extracts fields confidently, checked only against rules that were themselves learned from its own output, has no route to finding out it's wrong. Whatever does the checking needs a path back to an outside answer, even a slow one.
What a rule is now
A rule used to be a function. Now it's a function plus a claim: this was true, in this place, as of this date, because of this rejection. Claims go out of date, and the system should expect that instead of being surprised by it.
If you work against a slow or silent source of truth (a regulator, a bank's batch process, a partner API that only emails you errors), how do you retire rules that might no longer be true? I'd like to hear what's worked.
CertiSetu is an independent private platform, not a government portal. Certificates are issued and approved by the relevant government authorities, and anyone can apply directly on the UP e-District portal. For the applicant's side of this, see why applications get rejected and how to read a government notification. The form design side is on our Hashnode blog.
Top comments (0)