DEV Community: CertiSetu

Your validation rules can't tell you when they go stale

CertiSetu — Tue, 29 Sep 2026 09:28:02 +0000

Last week I wrote about validating applications when the only real test takes two weeks. Short version: CertiSetu files domicile, income, caste and EWS certificate applications to Uttar Pradesh's e-District system. A rejection comes back 7 to 20 working days later and rarely says which field caused it, so every check we can run has to run before the file goes out. When something does get rejected, a reviewer works out why, and the reason becomes a rule that runs on every application after it.

A comment on that post pointed at a hole in the loop. A rule that blocks an application stops that application from ever reaching the department, so the rule never gets tested against the department again. If the rule came from something temporary, like one tehsil's affidavit preference that month, it can go stale and keep blocking valid files with nothing ever telling you.

That's right, and it's a bigger gap than it first looks.

A blocking rule never hears back

Look at what each outcome can teach a rule. When a rule lets a file through, the department eventually accepts or rejects it, and that answer is evidence. When a rule blocks a file, the file is never submitted, so there is no answer.

The evidence is lopsided as a result. A rule that is too loose shows up in the rejection log. A rule that is too strict looks exactly like a rule doing its job. In our own records, every false positive reads as a mistake we caught.

There was a good post here this week about how a type guard can silently drift from the TypeScript type it's meant to describe. This is the same failure with a slower clock. The check still agrees with itself. It has stopped agreeing with reality, and nothing in the system is built to notice.

Rules don't all age at the same rate

The last post sorted checks into three groups, and they go stale very differently.

A check that stands on its own ("this date parses", "this file opens") doesn't really go stale.

A check across two documents mostly doesn't either. The department is not going to start accepting an Aadhaar and a ration card with two different spellings of the applicant's name. (If you're an applicant stuck on exactly that, there's a guide on fixing a name mismatch between Aadhaar and ration card.)

The third group is the risky one: rules we learned from a rejection and can't verify ourselves. One tehsil wanted a particular affidavit format. One office started asking for a more recent utility bill. Each of these came from a single rejection, in one place, at one time. They're the rules most likely to be wrong six months later, and they're also the ones we're most tempted to make blocking, because the rejection that produced them was painful.

So the first change is small: a rule has to say which kind it is.

Every rule records where it came from

{
  id: "affidavit-format-tehsil",
  kind: "learned",          // "intrinsic" | "relational" | "learned"
  source: {
    rejectionId: "rej_...",
    observedOn: "2026-08-14",
    scope: { district: "...", tehsil: "..." },
  },
  reviewBy: "2026-11-14",
  mode: "warn",             // "warn" | "block"
  applies: app => /* ... */,
  check: app => /* ... */,
  evidence: ["affidavit"],
  message: "This tehsil rejected this affidavit format recently.",
}

Three fields matter here.

source ties the rule to the rejection that created it. Anyone reading the rule later can see why it exists and judge whether that reason still holds. This was the other half of the comment's suggestion, and it's cheap.

scope keeps a rule learned in one tehsil from applying to the whole state. Without it, one office's habit turns into a statewide requirement the moment someone writes the check.

reviewBy gives learned rules an expiry. When the date passes, the rule drops back to a warning until a reviewer looks at it again. Intrinsic and relational rules don't get one.

Start as a warning, earn the block

A learned rule starts out as a warning. The reviewer sees it, decides, and can file anyway. It becomes blocking only after a second, separate rejection points to the same cause.

This has a cost. Warnings are easier to wave through, and a reviewer who overrides a correct warning sends someone's file into a two-week wait for nothing. But the two kinds of mistake aren't the same size. A wrong warning costs a reviewer a few seconds. A wrong block can cost an applicant a certificate they were entitled to, and we would never find out it happened.

An override is the only test a rule gets

The comment's second suggestion was to let a reviewer override a rule and file anyway now and then, since that's the only way a check can learn it was wrong. I had been treating overrides as noise to minimise. They're the opposite: the only experiments the system runs on its own rules.

So each override gets logged against the rule it overrode, and the department's answer is attached to it when the file comes back. Over time a learned rule collects a small history:

overridden and then accepted, which is evidence the rule is wrong, at least in that scope
overridden and then rejected for the same reason, which is evidence it's right
overridden and then rejected for something unrelated, which tells us nothing about this rule

If overrides of a rule start coming back accepted in the same scope, the rule goes back to review. The numbers are far too small for statistics. The point is only that a rule now has some way to lose an argument.

The same trap applies to document reading

Most of the tooling attention right now is on AI that reads documents, so it's worth saying this applies there too. In the first post I argued that anything reading a phone photo of a certificate should treat "I could not tell" as a normal result. Staleness is the same problem one level up. A model that extracts fields confidently, checked only against rules that were themselves learned from its own output, has no route to finding out it's wrong. Whatever does the checking needs a path back to an outside answer, even a slow one.

What a rule is now

A rule used to be a function. Now it's a function plus a claim: this was true, in this place, as of this date, because of this rejection. Claims go out of date, and the system should expect that instead of being surprised by it.

If you work against a slow or silent source of truth (a regulator, a bank's batch process, a partner API that only emails you errors), how do you retire rules that might no longer be true? I'd like to hear what's worked.

CertiSetu is an independent private platform, not a government portal. Certificates are issued and approved by the relevant government authorities, and anyone can apply directly on the UP e-District portal. For the applicant's side of this, see why applications get rejected and how to read a government notification. The form design side is on our Hashnode blog.

When your validation feedback loop is two weeks long

CertiSetu — Wed, 23 Sep 2026 05:35:03 +0000

A card gets declined in under a second. A pincode service tells you the city does not match while the user is still looking at the field. You write a rule, send a bad value, read the error, fix the rule.

We do not have any of that.

CertiSetu files applications for four Uttar Pradesh government certificates: domicile, income, caste and EWS. A user verifies once through DigiLocker, uploads their documents, and the application goes to the state's e-District system. From a complete filing, the first three take 7 to 10 working days. EWS takes 15 to 20.

If the department rejects the file, that is when we find out. And the rejection usually does not say which field caused it.

So the only authoritative test we have takes two weeks and comes back with roughly one bit of information.

What that rules out

The normal loop is to ship a validator, watch it fail in production, then tighten it. That works because production answers quickly and specifically.

Here production answers slowly and vaguely, and every iteration costs a real person a fortnight and usually a deadline. You cannot learn by submitting.

Anything we are able to decide ourselves has to be decided before the file goes out, because afterwards we are blind.

Three kinds of check

Sorting the checks turned out to be more useful than writing any one of them.

Some things are true or false on their own. A date parses. A file opens. An income figure is a number and not a string with a rupee sign in it. These are cheap, they belong in the browser, and they are almost never why something gets rejected.

Some things are only true or false in relation to something else the user gave us. The name on the Aadhaar has to match the name on the ration card. The address on the electricity bill has to be the address typed into the form. An EWS application needs an income certificate from the right financial year, not just any income certificate. This is where the rejections actually come from.

And some things only the department can decide, like whether a particular tehsil is accepting a given affidavit format this month. We cannot check those at all. We can only write down what happened.

The middle group is the whole problem, and it does not fit the shape most validation libraries are built around.

Field validators are the wrong shape

A field validator takes one value and returns a verdict:

const validators = {
  annualIncome: v => Number.isInteger(v) && v > 0,
  applicantName: v => v.trim().length > 1,
};

The checks that matter need the whole application, and they need to name the two things that disagree, because a person is going to have to open both. Something closer to:

{
  id: "name-match-aadhaar-ration",
  applies: app => app.docs.aadhaar && app.docs.rationCard,
  check:   app => norm(app.docs.aadhaar.name) === norm(app.docs.rationCard.name),
  evidence: ["aadhaar", "rationCard"],
  message: "This name is spelled differently on these two documents.",
}

That evidence list is the part worth keeping. "Validation failed" tells whoever has to fix it nothing. "These two files disagree, here they are" is a job someone can finish in a minute.

A slow oracle needs a fast stand-in

Since we cannot ask the department anything, we put a person in front of it. Every application is reviewed by a verified agent against the department's current rules before it is filed.

That is a bottleneck. It is also the only fast feedback in the system. A reviewer catches a mismatched name in a minute, where the department catches it in ten working days and does not say what it was.

The reviewer is also where new rules come from. When something is rejected, the reason gets worked out once and then becomes a check that runs on every application after it. A rejection that does not turn into a rule costs two weeks and teaches nothing.

The inputs are photographs

This shaped the code more than I expected. People fill these forms on a phone, at night, with the document lying flat on a bed. The image is rotated. The flash has blown out one corner. A ward councillor's stamp is legible to a human and not to much else.

Anything that reads these automatically has to treat "I could not tell" as an ordinary outcome rather than an error. We would rather send an unclear document to a person than let a confident parser read the wrong year off a smudged stamp and drop the file into a two-week hole.

Where this leaves us

You cannot iterate against the system of record when the system of record answers in working days. So you iterate against your own record of what it rejected, and you spend the engineering effort on the step before submission instead of the step after it.

None of this is a hard distributed systems problem. It is one slow, vague, unmockable dependency, and nearly every design decision here falls out of that single fact.

CertiSetu is an independent private platform, not a government portal. Certificates are issued and approved by the relevant government authorities, and anyone can file directly on the UP e-District portal. Our guides go through the document requirements for each certificate, and there is more on the form design side of this at certisetu.hashnode.dev.