DEV Community: Preflight

A document QC tool — and how to check it

Preflight — Mon, 20 Jul 2026 21:40:00 +0000

A check that cannot be determined from the file says "not evaluable" — never a silent pass.

The problem

A check that cannot be determined from the file says "not evaluable" — never a silent pass. It looks fine right up until it doesn't — in a viewer, on screen, in the happy path — and the failure only shows at the point where it's expensive to fix.

What actually happens

This is a determinable property of the file: you can check it deterministically, without running the document and without guessing. Preflight does exactly that as part of its correctness-first pipeline.

The honest caveat

not_evaluable is a first-class verdict; it never counts as a pass. That distinction is the whole point of the tool — a check that can't be determined from the file says not evaluable, never a silent pass. A QC tool that passes everything hasn't checked your document; it has flattered it.

Try it

Preflight parses the print formats most tools can't open (PDF, AFP/MO:DCA, PCL 5, PostScript, IPDS) and runs deterministic checks like this one — free, in your browser, nothing uploads. → https://preflightdocs.com

A recipient who gets nothing: detecting a whole missing mail piece with order-independent set continuity

Preflight — Thu, 16 Jul 2026 21:22:51 +0000

One check I built catches page-level defects inside a mail piece: a statement that should be four pages
and printed three, a duplicated page, a page index out of range. It reads a printed keyline — the little
control line like 04 01 meaning "page 1 of a 4-page piece" — groups pages into pieces, and verifies each
piece is whole. Good for the inside of a piece.

It is completely blind to a worse defect: an entire piece that never exists. Account 100238 was in the
input file and produced no output — no pages, no keyline, nothing to group. The run looks internally
perfect. Every piece that printed is complete. And one household gets silence where a statement, an EOB, or
a legally-required notice was supposed to be.

Why you can't just sort and scan

The obvious approach — collect every piece's serial, sort, walk the list for gaps — breaks on real mail.
Production streams are presorted: reordered into carrier walk sequence for postal discounts. Pieces
arrive deliberately out of order, so "the next serial is smaller than the last" is the normal, correct
state, not an anomaly. Any check that assumes ordering drowns a presorted run in false positives.

So the check has to be order-independent. Treat the serials as a set and ask set questions: within the
observed range, is every expected serial present exactly once? That reframes the problem away from sequence
and toward continuity of a set — which is what actually matters. Nobody cares what order the pieces printed;
they care that everyone who should have gotten one, did.

The findings — and the honest one

From the set of per-piece serials, a few conditions fall out cleanly:

piece missing — a serial absent from inside the observed range
piece duplicate — the same serial on more than one piece
out of range — a serial outside the declared bounds
count mismatch — a total that doesn't match what was declared

Per client you choose whether serials must be strictly contiguous or merely unique.

The subtle one is the boundary. If the very first or very last piece of a run is the missing one, the gap
is invisible — there's nothing beyond it to reveal a hole. You cannot detect a missing edge from the data
alone; you need an external anchor: a declared expected range or count. So when a run's edge can't be
proven either way, the check doesn't quietly assume it's fine. It emits a coverage disclosure stating
exactly what it could and couldn't guarantee, and points at the declaration that would close the gap. Same
rule as everything else: the limit of what the data can prove is reported, never papered over.

Full write-up (with how it reuses the same keyline as the page check):
https://preflightdocs.com/blog/detecting-a-missing-mail-piece?utm_source=devto&utm_medium=article&utm_campaign=piece
The free analyzer: https://preflightdocs.com/?utm_source=devto&utm_medium=article&utm_campaign=piece

The third verdict: why a document QC tool should be allowed to say "I don't know"

Preflight — Wed, 15 Jul 2026 14:54:41 +0000

Most checkers answer one of two ways: pass or fail. I want to make the case that a QC tool with only
two answers is structurally dishonest, because it has to guess whenever it can't actually tell — and a
guess dressed as a green check is the most dangerous thing a validator can emit.

Here's the failure mode that shaped the engine I work on. Upload a scanned PDF with no text layer to an
accessibility checker. It looks for tagged headings, finds none it can read, and prints pass — not
because the document is accessible, but because there was nothing it could parse, and its vocabulary only
contained "pass" and "fail." Absence of a detected problem got reported as absence of a problem. Those
are different statements, and the gap between them is where compliance tools quietly lie.

Three states, not two

So every check returns one of three things: pass, fail, or not evaluable. The third is a
first-class citizen, not an error path. It means: the tool looked, and it cannot honestly determine the
answer from what it was given. No text layer to assess. No template declared for the thing being checked.
A 2D barcode that would need image decoding that isn't shipped yet. A resource the parser reached but the
module doesn't cover.

The rule is absolute: a check that cannot run is never allowed to read as a pass. If we can't evaluate
it, we say so, in those words, and we say why.

Why it's harder than one function

Adding a third return value is easy in isolation. Keeping it honest across an entire pipeline is not,
because "not evaluable" is the state most likely to get flattened at a boundary. A module throws on a
malformed stream; if the surfacing layer coerces that into an empty findings array, the public API
cheerfully reports complete, zero findings — a false green built from a swallowed exception.

I have found and fixed that exact class of bug more than once: a not-evaluable or errored signal dropped
at the CLI, the webhook, the scorecard, or the JSON boundary — each one turning "we don't know" back into
"looks fine." The lesson: the verdict has to be carried, not recomputed. Coverage travels alongside
findings all the way out. The CLI renders it and exits non-zero. The webhook marks the result partial.
The in-app view labels how many checks actually ran vs were not evaluable vs errored. The counts have to
add up, or a test fails.

The cost, and why it's worth paying

A tool that says "not evaluable" a lot looks less impressive than one that prints green everywhere. That's
a real cost — it's a worse demo than a wall of check marks. But the user here is someone whose statements,
checks, or EOB mail is already going out the door. For them a false pass isn't a missing feature; it's the
incident. An honest unknown lets them go look. A dishonest pass tells them not to.

A validator that never fails anything is just a mirror. A validator that never admits it doesn't know is
worse: a mirror that lies about what it reflects.

I write about this and the rest of the engine here — this piece in full, with the pipeline detail:
https://preflightdocs.com/blog/the-not-evaluable-verdict?utm_source=devto&utm_medium=article&utm_campaign=noteval
And you can watch the third verdict happen on a real file in the free analyzer:
https://preflightdocs.com/?utm_source=devto&utm_medium=article&utm_campaign=noteval

Most teams can't open the file that prints their statements — parsing AFP in the browser

Preflight — Thu, 09 Jul 2026 20:14:48 +0000

AFP — Advanced Function Presentation — predates the web. It came out of IBM in the 1980s, moves to production printers over IPDS, and still composes a huge share of the world's bank statements, insurance documents, and healthcare EOBs. If you work anywhere near transactional print, AFP files flow through your systems daily.

And yet: ask a team to answer "is this AFP batch structurally sane?" and watch what happens. The tooling that can even open the format is enterprise software — licensed per seat, installed somewhere specific, owned by another department. The format's documentation (MO:DCA — Mixed Object Document Content Architecture) is public via the AFP Consortium, but public and approachable are different things. Most vendors in this space sell conversion — AFP in, PDF out. Almost nobody just tells you whether the file is well-formed.

The format is friendlier than its reputation

An AFP file is a flat sequence of structured fields. Every field starts with a 0x5A carriage-control byte, then a two-byte big-endian length, a three-byte identifier (SFID), a flag byte, two reserved bytes, and payload. That's the whole framing story. No cross-references to resolve, no compression dictionary to bootstrap — you can walk an AFP file with a while loop.

Structure comes from Begin/End pairs. BDT…EDT wraps the document; BPG…EPG wraps each page; groups, presentation-text objects, and overlays nest inside in last-opened-first-closed order. Names inside the fields are EBCDIC — code page 500, another small museum piece — and resource references (fonts via MCF, overlays via IPO, page segments via IPS) are eight-character EBCDIC names pointing at objects that live outside the file, in a resource library on the production system.

What a structural validator can honestly check

Our free validator does four things, all derivable from the bytes alone:

Framing — every field opens with 0x5A, declares an in-bounds length, and the stream ends exactly at a field boundary. Truncation is the classic transfer failure; FTP in ASCII mode has ruined more AFP files than any parser bug.
Begin/End nesting — a plain stack.
The document envelope — exactly one BDT…EDT, pages nested inside.
Inventory — every structured field with its name, byte offset, and length, plus the page count and every referenced resource name.

Just as important is what it refuses to do. An SFID we don't recognize is reported as unknown — counted, located, flagged as a warning — not guessed at. Whether a referenced font or overlay actually exists in your resource library is not knowable from the file, so we say exactly that instead of implying it was verified. (My rule for the whole product: anything the tool can't actually determine is reported "not evaluable" — never silently passed.)

Why client-side was the whole point

AFP files are statements and EOBs — they carry names, addresses, account numbers, health information. The reason "just upload it to a converter site" never became the norm in this industry is that nobody responsible for that data should be uploading it anywhere casually. So the validator runs entirely in the browser: the parser is compiled into the page, the file is read locally, and there is no upload endpoint behind it. Parsing a structured-field stream is cheap — a multi-hundred-page statement run inventories in milliseconds on a laptop — and the file never touches a network.

Try it (free, no signup, there's a 20-second demo on the page): https://preflightdocs.com/tools/afp-validator?utm_source=devto&utm_medium=article&utm_campaign=afp

The MO:DCA reference is public via the AFP Consortium — if your team fights this format, the structural walk above is entirely buildable from spec, and honestly, more people should.

A structurally valid USPS barcode can still encode garbage — validating IMb from the bars

Preflight — Thu, 09 Jul 2026 20:14:42 +0000

Every piece of US commercial mail carries an Intelligent Mail barcode — 65 vertical bars, each in
one of four states (full, ascender, descender, tracker). Encoded inside: a barcode ID, service
type, mailer ID, serial number, and the delivery-point routing code. If it's wrong, mail gets
misrouted, automation discounts evaporate, and assessments follow.

Here's the part that surprised me when I built a validator: most "IMb checks" only verify that
the bars look like an IMb. Sixty-five bars, four valid states each, plausible geometry — pass.
But the encoding pipeline between your data and those bars is long, and a bug anywhere in it
produces a barcode that is structurally perfect and semantically garbage.

The USPS spec (USPS-B-3200) defines the pipeline roughly like this:

The tracking + routing fields pack into a single large integer via positional arithmetic (the routing code contributes up to ~10^11 of the space — this is effectively base conversion with mixed radixes).
An 11-bit frame check sequence (CRC) is computed over that integer's byte representation.
The integer is converted into ten 3-digit codewords (more base conversion).
Each codeword maps to a 13-bit constant-weight character — every valid character has exactly five bits set (weight 5) or two bits set (weight 2, selected by one FCS bit); the remaining FCS bits flip specific character bits.
The 130 character bits interleave onto the 65 bars (each bar's ascender/descender halves come from different characters).

So validation has levels:

Level 0 — geometry: 65 bars, valid states. Catches print defects, nothing else.
Level 1 — character validity: de-interleave the bars and check that every 13-bit character is in the constant-weight codebook. Catches some corruption; still passes many encoder bugs.
Level 2 — the FCS round-trip: reverse the whole pipeline — characters → codewords → the big integer (undoing the base conversion) — recompute the 11-bit CRC from the recovered payload, and compare it against the FCS bits actually embedded in the characters. A mismatch proves the barcode was produced by a broken encoder, even though every bar and every character looks valid.

Level 2 is the one that catches real production incidents: an off-by-one in codeword packing, a
routing code that got truncated upstream, a library that silently handles 130-bit arithmetic
wrong. All of those produce beautiful, scannable, wrong barcodes.

A few implementation notes for anyone building this:

You need exact big-integer arithmetic (the payload exceeds 64 bits — 10^11 × the tracking fields). Any float contamination anywhere corrupts the round-trip.
The reverse base conversion is the fiddly part: the spec is written encode-forward, so the decode is "run the multiplications backwards as divisions and prove you get clean remainders."
Test against the spec's published example vectors first, then against barcodes from encoders you trust, then — the fun part — against deliberately corrupted ones. A validator that never fails anything is just a mirror. (My rule for the whole product: anything the tool can't actually determine gets reported "not evaluable" — never silently passed.)

I ship this as part of Preflight, a document-QC tool for print/mail — the free analyzer detects
IMbs and validates structure in the browser: https://preflightdocs.com/tools/imb-barcode-validator?utm_source=devto&utm_medium=article&utm_campaign=imb
The full FCS round-trip and field decode run in the USPS module. But the algorithm above is all
public spec (USPS-B-3200) — if you maintain mail software, you can build Level 2 yourself, and
you probably should.