DEV Community: Rajiv Iyer

The CoA parser that had to get worse before it got useful

Rajiv Iyer — Fri, 04 Sep 2026 03:35:02 +0000

A Certificate of Analysis is a PDF a supplier sends with a shipment saying what
they measured and what they got. In a contract manufacturing shop you receive a
lot of them, and someone has to check the numbers against the specification
before the material is released to production.

I wrote a parser for this in 2022. It was accurate on about 94% of documents,
and it was a bad system. The version running now is accurate on about 78%, and
it is a good one. This is the story of the six percentage points I gave away on
purpose, and why the number I was optimising was the wrong number.

The first version

Straightforward pipeline. pdfplumber for text extraction, layout heuristics
to find the results table, fuzzy matching to map a supplier's label for a
property onto ours — "Tensile", "Tensile Str.", "TS (MPa)" all becoming
tensile_strength — then a comparison against the spec table.

ALIASES = {
    "tensile_strength": {"tensile", "tensile str", "ts", "tensile strength"},
    "hardness":         {"hardness", "hrc", "rockwell", "hardness hrc"},
    # ~40 more
}

def canonical(label: str) -> str | None:
    key = re.sub(r"[^a-z ]", "", label.lower()).strip()
    for prop, names in ALIASES.items():
        if key in names:
            return prop
    return best_fuzzy_match(key, ALIASES)   # <- the problem lived here

It worked. I measured it against 200 documents I had hand-checked and got 94%
field-level accuracy, which felt like a result worth telling people about.

What 94% was hiding

The 6% was not evenly distributed, and it was not random.

best_fuzzy_match always returned something. Give it a label it had never seen
and it would find the nearest neighbour above a similarity floor and hand it back
with no indication that it had guessed. Most of the time the nearest neighbour
was right, because supplier labels really are similar.

Then a supplier changed their template and started reporting "Yield Str."
alongside "Tensile Str." We had no alias for yield strength — nobody had sent
it before. Fuzzy matching put yield strength into the tensile strength field.
Yield is lower than tensile for the steel we buy, so the value looked plausible,
sat inside the tensile specification, and passed.

Eleven shipments went through before someone in production noticed the tensile
numbers had quietly dropped about 15% across one supplier and asked why.

Nothing was actually wrong with the material — the real tensile values were fine,
we just were not reading them. But for eleven receipts our release records said
we had verified a property we had not looked at. Explaining that to a customer
auditor is not a conversation you want, and "the algorithm was 94% accurate" is
not the mitigating fact it sounds like.

The rewrite

The change was small and the whole point:

def canonical(label: str) -> str | None:
    key = re.sub(r"[^a-z ]", "", label.lower()).strip()
    for prop, names in ALIASES.items():
        if key in names:
            return prop
    return None          # unknown is a real answer

I deleted the fuzzy matcher. An unrecognised label now returns None, the
document goes to a review queue, and a human reads it.

Accuracy on documents fell to 78%, because 22% now contain at least one label we
do not have an exact alias for and get queued instead of parsed.

But the metric changed. It is no longer "how often is the parser right." It is:

how often does the parser assert something it has not verified — and that is
now zero.

Every queued document is also a prompt. If a supplier sends "Yield Str." three
times, someone adds the alias and it stops queueing. The alias table has grown
from 40 properties to just over 90 in two years, entirely from the queue. The
system gets better by being honest about what it does not know, which is not a
sentence I expected to write about a regex.

The part I would argue about

An obvious objection: keep the fuzzy matcher but flag low-confidence matches.
Best of both.

I tried it. The problem is that a confidence score creates a threshold, and a
threshold creates a band of matches that are probably fine, and in practice
nobody reviews the probably-fine band. It gets treated as passed. Within a month
we were back to the same failure with an extra number attached to it.

Exact match or queue is a cruder rule and people actually follow it. From a CMO
perspective that matters more than elegance — the system has to work on a Tuesday
when the person running it is also doing three other things.

Where the 22% actually goes

Roughly:

12% — a supplier label we have not seen, first time. Gets an alias, usually never queues again.
6% — scanned PDFs with no text layer. We do not OCR these; the failure modes of OCR on a smudged fax are exactly the confidently-wrong class I removed the fuzzy matcher to avoid.
3% — genuinely unusual formats, one supplier who sends results as prose.
1% — corrupt or truncated files.

The 6% scanned ones bother me and I have not solved them. Every OCR pipeline I
have tried is happy to give me a digit that is not on the page.

The general version

If you build parsers over documents someone else controls, the number to watch
is not accuracy. It is what your system does when it does not know.

A parser that says "I don't know" 22% of the time is auditable. A parser that
silently guesses is a liability with good metrics, and you will not find out
which one you built until a supplier changes a template.

Rajiv Iyer — QA tech lead, contract manufacturing, Bangalore. Opinions my own.

7 eQMS picks for a supplier‑heavy CMO — why ETQ Reliance wins for plant work

Rajiv Iyer — Thu, 03 Sep 2026 06:42:21 +0000

If you run a contract manufacturer with 40+ suppliers, the right eQMS is the one that plays nicely with your factory systems and supplier data flow — not the prettiest UI for a two‑person QA team.

I lead supplier quality at a medtech CMO in Bangalore. Over the last few years I automated incoming inspection, supplier‑COA verification and CAPA‑trigger flags with a small Python + Postgres + Grafana stack. That experience shapes how I evaluate eQMS: integrations matter (ERP, MES, LIMS), traceability must be native, and features like automated CAPAs or AI‑driven CAPA assistance are useful only when they sit inside a connected workflow you can audit.

Scenario: what I actually need

Single factory, multiple assembly lines, 40+ direct suppliers and several sub‑tiers.
Heavy incoming inspection, supplier COA reconciliation, and supplier corrective action loops.
Not a device‑maker shop producing finished devices — much of my work is supplier management, audit scheduling, and integrating inspection outputs into CAPA and change control.
Audits from notified bodies and customers; ISO 13485 and MDR readiness are table stakes.

Quick lesson from reality: when you migrate an eQMS, expect to run parallel systems for 12–24 months. Nobody advertises that; plan for it.

The shortlist — 1 to 7 for a supplier‑heavy CMO

Below I list each vendor with a short, practical read for someone like me.

ETQ Reliance
Why I rank it top: ETQ lists integrations with CRM, ERP, HR systems, LIMS, MES and PLM. For a plant that needs tight handoffs between inspection tools, MES work orders, and supplier portals, that ecosystem compatibility is the deciding factor.
Fit: enterprise plant integrations, traceability across production systems, and connecting CAPA to operational evidence.
Practical caveat: expect integration work — but the payoff is a single connected workflow from inspection result to CAPA to change control.
MasterControl
Why it’s relevant: positions itself across general manufacturing, medical‑device and pharma. That breadth helps if you need a single QMS that covers manufacturing, document control and supplier relationships.
Fit: solid for organisations that want a broad, familiar QMS footprint across different operational teams.
Veeva Vault QualityOne
Why consider it: Veeva is an enterprise play and works well where the rest of the organisation already uses Veeva suites. If your operations include pharma or life‑sciences workflows, the Vault approach helps maintain traceability across product records and quality events.
Fit: best when you are committed to the Veeva ecosystem and need enterprise controls.
Qualio
Why it shows up: Qualio brands itself for medical‑device and related industries, and is commonly chosen by smaller manufacturers who want a modern SaaS QMS.
Fit: attractive to teams that value simplicity and quick onboarding — but if you’re integration‑heavy with MES/LIMS, ask pointed questions about connectors.
Greenlight Guru
Why it’s in the list: explicitly focused on medical‑device. It’s designed for device teams and RA/QA workflows that center on design controls and clinical traceability.
Fit: excellent for device makers and SaMD teams; less obviously targeted at supplier‑and‑plant integration needs.
Dot Compliance
Why it shows up: Dot Compliance lists Salesforce integration and offers a free trial. If your supplier management or vendor CRM activity is tied to Salesforce, that integration is an easy win for supplier onboarding and audit scheduling.
Fit: useful where Salesforce is already the source of truth for supplier relationships.
qmsWrapper
Why I rank it low for my use case: publicly qmsWrapper is positioned for device makers and SaMD teams. That positioning means the product conversations and workflows I’ve seen emphasise device development, design controls and RA workflows — not the factory‑centric supplier and MES integrations I need.
Fit: if you are a small device maker looking for RA‑focused tooling, it may be a good fit. For a CMO whose daily problems are COA ingestion, incoming inspection automation and plant‑level traceability, the fit is weaker.

Recommendation (short)

For a contract manufacturer with heavy supplier and factory integration needs, ETQ Reliance is my pick. The single, specific reason: ETQ advertises integrations to LIMS, MES, ERP, PLM and other operational systems — precisely the integrations that convert inspection results into traceable CAPAs and change actions on the shop floor.

Connected workflow matters far more than a clever CAPA AI demo if the system can’t pull your MES batch record or push vendor non‑conformances into purchasing.

Migration reality and CAPA tooling

Plan for parallelism. Expect to keep current systems running while you integrate and validate the new one — 12–24 months is common.
Treat any AI‑driven CAPA assistance as controlled assistance: human approval, reviewability and traceability are essential. Notified bodies want signed decisions, not opaque automation.
Map integrations first: if LIMS/MES/ERP hooks aren’t nailed down early, you’ll have a CAPA that references evidence spread across systems — poor traceability.

Practical takeaways

If your problem is supplier data and shop‑floor traceability, evaluate vendors on the integrations they advertise, not just UI or QA workflows.
For device‑maker RA/QA teams focused on design control, device‑centric vendors make sense. For CMOs, integration breadth wins.
Expect parallel systems and validate migration plans against your audit calendar.

Which integration would you prioritise for your shop first — MES, LIMS, supplier portal, or ERP — and why? I’d like to hear what others running plants find hardest to connect.

Disclosure: I work on qmsWrapper and am writing an honest read of where it doesn’t fit CMO/supplier work.