DEV Community: donerightlabs

An AI Assistant Handed Me 6 'Real' Invoice Samples. 5 Were Completely Made Up

donerightlabs — Thu, 10 Sep 2026 11:05:09 +0000

I was building a parser for Vietnamese electronic invoices — the XML files tax authorities require for every VAT invoice issued in Vietnam. The goal: read a file from any provider, whoever issued it, and normalize it into one clean JSON shape.

To sanity-check it against real-world variation, I asked another AI assistant to help me gather sample invoice XML files from a handful of different well-known Vietnamese e-invoice providers, plus one generic "tax authority standard" file — six in total.

I ran my parser against all six. It rejected five of them outright.

My first reaction was: great, my code is broken on day one, five different providers, five different failures. My second reaction, a few minutes later, was that this was actually the correct outcome — and figuring out why took me somewhere I wasn't expecting.

The catch

Before assuming my code was wrong, I did something boring: I read the README the other AI had written for the sample files. Buried in it:

"the data inside (company names, tax codes, amounts) is simulated test data, not actual issued invoices... sourced from an open-source repo."

Simulated. Not "close to real." Simulated by a developer who'd never seen an actual Vietnamese invoice, writing plausible-looking test fixtures for their own unrelated project on GitHub — repurposed by an AI assistant as "sample invoices" because they were shaped like the right kind of thing.

So instead of trusting my own tests, I went and checked the actual tag names against the legal source of truth.

Here's the thing about Vietnamese e-invoices: since 2022, every provider is legally required to use the exact same XML tag structure, defined by the tax authority. Not "similar." Identical. TTChung, NBan, NMua, MST, DChi — Vietnamese abbreviations, mandated by decree, because the tax authority's own systems have to parse every invoice regardless of which vendor issued it.

Five of my six "sample" files used tags like <SellerInfo>, <BuyerTaxCode>, <InvoiceNumber>. Clean, readable, very sensible-looking English tag names.

Tag names that, as far as I can tell, no real Vietnamese e-invoice has ever used. Nobody in Vietnam is legally allowed to invent their own — that's the entire point of the mandate. My parser hadn't failed. It had correctly identified five pieces of confident, internally-consistent fiction and said so. Only the sixth file used the real government-mandated structure, and that's the only one it accepted.

What I did instead

I could have made the parser "smarter" — taught it to also recognize the invented English tag names, so it would accept all six and I'd have a tidier-looking demo. I didn't. Teaching a parser to accept a format that doesn't exist in reality isn't robustness. It's just a more elaborate way of being wrong, with extra confidence — and the first real invoice that came in later would have had to fight through that noise instead of matching cleanly.

So I built strictly to the real legal spec, kept the one sample that actually matched it, and waited for real data.

Then real invoices started arriving

Over the next few days, the person I was building this for started digging up actual XML files from their own email and provider portals — a utility bill, a couple of online purchase receipts, a clinic invoice, an old 2021 logistics invoice, eventually a genuine business-to-business invoice.

Three bugs surfaced. None of them showed up on a single one of the AI-generated samples, because you can only find these by touching the real thing.

Bug 1 — the privacy leak. My parser had a "raw fallback" mode: alongside the clean normalized output, it also dumped every tag/value pair it found, so nobody would lose data to a field I hadn't mapped yet. I'd also built a privacy feature — if a buyer had no tax code (a strong signal they're a private individual, not a business), their name and address got redacted from the output. Redacted from the clean output, that is. Nobody told the raw fallback dump about that rule. The first real invoice I tested — a utility bill — leaked the buyer's actual name, home address, and email straight through the "debug" field I'd added for convenience. The privacy feature and the debug feature had never been told about each other.

Bug 2 — silently dropping the buyer's name. The real spec has two different tags for "who bought this": one for a company name, a different one for an individual's name. I only knew about the company-name tag — none of the fake samples had ever used the other one, because whoever wrote them didn't know it existed either. Real invoices from individual buyers started coming in using that second tag. My parser read them, found nothing under the tag it was looking for, and happily reported "fully processed, no issues." It wasn't lying exactly. It just didn't know what it didn't know.

Bug 3 — the one that actually worried me. A 2021 invoice came in using an older schema version than everything I'd tested. Same overall shape, but the invoice number and issue date — the two fields you'd use to identify or deduplicate an invoice — lived under different tag names entirely. My parser again reported total success while silently returning null for both.

That's the one that made me change how the whole system reports errors. It's not enough to check "did I find the big structural block." I had to start checking "did I actually get a value for the fields that matter," and say so explicitly when I didn't — instead of a green checkmark hiding a null.

The part I keep thinking about

The fake samples weren't malicious, and the AI that made them was upfront that they weren't real — I just could have easily not checked. They were also useless as an adversary. A genuinely broken test file breaks loudly. A plausible-but-fictional one passes quietly and teaches you nothing, because it was generated by something optimizing for "looks like a Vietnamese invoice" instead of "is one."

Real data doesn't have that problem. It doesn't care if it's plausible. It just is what it is, including two tag names I'd never heard of and a schema version I didn't know still existed in the wild.

I don't think the lesson here is "don't use AI-generated test data." I used AI for plenty of this build, including help gathering that first batch of samples. The lesson is narrower: when you're validating against an external, legally-defined standard you don't fully control, "generated a file that passes my own tests" and "matches reality" are two different claims, and only one of them is checkable — by going back to the source of truth instead of trusting the fixture.

A few questions I'd ask if I were reading this

Wait, so is the AI-generated test data "bad"? Not really — it was honestly labeled as simulated, and it was still useful as a starting shape. The mistake would've been mine, not the tool's, if I'd stopped checking there.

What does the parser do now when it hits a tag name it's never seen? It doesn't guess. It reports exactly what it found, flags which expected fields came back empty, and includes the raw data it read so a human can see what actually happened instead of a silent null.

Couldn't you just support every tag name variant you find? For invented, non-standard ones, no — that would mean trusting fiction as if it were law. For real variants tied to an older but still legally valid schema version, yes, and that's exactly what happened with bug 3.

Is this a Vietnam-specific problem? The specific tags are. The pattern — an AI-shaped test fixture that's plausible enough to pass your own tests without matching the actual external standard you're building against — isn't.

The parser is live on Apify Store as Vietnam E-Invoice XML Normalizer — callable directly by AI agents over MCP if you're building bookkeeping tools that need to ingest Vietnamese invoices.

Three Playwright/Apify bugs that took me way too long to find (and the fixes)

donerightlabs — Sun, 06 Sep 2026 10:04:46 +0000

Three Playwright/Apify bugs that took me way too long to find (and the fixes)

I recently shipped Vietnam Tender Intelligence — a keyword-based monitor for Vietnam's national e-procurement portal, built with Playwright + Camoufox (a hardened Firefox build for stealth automation). It works, and it's live: it turned out to be a much better debugging exercise than I expected. Three bugs in particular cost me way more time than they should have, so I'm writing them down for whoever hits the same wall next.

1. A silent version mismatch between Playwright and Camoufox

Right after wiring up the browser launch, every run failed with:

Found property "<root>.viewport.isMobile" - false which is not described in this scheme

Nothing in my code touched isMobile. Turned out Playwright 1.61 added that field to the internal Browser.setDefaultViewport protocol call, and Camoufox's Juggler layer (the protocol it uses to drive Firefox) didn't recognize it yet. It's a real upstream incompatibility, tracked in daijro/camoufox#653.

The counter-intuitive part: the official Apify base image is tagged apify-python-playwright-camoufox:3.14-1.61.0 — so the natural instinct is to match your playwright pin to 1.61.0. Don't. That's the broken version for this combo. Pinning playwright==1.60.0 fixed it immediately.

Lesson: when a base image tag looks like a version recommendation, verify it against the actual compatibility matrix of the libraries involved — it can just be a taxonomy label, not a peer-dependency promise.

2. `.click()` on a text input can hang forever; `.fill()` doesn't

This one took the longest to nail down because it was intermittent. Some runs would sail through in 40 seconds; others would sit at Locator.click() until a timeout fired, with no error, no popup, nothing actionable in the trace.

After enough repeated runs, I noticed the timeouts always stalled at the exact same call: clicking a text input immediately before typing into it — a completely standard "click to focus, then type" pattern. Switching to locator.fill(value) — which focuses the element as part of its own actionability checks, without dispatching a raw mouse click — made the hang disappear across dozens of subsequent runs.

I never found a fully satisfying root cause (my best guess: something in the click-dispatch path occasionally not resolving cleanly under a residential-proxy connection with irregular latency, which fill()'s internal path avoids). But the fix generalizes: if you don't need the literal mouse-click side effect, .fill() is strictly more robust for text inputs — one less network round-trip and no dependency on precise coordinate/z-index resolution to succeed.

3. Apify's automated Actor tests use your default input — and an empty dataset counts as a failure

Apify runs every published Actor daily with its default/prefilled input and expects: success, completion within 5 minutes, and a non-empty dataset. Miss any of those on 2 of the last 3 daily runs and the Actor gets silently flagged "Under maintenance" (hidden from Store search) — you find out by email.

Two non-obvious ways to trip this that aren't really "bugs" in the traditional sense:

If your default input is a narrow query that occasionally has zero real-world matches, a perfectly correct run with zero results still counts as a failure. Pick a default keyword/filter broad enough to (almost) always return something.
If you've added retry logic to make individual runs more resilient (good for real users, who don't mind an extra 2 minutes), make sure the worst case — including retries — still fits inside 5 minutes. A retry loop tuned for "eventually succeed" and a hard 5-minute CI-style SLA pull in different directions; you have to explicitly budget for both.

Also: pick a genuinely persistent store, not the default one

Small one, but it cost a day of "why isn't my session persisting" confusion: Actor.getValue() / Actor.setValue() operate on the current run's default key-value store, which is fresh every run. To actually persist state (a saved cookie jar, in my case) across separate Actor executions, open a named store instead — Actor.openKeyValueStore(name=...) in Python — and read/write through that. Obvious in hindsight; not obvious from the method names.

None of these are exotic. They're the kind of thing you find in 20 minutes with a good search — once you know the exact term to search for. Hopefully this saves the next person that 20 minutes, or the day I lost on the key-value store one.

If you're building something similar (Playwright + Camoufox + Apify), happy to compare notes in the comments. And if you're curious what it looks like running end to end, the Actor itself is here: Vietnam Tender Intelligence.

DEV Community: donerightlabs

An AI Assistant Handed Me 6 'Real' Invoice Samples. 5 Were Completely Made Up

The catch

What I did instead

Then real invoices started arriving

The part I keep thinking about

A few questions I'd ask if I were reading this

Three Playwright/Apify bugs that took me way too long to find (and the fixes)

Three Playwright/Apify bugs that took me way too long to find (and the fixes)

1. A silent version mismatch between Playwright and Camoufox

2. .click() on a text input can hang forever; .fill() doesn't

3. Apify's automated Actor tests use your default input — and an empty dataset counts as a failure

Also: pick a genuinely persistent store, not the default one

2. `.click()` on a text input can hang forever; `.fill()` doesn't