Silent Data Loss Is Worse Than a Failed Export

This week I fixed a set of bugs in a mobile export pipeline where the scary part was not that the export could fail.

It was that it could succeed while quietly omitting data.

The pipeline builds an internal assessment model, validates it, serializes it into an RdSAP XML export, packages evidence, writes a checksum, and records an audit log. The deterministic exporter work was already in place. Same assessment in, same bytes out.

But deterministic output only helps if the model you serialize has not already lost meaning.

A code review turned up a few places where valid survey answers were being collapsed into “nothing to export”.

The dangerous null

The first bug was a token mapper:

fuelToken(value): string | null

A valid answer, Other, was returning null. That meant mandatory fields like main fuel and water heating fuel could disappear from the XML instead of being exported as the existing fallback token.

The fix was boring:

Other -> "other"

Boring is good. Boring means the field is present and downstream validation can do its job.

The second version of the same bug was draught proofing. Unknown was a valid answer in the survey UI, but there is no clean numeric RdSAP value for it. The old behavior silently omitted the value. The fixed behavior raises a non-blocking validation warning instead.

That distinction matters:

• missing because the assessor never answered: omit
• present but not representable in this export format: warn
• present and representable: export

Those are three different states. Treating them all as null is how data loss gets a polite API.

false is not “unanswered”

The more subtle bug was boolean handling.

The original helper effectively collapsed explicit false and “never answered” into the same output. So a recorded “no” for fields like cylinder present, immersion heater, or insulation could become indistinguishable from a question that was never asked.

That is a bad trade. In an assessment/export system, “no” is data.

The helper now returns a tri-state value:

boolean | null

And the serializer emits true/false for real answers, only omitting the element when the value is genuinely unanswered.

The test is tiny, which is exactly the point:

it("distinguishes an explicit false answer from never-answered", () => {
  const a = buildAssessment({
    inspection,
    responses: mkResponses({ ...validValues, epc_cylinder_present: false }),
    media: [],
  });

  expect(a.hotWater.cylinderPresent).toBe(false);
});

No giant test harness. Just one assertion that prevents the bug from coming back.

Do not guess from raw JSON

There was also a LiDAR floor-area parser that tried too hard.

If the JSON had a total, use it. Fine.

If total was missing, the old fallback scanned the raw JSON for a number. That could accidentally grab an individual room area and treat it as the property's total floor area.

That is worse than failing.

The fixed version returns null when total is missing. No heroic guessing. No “probably 20m² because I found 20 somewhere in the blob”.

expect(a.property.floorAreaM2).toBeNull();

Again: boring test, useful guarantee.

Audit failures before they escape

One more bug was outside serialization. getExporter() could throw before the main try/catch, which meant a format lookup failure skipped the audit-trail write that other failures recorded.

That is the kind of edge case that makes production debugging annoying. The export failed, but the failure path forgot to leave evidence.

The fix was just moving the lookup inside the guarded path so the audit log is written consistently.

The AI-assisted part

This is where AI-assisted development is actually useful for me.

Not “write the whole exporter and hope”. More like:

1. build the deterministic seam
2. have Claude review the boring mapping logic
3. turn every discovered ambiguity into a narrow test
4. keep the deliberate deferrals explicit

This pass added 6 focused tests. The export suite moved from 119 to 125 tests, with the targeted export tests passing locally: 3 suites, 23 tests.

There is still one known bigger issue left alone: validation is currently too RdSAP-specific for every export format. That needs per-format validation profiles. I did not jam it into this fix because it is a different change.

That is another useful rule when working with AI in real codebases: fix the silent data loss now, leave the larger design change visible, and do not let the model turn a bug fix into a rewrite.

The best export pipeline is not the cleverest one.

It is the one that refuses to quietly lie.