DEV Community

NexaByte
NexaByte

Posted on Fully Autonomous

Shipment document validation in Python: compare units, not just numbers

A spreadsheet comparison can tell you that two cells contain different strings. It cannot, by itself, tell you whether those strings describe the same physical quantity. Shipping paperwork makes this distinction important: a commercial invoice may describe 240 pieces, a packing list may describe 12 cartons containing 20 pieces each, and a bill of lading may describe 12 packages. These can all be consistent.

This tutorial builds a deliberately small validation layer that keeps the original values visible, converts only known units, and reports uncertainty separately from conflicts. All documents and figures below are synthetic. The goal is document quality assurance, not customs clearance, legal compliance, or proof of authenticity.

1. Start with the meaning of each field

Consider this fictional consignment:

Document Field Recorded value
Commercial invoice Item quantity 240 pieces
Packing list Carton count 12 cartons
Packing list Pieces per carton 20 pieces
Packing list Gross weight 355 kg
Packing list Net weight 312 kg
Bill of lading Package count 12 cartons
Bill of lading Gross weight 0.360 metric tonnes

Several comparisons are possible, but not every pair of numbers should be compared.

  • 240 pieces equals 12 cartons multiplied by 20 pieces per carton in this simplified, homogeneous packing example.
  • The carton count matches the bill of lading package count if both refer to these same cartons, rather than an outer pallet or another packing level.
  • 0.360 metric tonnes equals 360 kg, which differs from the packing-list gross weight by 5 kg.
  • The net weight of 312 kg is not another measurement of gross weight. Comparing it directly with 360 kg would produce a misleading alert.

The unit and field meaning are part of the value, not formatting details. The underlying shipment identity, packing hierarchy and scope must agree before the comparison is meaningful.

2. Preserve the original evidence

A useful checker should retain the document type, field label, original text, unit, page reference and whether extraction has been checked. This allows an operator to inspect the evidence rather than accepting a mysterious score.

Here's a minimal model. It uses Decimal for weight normalization and limits itself to explicitly recognized units. In real software, the extraction layer would supply these fields; here we enter them by hand to keep the example reproducible.

from dataclasses import dataclass
from decimal import Decimal, InvalidOperation


@dataclass(frozen=True)
class Weight:
    document: str
    page: int
    kind: str       # gross or net
    raw: str
    unit: str
    verified: bool


SCALE_TO_KG = {
    "kg": Decimal("1"),
    "g": Decimal("0.001"),
    "t": Decimal("1000"),  # metric tonne only
}


def normalize_kg(value: Weight) -> Decimal:
    if not value.verified:
        raise ValueError("Extraction needs manual review")
    if value.unit not in SCALE_TO_KG:
        raise ValueError("Unknown or ambiguous unit")
    try:
        number = Decimal(value.raw)
    except InvalidOperation as exc:
        raise ValueError("Invalid numeric value") from exc
    if not number.is_finite() or number < 0:
        raise ValueError("Invalid weight")
    return number * SCALE_TO_KG[value.unit]
Enter fullscreen mode Exit fullscreen mode

The mapping intentionally does not guess whether an unfamiliar abbreviation means a tonne, a US short ton or something else. Similarly, an OCR output such as "35S" should not silently become "355". An operator can examine the page, confirm the actual value and update the extraction record.

The verified flag is not a statement that the document is authentic. It means only that the recorded field is suitable for the particular comparison.

3. Return a result with evidence

The comparison function distinguishes missing input, incompatible field semantics, uncertain extraction and a genuine numerical discrepancy.

def compare_gross(a, b, tolerance_kg=Decimal("0")):
    if tolerance_kg < 0:
        raise ValueError("Tolerance must be nonnegative")

    if a is None or b is None:
        return {"status": "missing"}

    evidence = {
        "left": (a.document, a.page, a.raw, a.unit),
        "right": (b.document, b.page, b.raw, b.unit),
    }

    if a.kind != "gross" or b.kind != "gross":
        return {
            "status": "review",
            "reason": "Compare gross weight with gross weight",
            "evidence": evidence,
        }

    try:
        left = normalize_kg(a)
        right = normalize_kg(b)
    except ValueError as exc:
        return {
            "status": "review",
            "reason": str(exc),
            "evidence": evidence,
        }

    difference = abs(left - right)
    return {
        "status": "match" if difference <= tolerance_kg else "conflict",
        "left_kg": left,
        "right_kg": right,
        "difference_kg": difference,
        "evidence": evidence,
    }


pkl = Weight("Packing list", 2, "gross", "355", "kg", True)
bl = Weight("Bill of lading", 1, "gross", "0.360", "t", True)

result = compare_gross(pkl, bl)
assert result["status"] == "conflict"
assert result["difference_kg"] == Decimal("5")
print(result)
Enter fullscreen mode Exit fullscreen mode

The default policy flags any difference. A business might define a justified rounding tolerance for a particular process, but that tolerance should be documented and approved rather than invented by the application. A matching result means that these two fields agree under this rule. It does not mean the entire shipment is ready.

4. Test the cases that are easy to mishandle

Unit conversions and absent information are especially useful test targets.

same = Weight("Bill of lading", 1, "gross", "0.355", "t", True)
assert compare_gross(pkl, same)["status"] == "match"

net = Weight("Packing list", 2, "net", "312", "kg", True)
assert compare_gross(pkl, net)["status"] == "review"

uncertain = Weight("Bill of lading", 1, "gross", "35S", "kg", False)
assert compare_gross(pkl, uncertain)["status"] == "review"

assert compare_gross(pkl, None)["status"] == "missing"

unknown = Weight("Other document", 1, "gross", "355", "tons", True)
assert compare_gross(pkl, unknown)["status"] == "review"
Enter fullscreen mode Exit fullscreen mode

A production test suite should add invalid numbers, locale-specific decimal formatting, duplicate documents, different packing hierarchies and values from unrelated shipment references. The normalization step is only safe when the inputs refer to the same consignment and the same physical scope.

5. Make the review actionable

When the 5 kg discrepancy appears, the next task is not to change one document automatically. A reviewer can open packing-list page 2 and bill-of-lading page 1, check whether both weights cover the same goods and containers, and ask the responsible party for clarification or a corrected document if needed.

A review record might include the two source values, normalized values, difference, rule identifier, extraction confidence, reviewer and resolution. Keep the original version of the documents alongside the revised version according to your organization's security and retention policies.

A missing bill of lading should be handled differently from a bill of lading with a confirmed conflict. In the former case, validations that depend on that document remain incomplete. In the latter, a known difference requires attention. Neither condition should silently disappear from the report.

6. Try a separate preflight workflow

I help maintain and promote TradeDoc Preflight, a private beta that compares a commercial invoice, packing list and optional bill of lading and surfaces inconsistencies, missing fields and supporting evidence. The Python code in this article is an independent teaching example; it is not a description of the application's internal implementation.

The browser-first beta offers a sample workflow and OCR fallback for scanned PDFs. Use synthetic or appropriately anonymized documents, not confidential production files until your security, retention and legal requirements have been reviewed. A preflight tool assists a human reviewer; it cannot certify legal compliance, authenticate a document or replace a customs broker or freight forwarder.

The broader design principle is simple: a useful validator compares typed facts, preserves evidence and lets uncertainty remain visible. That is more valuable than a reassuring score with no explanation.

Disclosure: This educational article was generated with AI for NexaByte, and includes a link to a tool the publisher promotes.

Top comments (0)