Extracting Claim Numbers and Dates of Loss From an Insurance Claim Form

#ai #machinelearning #llm #tutorial

A first notice of loss carries four or five identifiers that are all eight to twelve characters long, all alphanumeric, all printed within two inches of each other, and none of which has a check digit. That is the entire difficulty of this document, and no amount of prompt tuning solves it.

Four numbers that look the same

Take a property loss notice on the ACORD form set — the standardised forms most North American carriers and agencies exchange, published by ACORD. Near the top of the page you will typically find the agency’s customer or producer code, the policy number, the effective and expiration dates, and a block reserved for the carrier’s own claim number, which is blank on the form as submitted and populated later. Further down there is an adjuster or file reference. Downstream of the carrier the same document may pick up a TPA file number and a litigation matter number.

A model asked for “the claim number” will return one of them with high confidence. Which one depends on which happens to sit nearest a label the model recognises, and on a form where the claim number box is empty it will happily return the policy number, because the policy number is the most number-like thing on the page. This is not a hallucination in the usual sense — the value is real and it is on the document. It is in the wrong field.

The fix is to make the schema demand all of them at once, with the label text that supported each one. Asking for four identifiers separately forces the model to allocate rather than to pick, and returning the supporting label gives you something to check against. A claim number extracted from a box labelled “Policy Number” is a rejection, not a low-confidence pass.

Why you cannot validate a claim number

Several document families in this wave hand you an arithmetic check: a container number carries an ISO 6346 check digit, an IBAN carries mod-97, a VIN carries a documented weight table. A claim number carries nothing. Formats are carrier-private, they change when a carrier migrates claims systems, and a carrier may run two formats simultaneously for legacy and current claims. There is no registry to validate against.

So the validation has to be relational rather than arithmetic, and three checks do most of the work:

Per-carrier format. Learn the pattern from claims you already hold for that carrier rather than writing a regex from one sample. Store it as a per-carrier rule with a match rate, so a carrier that changes format shows up as a rate collapse rather than as a silent flood of rejects.
Policy period containment. The date of loss should fall inside the policy’s effective and expiration dates. When it does not, either a date is wrong or the wrong policy was matched, and both are worth a human. See how those dates are printed before you write the comparison, because the boundary is not midnight.
Cross-document agreement. The same claim number appears on the acknowledgement letter, the adjuster’s report and the payment record. Agreement across two independently extracted documents is a stronger signal than any single per-field confidence score.

Date of loss is not one date

The date of loss field looks like the simplest thing on the form and it is the field most likely to be wrong, because the document frequently contains several dates that are all true.

A form may carry a date of loss, a time of loss, a date the insured discovered the loss, a date the insured reported it to the agency, and the date the notice itself was prepared. On a continuing loss — water damage found weeks after a pipe began leaking, a defect that manifested over a period — the notice may state a range, or a date qualified as approximate. Models flatten a range to its first value without saying so. If your schema has one dateOfLoss string, you have designed the flattening in.

Model it as a small object instead: a start date, an optional end date, a precision marker (exact, approximate, range, unknown), and the raw string as printed. The raw string is not redundant. It is the only thing that lets a reviewer settle an argument without reopening the PDF, and it is what you re-parse when you fix a date-format bug six months later.

Date of loss and date of claim are different fields with different consequences, and which one drives anything depends on the policy that responds, not on the notice. Extract both; do not derive one from the other.

Carrier layouts defeat position rules

The forms are standardised; the documents you receive are not. The same notice arrives as a filled PDF form with real field objects, as a flattened scan of a printout, as a fax at 200 dpi with a black border, as a carrier’s own branded reworking of the same field set, and as an email body a broker typed by hand containing the same six facts in prose.

Two consequences. First, if the PDF has an AcroForm field layer, read it before you send anything to a model — the field names in a filled form are more reliable than any visual reading of it, and the check is cheap. Second, for everything else, do not build rules on coordinates. A rule that says “the claim number is in the top right quadrant” survives one carrier. Anchor on label text and on the relationship between a label and its nearest value box, which is the same problem layout-aware document understanding exists to solve.

The fax case is worth calling out because it fails in a specific direction: at low resolution and after a generation or two of re-faxing, digit confusion is systematic rather than random. 3 and 8, 5 and 6, 0 and O, 1 and 7 in a hand-printed field. A claim number without a check digit gives you no way to catch it, which is the argument for the cross-document agreement check above rather than for a better OCR pass.

Build the extractor

The schema below is the shared core across carriers. Everything carrier-specific goes in a separate object so that adding a carrier does not change the contract downstream consumers read.

{
  "type": "object",
  "required": ["identifiers", "dateOfLoss", "policyholder"],
  "properties": {
    "identifiers": {
      "type": "object",
      "properties": {
        "claimNumber":   { "$ref": "#/$defs/labelledValue" },
        "policyNumber":  { "$ref": "#/$defs/labelledValue" },
        "adjusterFileNumber": { "$ref": "#/$defs/labelledValue" },
        "producerCode":  { "$ref": "#/$defs/labelledValue" }
      }
    },
    "dateOfLoss": {
      "type": "object",
      "required": ["start", "precision", "raw"],
      "properties": {
        "start":     { "type": "string", "format": "date" },
        "end":       { "type": ["string", "null"], "format": "date" },
        "timeOfDay": { "type": ["string", "null"] },
        "precision": { "enum": ["exact", "approximate", "range", "unknown"] },
        "raw":       { "type": "string" }
      }
    },
    "policyholder": {
      "type": "object",
      "properties": {
        "name":    { "type": "string" },
        "address": { "type": ["string", "null"] },
        "role":    { "enum": ["named_insured", "claimant", "third_party"] }
      }
    },
    "adjuster": {
      "type": ["object", "null"],
      "properties": {
        "name":  { "type": "string" },
        "email": { "type": ["string", "null"] },
        "phone": { "type": ["string", "null"] }
      }
    },
    "lossDescription": { "type": ["string", "null"] }
  },
  "$defs": {
    "labelledValue": {
      "type": ["object", "null"],
      "required": ["value", "label"],
      "properties": {
        "value": { "type": "string" },
        "label": { "type": "string" }
      }
    }
  }
}

Check for a form field layer first. If the PDF carries AcroForm fields with values, emit them and skip the model entirely for those fields. Keep the model path for the loss description, which is free text even on a filled form.
Send the page image and the extracted text together, and require the schema above with strict structured output so that a missing claim number comes back as null rather than as an invented string. Provider support for strict schemas differs; see which providers enforce the schema rather than merely encouraging it.
Reject any identifier whose label does not match the field it was assigned to. A value returned under claimNumber with a label of “Policy Number” is the single most common failure and it is caught by a string comparison.
Resolve the policy. Look the policy number up, then assert that dateOfLoss.start falls inside the policy period. Route failures to review with both the extraction and the policy record attached.
Re-run identifier extraction on the acknowledgement letter when it arrives and compare. Disagreement between two documents on the same claim number is the check that replaces the check digit this document does not have.

Claim intake is bursty and mixed: filled PDFs need nothing, faxes need a vision model, and some models will refuse a document that reads as a personal record and return a refusal rather than a schema violation. Running the queue through a gateway means the refusal path is a fallback to a second provider rather than a dead letter, and a spend cap on the vision route stops a backlog of re-faxed scans from quietly costing more than the claims they describe.