Give Your Document OCR Pipeline a Memory

After OCR, most pipelines throw the layout away and re-discover the same form from scratch every time. textract-field-memory is a small, zero-dependency Python layer that remembers where fields sit, checks they landed in the right place, tells you which layout a document is, and warns you when a form quietly changes. It doesn't do OCR. It makes the OCR you already run a lot more useful.

Where this started

I kept running into the same annoying gap on document pipelines. You send a form through Textract (or Tesseract, or whatever), you get back field names and bounding boxes, you pull the values, and then you throw all the positional information away. The next copy of that exact same form shows up an hour later and the pipeline treats it like it's never seen anything like it before.

That bugged me for a few reasons.

Nothing checks that a field came from the right spot on the page. If "Total" gets read out of the shipping box instead of the amount box, the value still looks like a number, so it sails downstream and nobody blinks.

Nothing notices when a form changes. A vendor nudges their invoice template, fields move a centimeter, and you find out three weeks later when a downstream job starts choking on garbage.

And there's no cheap way to ask "have I seen this layout before?" So every document, familiar or not, takes the same expensive road.

So I wrote a little library that sits after OCR and keeps the memory the pipeline was throwing out.

What it is, and what it definitely isn't

It's a spatial memory layer. You hand it field names plus normalized bounding boxes (x, y, width, height between 0 and 1). It learns roughly where each field tends to appear on a given template, and from then on it can validate positions, identify which template a document belongs to, and watch for drift.

It is not an OCR engine and it never touches pixels. Something upstream has to extract the fields first. This library just remembers where they ended up and reasons about that.

Under the hood there's nothing exotic going on, which I think is a feature. Spatial scoring is intersection-over-union plus the distance between box centers. Name matching is a normalized Levenshtein distance. Template identification is Jaccard overlap of the field-name sets combined with the spatial score. No model, no training step, no network calls. It's pure standard-library Python with zero dependencies, and templates are just JSON files on disk.

A quick look at the API

from field_memory import TemplateMemory

memory = TemplateMemory(store_path="./my_templates")

# 1. Learn — feed documents after OCR extraction
memory.record(document, template_id="employment-form")

# 2. Locate — spatial field lookup with a confidence score
matches = memory.locate(new_doc, "Employee Name")
# → FieldMatch(combined_score=0.95, spatial_score=0.93, within_expected_region=True)

# 3. Identify — which known layout is this?
match = memory.identify_template(new_doc)
# → TemplateMatch(template_id="employment-form", similarity_score=0.87)

# 4. Detect drift — has the layout shifted?
drift = memory.detect_drift(new_doc, "employment-form")
# → DriftReport(is_drifting=False, overall_drift_score=0.02)

Clustering comes for free. If you call record() and leave out the template_id, it either merges the document into a template it recognizes or spins up a new one. Run that for a while and you end up with a map of every distinct layout flowing through your pipeline without ever configuring one by hand.

Why there's no LLM inside

For a layout you've already seen, answering "where's this field?" is basically a dictionary lookup. Sub-millisecond, and free. The same question routed through an LLM is half a second to two seconds and costs you per field. Paying that toll on forms you've processed a thousand times is silly.

So the design leans into being complementary instead of competitive:

def smart_extract(document, field_name):
    matches = memory.locate(document, field_name)
    if matches and matches[0].within_expected_region and matches[0].combined_score >= 0.85:
        return matches[0].key_value, "spatial"   # free, instant
    return call_llm(document, field_name), "llm"  # pay only when needed

Think of it as the cheap first pass that decides whether you even need the expensive one. The 0.85 threshold is just a starting point. Crank it up if a wrong answer is costly and you'd rather fall back to the LLM more often.

Where it actually helps

Domain	What it remembers	What it catches
Invoice processing	Where fields sit per vendor layout	Vendor template changes, wrong-position reads
Insurance claims	Field positions across many form types	Unknown forms that need routing for review
Healthcare intake	Where fields land on clinic forms	Layout revisions before they break downstream
Manufacturing QA	Component positions on boards	Parts that shifted from baseline

The only real requirement is that your data has named things with bounding boxes. That's it.

Some numbers, and the caveats that go with them

I put it through a stress test using real Textract (FORMS) on generated PDFs plus scanned, deliberately degraded versions of the same documents.

Layout identification came out at 100% across five vendors whose invoices look nothing alike, trained on just three documents each. On dense tax returns with 33 fields crammed into a tight three-column grid, roughly 0.03 units apart, it located every field. Drift detection behaved the way I hoped: as I pushed forms further off their baseline, the drift scores climbed smoothly from 0.012 up to 0.047 and tripped the flag once they crossed the threshold. On injected anomalies (shifted, scattered, reversed, wrong-form) it caught four of five. And because it's all local computation, lookups stay under a millisecond.

Now the honest part.

The benchmark runs on synthetic PDFs generated with ReportLab and their scanned variants, not a big real-world labeled corpus. Sample sizes are small, around 20 forms per scenario.

There's no proper precision/recall/F1 yet because I don't have a ground-truth label set built. The numbers above are operational rather than a formal comparison against baselines, and I'd treat them that way.

The one anomaly it missed was the "same layout, just fewer fields" case. The fields that remain are still sitting in the right places, so spatially nothing looks wrong. You could argue that's correct behavior, but it's worth knowing going in.

Handwritten forms are a dead end, because Textract FORMS gives back nothing to learn from. That's an upstream limitation, not something this library can paper over.

I'd rather you hit these on the page than in production.

Try it

Clone it and install from source. This works right now, nothing to sign up for:

git clone https://github.com/aws-samples/sample-textract-field-memory
cd sample-textract-field-memory
pip install -e .
python examples/demo.py   # runs on synthetic data, no AWS needed

demo.py walks through template learning, spatial matching, anomaly detection, and drift analysis on synthetic data, so you can watch the whole thing work without credentials or config.

Repo: github.com/aws-samples/sample-textract-field-memory

It lives under aws-samples but there's no AWS dependency in it. Pure standard-library Python, works completely offline with any OCR source that gives you field names and boxes. MIT-0 licensed.

If you run document pipelines, the thing I'd most like feedback on is turning these stress tests into a real labeled benchmark. And if the "remember it, stop re-discovering it" idea resonates, you've got the whole pitch.