The feature that sounded simple in the spec
"Let teachers scan a paper test and have it auto-graded." That was the one-line spec for a feature we shipp
ed in Cursora Assess, our grading tool. It sounded like a weekend project. It was not.
Why printed multiple-choice OCR is the easy 80%
If every test were a single canonical layout with clean printed bubbles, this is a solved problem -- that's
exactly what commercial scantron hardware has done since the 1970s. Our first version handled that fine: g
enerate a test as a PDF with a fixed answer-grid layout, print it, scan it back, and a well-tuned OCR/bubbl
e-detection pass gets you close to 100% accuracy.
Where it actually gets hard
Two things blow up the easy version:
Randomized variants. To make cheating-by-glancing-at-your-neighbor harder, Assess generates multiple sh
uffled variants of the same test -- different question order, different answer order per question. That mea
ns the grading pass can't assume a fixed answer key position; it has to read a variant identifier off the s
heet first, then grade against the matching key, not just "the" key.
Real-world scans, not clean lab conditions. Teachers photograph these with a phone, at an angle, someti
mes in bad lighting, sometimes slightly creased. A bubble that's 90% filled versus 40% filled versus "start
ed to fill then scribbled out" all need different handling, and a naive brightness threshold falls over imm
ediately once the photo isn't perfectly lit.
What actually worked
- Perspective correction before anything else -- detect the page's corner markers and de-skew/de-warp the i mage so the grid math downstream can assume a roughly rectangular, aligned sheet
- Per-bubble fill-ratio scoring rather than a single global threshold, so lighting gradients across one pho to don't tank accuracy on one side of the page
- Variant ID read as its own first-class OCR step, gating which answer key the rest of the pipeline grades against
- A confidence score per answer, not just a binary read -- low-confidence bubbles get flagged for the teach er to eyeball rather than silently guessed
The honest caveat
This isn't magic, and we don't pretend it is: genuinely ambiguous marks (half-erased, two bubbles filled) s
till get flagged for a human to resolve rather than auto-resolved with a coin flip. Grading in seconds inst
ead of an evening doesn't require 100% unattended accuracy -- it requires being right when it's confident a
nd honest when it's not.
If you've built anything in this space (form OCR, scantron-alternative pipelines, handwriting recognition)
I'd like to compare notes -- drop it in the comments.
Top comments (0)