My game wiki's build fails if a number isn't on the screenshot it cites

#javascript #macos #ocr #testing

I run a small fan wiki for Dressmaker, a sewing sim that came out on Steam this month. The page people actually use is a table of the game's commissions: 37 cards, each with a client, a description, and a list of thresholds like Quality: 75, Elegant: 65, Avoid: Black, plus a budget.

Those numbers are the whole product. If I write 75 where the card says 80, someone sews a dress to 75, the client rejects it, and they redo an hour of work. Nothing in a normal build catches that. TypeScript is happy, links resolve, the page renders.

So the build now re-reads every card against the screen it came from, and fails if it can't find the words and numbers there.

Where the text comes from

Every card in my data file has a seen list: which screenshots or video frames it was copied from. Three kinds of source: frames from a recorded playthrough, screenshots in a Steam community guide, and screenshots in a press article.

I didn't want to type those screens out by hand, because then I'd be checking my typing against my typing. macOS ships a decent OCR engine (Vision), and you can call it from JXA without Xcode or Python bindings:

ObjC.import('Vision');
function run(argv) {
  const out = [];
  for (const p of argv) {
    const req = $.VNRecognizeTextRequest.alloc.init;
    req.recognitionLevel = 0;            // accurate
    req.usesLanguageCorrection = false;  // don't "fix" game words
    const h = $.VNImageRequestHandler.alloc.initWithURLOptions($.NSURL.fileURLWithPath(p), $({}));
    h.performRequestsError($([req]), null);
    const lines = [];
    for (let j = 0; j < req.results.count; j++) {
      lines.push(req.results.objectAtIndex(j).topCandidates(1).objectAtIndex(0).string.js);
    }
    out.push(p.split('/').pop() + '\t' + lines.join(' '));
  }
  return out.join('\n');
}

Run it with osascript -l JavaScript ocr.js *.png > cards.tsv. One line per image: a ref, a tab, the text. 65 images in total. Turning language correction off matters here; with it on, Vision tends to "correct" in-game words into ordinary English ones.

What the gate checks

A small Node script parses the data file and, for each card:

every ref in seen exists in the corpus
the card title is on every one of its screens
the description is on at least one of them
every threshold's label and exact number are on one screen
every rule sentence ("Avoid: Black", "Use only: Silk") is on one screen
the budget, if there is one, is printed next to the word Budget

Any miss is exit 1, and so is a missing corpus or a data file that parses to fewer cards than expected. A check that quietly passes when its input is empty is worse than no check.

OCR noise vs. real mistakes

OCR output is messy. "Exhibition" comes back as "Exbibition", bullets turn into • or r, and "to study" loses its space. So text matching has to be fuzzy, and the hard part is making it fuzzy enough for OCR without letting real errors through.

My first version allowed an edit distance of 12% of the phrase. Then I tested it by changing a description on purpose, swapping "picnic" for "party", and the check still passed. A single swapped word in a 60-character sentence fits inside 12%.

What works better is two rules together:

the whole phrase within 4% edit distance, matched over windows that start at word boundaries
and every word of five letters or more has to be on the screen, give or take one letter

const words = new Set(hay.split(' '));
for (const w of needle.split(' ')) {
  if (w.length < 5 || words.has(w)) continue;
  if (![...words].some(h => Math.abs(h.length - w.length) <= 1 && lev(h, w) <= 1)) return false;
}

"Exbibition" is one letter off "Exhibition", so it passes. "party" is nowhere near "picnic", so it fails.

Numbers get no tolerance at all. I strip digits before the fuzzy comparison and check each number with an exact regex instead. The game prints the same threshold two ways: Quality: 75 on an open card, and Quality 100/80 on a finished one (score/target). So the pattern accepts either label N or /N, and nothing in between.

Two bugs the gate had

The first: one good screen hid a bad one. A card can cite two screens. My title check originally used .some(): pass if the title is on any of them. A reviewer found a card citing a screenshot that actually showed a different commission. The other screen was correct, so the check passed. Now the title has to be on every cited screen. After that change I put the wrong screenshot back to confirm the build goes red, then took it out again.

The second was a number that went missing. On one Steam guide screenshot OCR read the Tassel row as just "Tassel", with no "5/5" after it. The gate correctly refused to confirm the threshold. The fix was not to loosen anything: I cropped the card region, OCR'd the crop as its own ref (dfs-26-crop reads "Tassel 5/5"), and cited both.

Cost

The script is about 170 lines and takes 30 ms. Most of the effort went into the OCR corpus, not the checker. My takeaway: if a page's value is its numbers, the build should read the numbers back from the source, not from my own notes.

The finished table is the Dressmaker commission finder, if you play the game and want to look up a card before you start cutting fabric.

DEV Community