DEV Community

Arthur031221
Arthur031221

Posted on

Search the words inside your screenshots without uploading them

I save screenshots so I will remember things. Boarding passes, door codes, a receipt, a setting I changed once. Then I need one of them and I scroll through a hundred thumbnails.

Snootling is a single HTML page that fixes that without an account or an upload. You drop screenshots on it, it reads the words in each one, and you search. The matched words light up inside the picture. Then it saves everything as one HTML file you keep.

A wall of 100 screenshots, a search for gate e29, and the saved file reopened offline

How it reads

The page carries tesseract.js 7, its WebAssembly core and the English model, so it downloads nothing. Four workers read pictures in parallel, and each word box is scaled back to the original size so the highlight lands on the right pixels at any zoom. The page's content security policy sets default-src 'none', so its own code cannot fetch or post anything. A policy does not cover navigation, so I call it a guard, not a proof. The tests record every request and require them all to be local.

The part that was not obvious

OCR drops words. In my first run, a small "Gate" label next to a huge gate code was read as noise, and the search gate e29 found nothing because every word had to match. Three changes fixed it:

  • If no picture has every word, show the closest ones, and weight rare words (e29) above common ones (gate).
  • Forgive one misread character in longer words.
  • For codes, forgive look-alikes such as 0 and O, or 1 and l.

The numbers

I drew 100 made up phone screenshots, 31 in dark mode and 20 in small type, each with one made up code. Then I read them in the page, searched for each code and checked that the lit words sit on it. Chromium 153 found 100 of 100 and Firefox 155 found 99. The saved file is 14 to 18 MB. I reopened it with the network off and repeated 20 searches: 20 of 20 in both. The method and the raw results are in the repo.

Synthetic screenshots are clean. Real ones will do worse.

What is rough

English only. No Safari testing. The saved file holds your full screenshots, so anyone with it can read what is in them, passwords included.

Try it

curl -LO https://raw.githubusercontent.com/Arthur031221/snootling/main/dist/snootling.html && open snootling.html
Enter fullscreen mode Exit fullscreen mode

Source and tests: https://github.com/Arthur031221/snootling. MIT licensed. I would like to hear how it does on your screenshots.

Top comments (2)

Collapse
 
launchgatecheck profile image
Launch Gate •

The 100-code test measures retrieval, but the look-alike rule needs a near-collision fixture too: two screenshots with codes E29 and E2O, plus a query for E2O when OCR reads that O as 0. Does an exact match rank above a normalized one, and can the UI label the latter as approximate?

For a gate or door code, finding a plausible neighboring screenshot is a different failure from finding none. I'd also include a code that appears nowhere in the set and check how the "closest ones" fallback presents it, so a fallback result isn't mistaken for a confirmed match. I haven't run Snootling; these are suggested retrieval-boundary fixtures from the matching rules described here.

Collapse
 
arthur031221 profile image
Arthur031221 •

Exact matches rank above look-alike matches: an exact word scores 3, while a code match after normalizing look-alike characters scores 0.7. The UI does not label an individual result as approximate. When a multiword query has no complete match, it labels the list "No screenshot has every word" and shows the closest results. A single unmatched code returns no results, with a "Nothing matched" message. You're right that the current fixtures don't test near-collisions like E29 and E2O, or whether a fallback could be mistaken for a confirmed match. I'll add those cases.