We all have that folder of 500+ screenshots we can never find anything in. I got tired of scrolling through mine looking for a receipt from three months ago, so I built Screenshot Vault — an app that reads the text in every screenshot on your phone and lets you search it like Google.
Here's how it's built, and a few decisions that ended up mattering more than I expected.
The stack
Expo (React Native) + Expo Router — file-based routing, EAS for native builds
@react-native-ml-kit/text-recognition — on-device OCR via Google ML Kit
expo-sqlite — local persistence
expo-media-library — reading the device's Screenshots album
Gemini 3.6 Flash — categorization, titles, tags, called through a small Next.js API route
Next.js + Vercel — landing page + the backend proxy
Decision #1: OCR stays fully on-device
My first instinct was to upload screenshots to a server for processing. Then I actually thought about what's in a typical screenshot folder — bank OTPs, payment confirmations, private chats, addresses. Uploading that to a database I control is a liability for me and a real risk for users if anything ever leaks.
Google ML Kit's text recognition runs entirely on-device, for free, with no server round-trip. So OCR happens locally, and the only thing that ever leaves the phone is the already-extracted text — sent to Gemini for a lightweight categorization call, not the image itself. This also happens to be cheaper to run at scale, since I'm not paying for image storage or bandwidth.
Decision #2: Don't block the user on a full scan
If someone installs this with 500 existing screenshots, running OCR + AI on all of them before showing anything would mean a 10-20 minute wait on first launch. That's an instant uninstall.
Instead, processing is staged and resumable:
CREATE TABLE screenshots (
id TEXT PRIMARY KEY,
uri TEXT,
text_content TEXT,
category TEXT,
ai_title TEXT,
tags TEXT,
ocr_done INTEGER DEFAULT 0,
ai_done INTEGER DEFAULT 0,
created_at INTEGER
);
On open:
- Show whatever's already in SQLite — instant, zero wait
- Insert bare rows for any new screenshots found (thumbnail visible immediately)
- Background pass: OCR everything with ocr_done = 0, newest-first
- Background pass: AI-classify everything with ocr_done = 1 AND ai_done = 0, newest-first
If the app gets closed mid-scan, it just picks up from wherever the flags left off next time — no reprocessing, no lost work. This is basically the same idea Google Photos uses for its "preparing your library" background indexing.
Decision #3:_ Never put the AI API key in the client_
Early version called Gemini directly from the app with the key in an env var. Realized pretty quickly that anyone who installs the APK can extract that key and rack up usage on my bill — env vars prefixed EXPO_PUBLIC_ get bundled straight into the client, no exceptions.
Moved it behind a single Next.js API route on the same Vercel project as the landing page:
export async function POST(req: NextRequest) {
const { text } = await req.json();
const apiKey = process.env.GEMINI_API_KEY; // server-only, never shipped to client
const res = await fetch(GEMINI_ENDPOINT, {
method: 'POST',
headers: { 'x-goog-api-key': apiKey },
body: JSON.stringify({ /* prompt asking for category + title + tags in one call */ }),
});
// ...parse and return
}
One call returns category + title + tags together instead of three separate requests — matters a lot on Gemini's free tier, which caps out fast.
Still early and Android-only right now, testing with a small group before a wider release. If you want to follow along or try it when it's ready: screenshot-vault-lac.vercel.app
Curious if anyone's tackled similar on-device vs. cloud tradeoffs for AI features — happy to talk through any of this in the comments.
Top comments (6)
OCR을 온디바이스에 남기고 이미지 대신 추출 텍스트만 분류 API로 보내는 경계가 설득력 있습니다. 특히 ocr_done과 ai_done을 분리해 최신 항목부터 재개하는 방식은 500장 첫 실행의 이탈을 줄이면서 실패 복구도 단순하게 만드네요. 한 가지 더 확인한다면 OTP·계좌번호처럼 민감한 패턴은 추출 텍스트 단계에서도 서버 전송 전에 로컬에서 마스킹하는 선택지를 검토해볼 만합니다.
you're right that the current boundary (image stays local, text goes to the API) isn't tight enough on its own. Adding local pattern-masking for things like OTPs and account numbers before the text ever leaves the device is a real gap to close, especially since on-device privacy is the whole pitch. Appreciate you thinking this through this carefully this is going in before I ship wider.
implemented this. Extracted text now goes through a local redaction pass (OTP codes, card numbers, long digit sequences) before it's sent for classification, so even the lightweight text-only API call has sensitive numbers masked. Full unredacted text stays local for search/display. thanks men
The staged/resumable processing is probably my favorite decision here. It turns a potentially long-running AI workflow into something the user doesn't have to care about.
I also like that the privacy boundary is explicit: image stays on-device, while only extracted text crosses the boundary.
One question I'd be curious about: as the library changes over time, how are you handling the validity of previously generated titles/tags? If the categorization was based on an older model or context, do you ever reconsider it, or is it treated as permanent once written?
right now it's treated as permanent once written, mostly because I haven't built a re-categorization pass yet.
Realistically it should be revisited if the model improves or a user's own tagging patterns change, stale tags could get confusing over time. My rough plan is to keep a lightweight version marker on each tag (which model/prompt version generated it) so I can selectively re-run categorization on older items later without redoing the whole library. Not built yet, but flagging it now so it doesn't become a mess to retrofit.
That’s a solid approach. The version marker also makes the generated metadata auditable instead of treating it as permanent truth.
I think the interesting next question is what should actually trigger re-categorization: a new model/prompt version, a change in the user’s behavior, or some signal that the existing classification is no longer reliable.
That distinction between “old” and “no longer trustworthy” seems really important for AI-generated metadata in general.