DEV Community

Cover image for Draku: Exorcising the Ghosts in Your Camera Roll with Offline Open-Weight AI 🦇
Devansh
Devansh

Posted on AI-assisted

Draku: Exorcising the Ghosts in Your Camera Roll with Offline Open-Weight AI 🦇

#devchallenge #hf26challenge #opensource #rust

Hacktoberfest Open-Source AI Challenge: Ghost in the Machine Submission 👻🌼

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

You are standing at the pharmacy counter. The pharmacist asks for your prescription. You know it is on your phone—somewhere between 4,000 WhatsApp forwards, food photos, and old boarding passes. You scroll past IMG-20261011-WA0004.jpg, Screenshot_2026-09-14.png, and scan_final_v2.pdf.

Screenshots are the most-used notebook on earth that nobody calls a notebook. The problem is that a screenshot stores pixels, not meaning. And inside those pixels live four kinds of digital ghosts haunting every laptop and phone:

  1. The Ghost of a Date: A warranty receipt that expires next week, a medical lab test due for a repeat, or a coupon that dies tonight—trapped inside a flat image where your calendar can't see it.
  2. The Ghost of a Secret: A national ID number, tax form, blood test report, or Wi-Fi password sitting unredacted in a folder that any app with gallery permissions can read.
  3. The Ghost of a Place: Hidden EXIF GPS coordinates and device serials silently attached to documents you are about to email or share.
  4. The Dead Weight: Last spring's OTP screenshots, expired boarding passes, and blurry duplicates quietly haunting your storage long after their purpose has passed.

Meet Draku—a fluffy black cat with crimson bat wings and eyes full of black flames. He sees everything on your device, and he tells no one.

Feature List

Draku is a 100% offline, open-source desktop application (built with Tauri) that reads every screenshot, photo, and PDF in a folder once using local PaddleOCR and Google's open-weight EmbeddingGemma, turning that single reading into four concrete outcomes:

  • Find: Turns IMG-20261011-WA0004.jpg into 2026-10-11 Dr Patel blood test prescription.jpg and embeds the semantic context so you can type "blood test" or "router password" in plain English and find it instantly.
  • Remember: Extracts dates alongside their surrounding context ("Warranty ends in 6 days", "Refill by 10 Nov") and generates a one-tap calendar reminder once you confirm.
  • Protect: Detects exposed ID numbers, card digits, and passwords at the exact pixel bounding box, burns an irreversible pixelation blur into a safe copy, and strips hidden GPS/EXIF metadata.
  • Let Go: Calculates a transparent, inspectable "dead file" score (based on age, category, passed dates, duplicate distance, and blur) to move dead OTPs and expired tickets into a rest folder—only when you press the button.

Zero bytes uploaded. Zero accounts. Zero settings screens.

Outcome Before After What produces it
Find IMG-20261011-WA0004.jpg 2026-10-11 Dr Patel blood test prescription.jpg, findable by typing "blood test" local OCR → rule-based reader → EmbeddingGemma-300M → semantic search
Remember A warranty screenshot quietly expiring A real .ics calendar reminder, one tap date extraction with surrounding words → confirm → local .ics
Protect A photo exposing an ID number and GPS A safe copy with the blur burned into pixels and location metadata stripped local OCR bounding boxes + EXIF → confirm → clean copy
Let go OTPs, dead tickets, duplicates Rested in a folder you can restore from transparent signals → an explainable dead score → "Set aside"

Demo

  • Interactive Live Web Demo & Landing Page: https://0dvsh0.github.io/draku/ (Try the interactive "One Screen" sandbox and the 4-outcome inspector right in your browser with zero setup!)
  • Desktop Release Installer (Windows x64 · 182 MiB with models bundled): Download Draku v0.1.0

Four Real Captures — Airplane Mode On the Entire Time

Every capture below is recorded directly from the real desktop application on Windows with Wi-Fi off and Airplane Mode visibly enabled—no cloud calls, no mockups, no staged telemetry. The full airplane-mode walkthrough script lives in the repo as DEMO.md.

1. Find — Search the Pixels, Not the Filename

Local PaddleOCR extracts the text, a structured rule-reader extracts human naming cues, and Google's EmbeddingGemma-300M embeds the reading into a 768-dimensional local vector. Type "internet password" and the router login surface appears immediately—pure meaning match with zero keyword overlap.

Find: semantic search without keywords

2. Remember — Trapped Deadlines Become Calendar Events

Dates are extracted alongside their surrounding contextual words, allowing Draku to recognize that "Warranty ends in 6 days" or "Refill by 10 Nov" is an actionable deadline rather than an old receipt date. One click generates an RFC-compliant .ics calendar file accepted by your system calendar—no cloud sync intermediary.

Remember: one-tap calendar reminder

3. Protect — Permanent Burn-In Redaction & EXIF GPS Stripping

PaddleOCR returns exact pixel bounding boxes for sensitive ID numbers, card digits, and credentials. Clicking "Make a safe copy" burns irreversible pixelation blur directly into new pixel buffers and strips hidden EXIF location metadata. The original document remains untouched.

Protect: burn-in redaction and GPS stripping

Before and after, side by side — the sensitive ID number and GPS coordinates visible in the first frame are permanently eradicated in the safe copy:

Protect: before and after

4. Let Go — Transparent Rest Scoring with One-Tap Restore

Expired boarding passes, old OTP screenshots, and blurry duplicates earn transparent, inspectable "dead file" scores with human-readable explanations. Clicking "Set aside" lays them to rest in .draku/rest/—never deleted, completely reversible with one-tap Undo.

Let go: explainable rest with Undo

Code

GitHub logo 0dvsh0 / draku

Draku

Your files, finally understood.

Draku is a free, open-source, on-device AI file manager for Windows. It reads every screenshot, photo and PDF on your machine — it finds what you need warns you before a date passes, hides what should stay private, and lets go of what is dead. Nothing leaves your machine.

Built for Hacktoberfest 2026 · Week 4 — Ghost in the Machine.

draku/
├── CONTEXT.md     ← project context: plan, stack, tokens, status (read first)
├── site/          ← the landing page (static HTML/CSS/JS + WebGL embers)
├── app/           ← the product: Tauri 2 desktop app (Rust + React/TypeScript)
│   ├── src/           One Screen UI, mascot design tokens (tokens.css)
│   ├── scripts/       bundle prep + one-time model fetch for source builds
│   └── src-tauri/     scanner · reader · OCR · embeddings · actions
└── site/About.md
…
View on GitHub

To build and run Draku from source (fetches the local models once, then runs completely offline forever):

git clone https://github.com/0dvsh0/draku && cd draku/app
npm install && node scripts/fetch-model.mjs
npm run tauri dev
Enter fullscreen mode Exit fullscreen mode

Clean Architecture & Codebase Layout 

draku/
├── site/               ← Interactive web sandbox & landing page (vanilla HTML/CSS/JS, WebGL embers)
└── app/
    ├── src/            ← The One Screen UI (React 19 + TypeScript, mascot design tokens)
    ├── scripts/        ← Model fetchers, NSIS installer packager, automated CI smoke test
    └── src-tauri/      ← Native Rust core touching every byte
        ├── scan.rs     ← Folder walker + dHash perceptual duplicate detection
        ├── extract/    ← The READER: PaddleOCR, PDF text parser, kamadak-exif, rule reader
        ├── embed.rs    ← Bundled EmbeddingGemma-300M loader + local cosine similarity search
        ├── score.rs    ← Transparent "Let Go" arithmetic engine — weights you can read
        ├── store.rs    ← SQLite (WAL mode) — single local embedded DB, zero daemons
        └── actions.rs  ← Safe copies, .ics generation, set-aside / restore (the ONLY disk mutations)
Enter fullscreen mode Exit fullscreen mode

The Embedding Model Lives Beside the Executable, and Never Talks

EmbeddingGemma's 4-bit quantized weights ship inside the installer as model_q4.onnx alongside an external initializer file. Draku's native Rust core loads both in memory directly into ONNX Runtime with DirectML hardware acceleration—without requiring CUDA or external services:

// app/src-tauri/src/embed.rs — the offline path is the only shipped path
let mut bundled = UserDefinedEmbeddingModel::new(read("model_q4.onnx")?, TokenizerFiles {
    tokenizer_file:          read("tokenizer.json")?,
    config_file:             read("config.json")?,
    special_tokens_map_file: read("special_tokens_map.json")?,
    tokenizer_config_file:   read("tokenizer_config.json")?,
})
.with_pooling(Pooling::Mean)
.with_quantization(QuantizationMode::None)
.with_external_initializer("model_q4.onnx_data".into(), read("model_q4.onnx_data")?);

TextEmbedding::try_new_from_user_defined(bundled, InitOptionsUserDefined::new())
Enter fullscreen mode Exit fullscreen mode

When you search "internet password", your query is converted to a 768-dimensional vector, and cosine similarity matches against local readings in SQLite. There is no HTTP client anywhere in the crate.

The Judge is Arithmetic You Can Read

Rather than relying on an unexplainable black-box classifier, score.rs exposes a plain struct of inspectable signals. Every weight is an auditable constant with a clear rationale:

// app/src-tauri/src/score.rs
pub fn judge(s: &Signals) -> Verdict {
    let mut score = 0.10;
    score += (s.age_days / 180.0).min(0.30);
    if s.is_otp      { score += 0.44; }   // one-time codes only live once
    if s.date_passed { score += 0.50; }   // the deadline is gone; the file overstayed
    if s.is_dup      { score += 0.50; }   // a near-identical file exists
    // …the result clamps to [0, 1]; above 0.60 a file earns the Let-go card
    Verdict { dead: score.clamp(0.0, 1.0), reason: choose_reason(&s) }
}
Enter fullscreen mode Exit fullscreen mode

Verified, Not Vibes

  • 42 cargo tests on the full OCR build: Covering EXIF GPS scrubbing, redaction bounding box math, and end-to-end vector queries against bundled Gemma weights. 41 tests also run on the light build (--no-default-features) for machines that cannot run PaddleOCR.
  • Automated CI Installer Lie Detector: scripts/smoke-test-installer.mjs, wired into GitHub Actions, silent-installs the NSIS package on a clean Windows runner and verifies every bundled binary: draku.exe, DirectML.dll, all 4 OCR models, and all 6 embedding files.
  • Verified Checksums: The release build is published with verifiable SHA-256 signatures: SHA256: 19473E9A5AEC88BDE82DF50CD697E7A9450A0B0EA88B49E65B57F89843E52791

How I Built It

Most AI tools bolt four separate pipelines onto four separate problems: one tool for semantic search, another for reminders, another for PII redaction, and a fourth for storage cleanup. That wastes compute and clutters the UI.

Draku is built around a single architectural rule: One reading. Four outcomes.

[ Screenshot / Photo / PDF ]
             │
             ▼
   ┌───────────────────┐
   │  Local PaddleOCR  │ ──► Extracts text + exact pixel bounding boxes + EXIF
   └─────────┬─────────┘
             │
             ▼
   ┌───────────────────┐
   │ Structured Reader │ ──► Parses dates, PII patterns, category & dead-score
   └─────────┬─────────┘
             │
             ▼
   ┌───────────────────┐
   │  EmbeddingGemma   │ ──► Generates local semantic vector embeddings
   └─────────┬─────────┘
             │
             ▼
      [ reading.json ]
             │
   ┌─────────┴──────────┬────────────────────┬───────────────────┐
   ▼                    ▼                    ▼                   ▼
[ FIND ]           [ REMEMBER ]         [ PROTECT ]         [ LET GO ]
Semantic search    Contextual .ics      Pixel-burned PII    Transparent
& smart rename     deadline alerts      & EXIF stripping    rest scoring
Enter fullscreen mode Exit fullscreen mode

When Draku scans a file like IMG-20261011-WA0004.jpg, it produces a single structured internal artifact (reading.json):

{
  "name":     "2026-10-11 Dr Patel blood test prescription",
  "category": "medical / prescription",
  "dates":    [{ "refill_by": "2026-11-10" }],
  "private":  ["patient_id", "gps"],
  "dead":     0.04,
  "network":  null
}
Enter fullscreen mode Exit fullscreen mode

The Stack & Design Decisions

Layer Choice Why It Earns Its Place
Shell & Core Tauri 2 · Rust Native filesystem speed, minimal memory footprint, and zero open sockets. Bundles everything into a clean 182 MiB installer instead of a bloated Electron container.
Semantic Memory Google EmbeddingGemma-300M (4-bit ONNX) Generates 768-dimensional embeddings entirely on-device from open weights, making true offline semantic search possible from the first run.
Local Runtime fastembed + ONNX Runtime + DirectML Hardware acceleration across modern Windows GPUs/NPUs with zero CUDA configuration. Loads external weight initializers directly in memory.
Vision & Text PaddleOCR PP-OCR (Apache-2.0) Detection + classification + recognition returning exact bounding boxes that directly feed Protect's pixel-burned redaction.
Duplicate & Metadata dHash + kamadak-exif 64-bit visual perceptual hashing detects rescaled copies; parses EXIF metadata to reveal hidden GPS coordinates and camera serials.
Frontend React 19 + TypeScript + Vite Type-safe IPC boundaries, crisp UI with mascot tokens, and @fontsource vendored typography (zero CDN calls).
Persistence SQLite (WAL) via rusqlite Single embedded database in the local app directory storing readings, vectors, and approved action history.
  1. Desktop Runtime (Tauri): Gives Draku native filesystem speed and a tiny memory footprint across Windows without bundling a bloated browser backend.
  2. Eyes (PaddleOCR + EXIF Parser): Instead of sending images to a multimodal cloud API, local OCR extracts raw text and exact pixel coordinates. Knowing the exact bounding box of an ID number allows Draku to burn pixelation directly into the image buffer so redactions can never be peeled off.
  3. Memory (EmbeddingGemma): Google's open-weight EmbeddingGemma embeds the structured reading locally on the user's CPU/GPU. When you search "pharmacy receipt" or "cholesterol check", EmbeddingGemma matches the meaning even if those exact words aren't in the filename.
  4. Judgment (Transparent Rule & Scoring Engine): For Let Go, I deliberately avoided a black-box classifier. Draku computes a transparent weighted score from inspectable signals—file age, category (OTP, ticket, receipt), whether the extracted date has passed, perceptual duplicate distance, blurriness, and byte size. You can read the exact reason Draku suggests letting a file rest.
  5. The 5 "No" Promises:
    • No upload: There is no cloud mode to toggle on. network is always null.
    • No silent deleting: "Let go" moves files to a rest folder you control.
    • No guessing out loud: Draku suggests; nothing changes on disk until you click confirm.
    • No accounts or subscriptions: Free, MIT-licensed, no telemetry.
    • No settings screen: If a feature needs a configuration toggle to be usable, the design isn't finished yet. One screen: three cards and a search bar.

Honest Boundaries

Windows is the primary platform where DirectML and WebView2 ensure a guaranteed, zero-friction experience out of the box. For hardware without OCR capabilities, Draku provides a lightweight build (--no-default-features): semantic search, dates, duplicate detection, and rest actions function without OCR, while Protect reports what can be discerned from names and metadata.

Why Does Open Innovation Matter?

If a tool asks to read your entire camera roll and Downloads folder, it is asking to read your life: your prescriptions, tax returns, national ID cards, bank statements, and private family photos.

You cannot build a trustworthy version of Draku on a closed, cloud-hosted API. Here is why open-weight models and open-source code are the only way this product can exist:

  1. It Reads Your Camera Roll—So You Must Be Able to Read It Back: When a proprietary cloud app claims "we don't train on your photos," you have to take their legal team's word for it. Because Draku pairs open-weight models (EmbeddingGemma and PaddleOCR) with an MIT-licensed codebase, anyone can inspect the source, unplug their Ethernet cable, switch on Airplane Mode, and verify that zero bytes ever leave the machine.
  2. No "Storage Ransom" Economics: Cloud photo organisers charge you a monthly subscription once your storage fills up—meaning they have a financial incentive for you to hoard dead screenshots. Because Draku runs on your own silicon at zero marginal cost per file, a tool that helps you clear space never tries to sell you more of it.
  3. Different Tools for Different Jobs (Not Everything Is Windows Recall): Unlike invasive background screen-recorders that capture everything you do every few seconds, Draku captures nothing on its own. It reads only the folder you hand it, executes one pass, and puts you in complete control.
  4. Built to Outlive Any Server: Closed APIs deprecate models, hike token prices, or shut down. Because the weights are bundled right inside the Draku installer (fetch-model.mjs), your personal file guardian will work identically ten years from now on an air-gapped laptop.

The Spirit of It

When I saw the Ghost in the Machine theme coinciding with Halloween and Día de los Muertos, I didn't want to build another spooky terminal chatbot pretending to be a poltergeist. I started thinking about the real ghosts that live inside our machines every single day.

Our devices are full of unfinished business: the prescription we forgot to refill, the warranty we missed by three days, the old ticket stub from a trip we took a year ago, and the dead OTP screenshots cluttering the drawer. In the spirit of Día de los Muertos, some things in our digital lives deserve to be remembered and honoured on time, our private secrets deserve to be protected, and the things whose time has passed deserve to be laid peacefully to rest.

That is why Draku is a tiny guardian familiar—eyes of black flame to read the dark pixels, bat wings to shield your secrets, and a quiet nudge to help you let go of what is dead.

Prize Categories

  • Best Use of Gemma ($200): Google's open-weights EmbeddingGemma-300M is Draku's core semantic engine, not an optional gimmick. The shipped Windows installer bundles the 4-bit ONNX weights, tokenizer, and external initializer, loaded into memory by fastembed + ONNX Runtime with DirectML acceleration. Every semantic search query ("internet password" finding router credentials, zero keyword overlap) and every human-readable filename originates from a Gemma-family model executing 100% offline.
  • Best Use of GitHub Copilot ($100): The automated build-and-verify workflow is orchestrated via GitHub Actions (ci.yml): linting and type-checking (React 19 + TypeScript + Vitest), dual cargo test suites (light and full OCR) with model caching, and a Windows runner job that builds the NSIS installer, silent-installs it in a clean environment, and verifies the integrity of every bundled model and binary.

Built with 🦇 for Hacktoberfest 2026 — Draku on GitHub · Landing page · Download the installer

Top comments (1)

Collapse
 
devanshdubey profile image
Devansh •

draku mascot credit: K | Pvmpkin (@pvmpkinart)