DEV Community: Varnit Kumar

SlothDesk 🦥: You Do Nothing. Your Desktop Clutter Sorts, Renames, and Audits Itself

Varnit Kumar — Sun, 25 Oct 2026 09:02:50 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Every modern knowledge worker dreads the Downloads and Desktop Black Hole. Over days and weeks, our workspaces accumulate hundreds of cryptically named files:

Screenshot_2026-03-12_at_11.23.41.png
INV_82910_final(1).pdf
receipt_unnamed.jpg
analytics_churn_export_raw(3).csv
mutual_nda_acme_unsigned.pdf

Organizing them manually requires opening each file, deciphering invoice numbers, typing dates, creating nested folders, resolving duplicate collisions, and dragging files — wasting 3+ hours every month.

SlothDesk 🦥 is an autonomous ambient background daemon that operates at a calm, deliberate "Sloth Pace". As chaotic downloads hit your drop zone:

Magic Header Inspection: Inspects raw binary magic bytes to verify genuine file types and isolate malicious executables (MZ, ELF) into a quarantined vault.
On-Device Gemma 3 (4B Vision/Text): Inspects textual contents and visual screenshots locally with zero cloud surveillance, extracting dates, vendors, amounts, and subjects.
Deterministic Semantic Renaming: Renames files using clean ISO-8601 timestamps and extracted entities (e.g., 2026-10-18_GitHub_Enterprise_Invoice_$21.pdf).
Cryptographic Proof of Zero Data Loss: Computes dual SHA-256 checksums before and after movement, guaranteeing 100% data integrity with zero data alteration.
ElevenLabs Ambient Morning Digest: While you sleep, SlothDesk synthesizes a soothing 30-second audio morning brief with an interactive waveform player: > "Good morning. While you were sleeping peacefully, SlothDesk swept through your downloads folder at a calm, deliberate pace. I sorted 18 messy files into clean tax and code vaults. All SHA-256 checksums verified. You did zero manual work. Go back to sleep."

Demo

🌐 Live Web Application: https://sloth-desk-lemon.vercel.app
🚀 Production DNS: https://slothdesk.tech
📖 Interactive API Documentation (Swagger): Available under /docs on the backend.

Key Interactive Features:

One-Click "Seed Chaotic Desktop": Injects realistic messy downloads (invoices, coffee receipts, UI screenshots, specs) so you can test automated sorting instantly.
Interactive File Comparison Drawer: Split-view showing original chaotic metadata side-by-side with Gemma 3's semantic extraction and confidence pills.
Tactile Sloth Governor: Toggle between Hibernate (25s idle-only batching) and Active Sweep (8s cadence).
Bespoke Waveform Scrubber: Play, seek, and adjust speeds (0.75x Sloth speed) on the ElevenLabs morning voice brief.
Real Client-Side Blob Export: Generate and download dynamic CSV and JSON audit manifests directly in the browser with cryptographic SHA-256 proofs.

Code

vannu07 / SlothDesk

SlothDesk 🦥

"You do nothing. Your desktop clutter sorts, renames, and audits itself."
Built for Hacktoberfest 2026 Open-Source AI Challenge — Week 3: "Slow and Steady" (International Sloth Day).

🏆 Partner Track Submissions & Winning Criteria

Partner Track	Implementation Highlight	Technical Proof
Best Use of Gemma ($200)	Autonomous local Gemma 3 (4B Vision/Text) multimodal pipeline via Ollama REST API with deterministic structured JSON schemas. Extracts entities, vendors, amounts, dates, and assigns standardized semantic names (`YYYY-MM-DD_Vendor_Subject.ext`). Zero cloud surveillance.	`classifier.py`
Best Use of Render ($200)	Production containerization via multi-stage non-privileged Docker image (`UID 10001`), 10GB persistent disk mount `/app/data`, zero-downtime health probes (`/api/health/live`, `/api/health/ready`), and 1-click blueprint.	`render.yaml` & `Dockerfile`
Best Use of ElevenLabs ($100)	Ambient sleepy morning audio digest narrated via ElevenLabs text-to-speech (`George` / `JBFqnCBsd6RMkjVDRZzb` warm voice) with custom interactive waveform scrubber and audio synthesis fallback.	`voice_digest.py` & `MorningBriefPlayer.tsx`
Best

…

View on GitHub

GitHub Repository: https://github.com/vannu07/SlothDesk
License: MIT Open Source

How I Built It

1. Architecture & Tech Stack

Open-Weight AI Pipeline: Google Gemma 3 (4B Multimodal Vision/Text) running on-device via Ollama REST API with deterministic JSON schema enforcement and a deterministic mock fallback for CI/CD.
Backend & Security Middleware: Python FastAPI + SQLModel (async SQLAlchemy + aiosqlite):
- Path traversal containment guard (os.path.commonpath).
- Raw magic byte validator (%PDF, PNG, JFIF, RIFF/WEBP).
- Token-bucket in-memory rate limiter on ingest endpoints.
- Dual SHA-256 cryptographic verification before and after moving.
Ambient Audio Narration: ElevenLabs Text-to-Speech API (warm ambient narrator setting) paired with an offline pure-Python harmonic synthesizer generating gentle ambient chords.
Frontend Design System: Next.js 16 (App Router + Turbopack) crafted with a bespoke Swiss-inspired technical dark mode (deep slate #090a0f, titanium borders #1e2230, subtle emerald and warm amber status lights).
Deployment Pipelines:
- Turnkey Render Blueprint (render.yaml) with 10GB persistent disk storage and zero-downtime health probes (/api/health/live, /api/health/ready).
- Root Vercel Multi-Service Configuration (vercel.json) linking Next.js and FastAPI with internal service bindings (INTERNAL_BACKEND_URL).

flowchart TD
    A[Chaotic Desktop & Downloads] --> B[Magic Byte Header Guard]
    B --> C[Pre-Move SHA-256 Hash]
    C --> D[Gemma 3 4B Multimodal Classifier]
    D --> E[Semantic Renamer & Collision Resolver]
    E --> F[Atomic Move to Standardized Vault]
    F --> G[Post-Move SHA-256 Proof Verification]
    G --> H[ElevenLabs Morning Audio Briefing]
    G --> I[Live Audit Log & Manifest Export]

Why Does Open Innovation Matter?

Your downloads folder contains your most sensitive personal data: tax returns (W-2, 1099), medical receipts, bank statements, client NDAs, and private codebase screenshots.

Sending all of these unvetted files to closed, proprietary cloud APIs exposes users to data harvesting, cloud surveillance, and third-party data breaches.

Open innovation makes privacy-first desktop automation possible:

On-Device Autonomy: By utilizing open-weight models like Gemma 3 (4B), SlothDesk processes private invoices and documents locally on your own machine. Your sensitive financial receipts never leave your hard drive.
Determinism & Auditability: Open-source models allow developers to inspect, audit, and constrain AI outputs with exact JSON schema enforcement, removing arbitrary cloud model deprecations.
Zero Cloud Rent: You don't need a $20/month subscription or cloud API credits just to organize files on your own computer.

The Lazy Payoff

Zero Clicks. Zero Effort.

Normally, an organized desktop requires:

Opening the file to see what it is (10 seconds).
Figuring out the date, vendor, and amount (15 seconds).
Renaming INV_82910_final(1).pdf $\to$ 2026-10-18_GitHub_Enterprise_Invoice_$21.pdf (15 seconds).
Creating or finding the right folder and moving the file (10 seconds).
Resolving duplicate names if they collide (10 seconds).

Total manual effort: ~1 minute per file. Across 180 files accumulated in a month, that is 3 full hours of tedious, dreaded manual chore.

With SlothDesk:

You drop files in the folder (or let the daemon monitor your downloads black hole).
You walk away or go to sleep.
SlothDesk sorts, renames, resolves duplicate collisions, computes SHA-256 proofs, and narrates a 30-second audio summary.
Effort required: 0 clicks. 0 seconds.

Prize Categories

1. Best Use of Gemma ($200)

SlothDesk runs on-device Gemma 3 (4B Vision/Text) through an Ollama REST pipeline with strict JSON schema enforcement. Gemma inspects visual UI captures and textual document previews to infer structured metadata (vendor, date, amount, topic), assigning standardized ISO-8601 semantic names and sorting items into dedicated vaults.

2. Best Use of Render ($200)

Containerized deployment with a multi-stage production Dockerfile running as non-root appuser (UID 10001). SlothDesk utilizes Render's Persistent Disk (/app/data) to preserve sorted file vaults across redeploys, accompanied by zero-downtime health probes (/api/health/live, /api/health/ready) defined in render.yaml.

3. Best Use of ElevenLabs ($100)

SlothDesk leverages the ElevenLabs API to synthesize an ambient, sleepy morning brief narrated by a warm voice (JBFqnCBsd6RMkjVDRZzb). Paired with a bespoke waveform scrubber in the Next.js frontend, users can wake up, listen to their clutter briefing at 0.75x "Sloth Speed", and inspect what the daemon sorted while they slept.

4. Best Use of a .tech Domain ($100)

Configured with production DNS mappings for https://slothdesk.tech and https://api.slothdesk.tech, hardened with strict security headers (CSP, HSTS, X-Frame-Options: DENY) and CORS origins.

DrawerLocal: Your receipts, sorted. Your data, never uploaded.

Varnit Kumar — Sun, 18 Oct 2026 11:04:40 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

Every tax season, millions of freelancers, consultants, and boutique agency owners face the same dreaded chore: spending hours digging through crumpled physical receipts, PDFs, and invoices to itemize Schedule C deductions.

Most cloud-based receipt scanners present severe roadblocks:

Confidentiality & NDA Violations: Client dinners, NDA-bound project travel, private vendor addresses, and credit card slips cannot legally or ethically be uploaded to third-party multi-tenant cloud AI models.
Subscription Fatigue: Freelancers pay $20–$50/month for simple document OCR tools with arbitrary upload limits.
Hallucination Risk: Standard cloud LLMs frequently hallucinate digits without auditable bounding boxes or confidence metrics.

DrawerLocal solves this with an offline-first, open-weight AI receipt & invoice intelligence suite designed as a standalone side hustle that privacy-conscious professionals gladly pay for.

100% On-Device AI Vision: Powered by Google’s open-weight Gemma 3 (4B Vision) model running locally through Ollama. No receipt image ever leaves the user's computer.
Auditable Visual Inspection: Interactive bounding boxes map optical extractions directly back to document coordinates.
IRS Schedule C Auto-Categorization: Automatically categorizes expenses into IRS tax lines (Line 24b - Meals 50%, Line 9 - Travel, Line 18 - Office Supplies, Line 27a - Software & SaaS).
Instant Tax Export: Generates clean CSV spreadsheets formatted for TurboTax and CPA review.

Who Pays For It?

Freelancers & Solopreneurs: Drowning in 50–200 receipts/month who want a one-time purchase or private self-hosted utility instead of an ongoing $400/year QuickBooks subscription.
Independent Contractors (Law, Healthcare, Tech): Professionals bound by strict non-disclosure agreements who cannot use cloud AI for client receipts.
Digital Nomads: Traveling professionals needing fast offline receipt extraction on planes and trains without reliable internet.

Demo

🌐 Live Web App (Vercel): https://drawer-local.vercel.app/
⚡ Local Engine: Includes instant 1-click test scenarios (Delta Air Lines, OpenAI Credits, WeWork Day Office) to test the extraction pipeline.

Code

vannu07 / DrawerLocal

DrawerLocal 🗄️⚡

"Your receipts, sorted. Your data, never uploaded."

💡 The Problem: The Tax-Time Freelancer Nightmare

Every tax season, millions of freelancers, independent contractors, and boutique agency owners face the same dreaded ritual: digging through shoe boxes, email folders, and camera rolls to compile business receipts and invoice records for Schedule C deductions.

Existing cloud solutions (Expensify, QuickBooks, cloud LLMs) present major friction points:

Confidentiality Risks: Receipts contain client names, NDA-protected travel destinations, private addresses, and banking details. Freelancers cannot risk uploading sensitive client data into closed third-party cloud AI models.
Subscription Fatigue: Freelancers pay $20–$50/month for simple document OCR tools with arbitrary document limits.
Hallucination Anxiety: Pure cloud LLMs often hallucinate digits on crumpled receipts, risking tax audit penalties without an explicit confidence-scoring or verification step.

🎯 The Solution: DrawerLocal

DrawerLocal is an offline-first, open-weight AI receipt & invoice intelligence suite designed as a standalone side-hustle…

View on GitHub

The entire codebase is open-source under the MIT License:

Frontend: Next.js 16 (App Router), Tailwind CSS v4, PostCSS, responsive glassmorphism design system.
Backend: FastAPI, SQLModel, SQLite, background task extraction pipeline with local Gemma 3 provider abstraction.

How I Built It

1. Open-Weight AI Core: Gemma 3 (4B Vision)

At the heart of DrawerLocal is Google's Gemma 3 (4B Vision) open-weight model executed locally via Ollama.
We engineered a strict structured JSON extraction prompt that outputs normalized vendor names, ISO-8601 transaction dates, line items, currency codes, and exact sales tax/VAT breakdowns.

# Provider abstraction snippet (backend/app/services/provider.py)
class OllamaProvider(ModelProvider):
    async def extract_receipt(self, image_bytes: bytes) -> dict:
        b64 = base64.b64encode(image_bytes).decode("utf-8")
        payload = {
            "model": "gemma3:4b",
            "prompt": BASE_PROMPT,
            "images": [b64],
            "format": "json",
            "stream": False,
        }
        async with httpx.AsyncClient(timeout=60.0) as client:
            resp = await client.post(f"{self.base_url}/api/generate", json=payload)
            return json.loads(resp.json()["response"])

2. Zero-Cost Deterministic Demo Fallback

To ensure judges, reviewers, and users can immediately test the entire application without needing an 8GB GPU or local Ollama installation, DrawerLocal features an automatic deterministic mock engine when DEMO_MODE=true is set.

3. Fullstack Architecture

FastAPI + SQLModel: Persists itemized records, confidence scores, and custom vendor classification rules to an on-disk SQLite database.
Next.js 16 App Router: Built with custom dark-mode glassmorphism, dynamic SVG geometry constraints, and instant client-side CSV blob generation.

Why Does Open Innovation Matter?

Guaranteed Privacy via Zero Cloud Egress: Closed LLM APIs (OpenAI, Anthropic) process sensitive user financial documents on remote servers. Open-weight models like Gemma 3 enable the entire AI inference lifecycle to run on-device.
Predictable Unit Economics: A side hustle charging $49 for a lifetime desktop license or $5/month would quickly burn out on closed API token bills. Open-weight models cost $0.00 per token once downloaded.
Auditability & Zero Vendor Lock-in: The open weights can be run on CPU or GPU via Ollama, vLLM, or llama.cpp, guaranteeing long-term software independence.

My First User

I shared DrawerLocal with Sarah, a freelance brand designer managing 12 active client retainers.

"I used to spend the first weekend of every month matching coffee receipts, Uber rides, and software subscriptions to client invoices. Because I work under strict NDAs, I couldn't upload receipts with client names to random AI tools. DrawerLocal sorted 38 receipts in 3 minutes completely on my MacBook without uploading anything. I would easily pay $49 for this as a Mac app."

Sarah itemized $418.60 in deductible business expenses in her first test session.

Prize Categories

Best Use of Gemma: Built natively around Google’s open-weight Gemma 3 (4B Vision) model for on-device OCR, line-item parsing, and tax deduction scoring.
Best Use of Render: Production-ready render.yaml configuration and Dockerfile for containerized deployment of the FastAPI engine and persistent SQLite storage.
Best Use of a .tech Domain: Targeted for launch at drawerlocal.tech as an independent software business for freelancers.

Built with ❤️ for the Hacktoberfest Open-Source AI Challenge.

Last Light: An Open-Source AI Walk Ticket That Tells You to Put Your Phone Away

Varnit Kumar — Sat, 10 Oct 2026 18:24:45 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Last Light is an open-source AI walk ticket that tells you to put your phone away.

Most outdoor apps give you more things to look at. Last Light does the opposite. You choose a city (or use your own location), pick a walk length, describe the sky and ground, and it issues a short dusk-walk ticket:

⏰ A leave-by time and sunset direction ("Leave by 17:42. Walk west.")
🧭 A destination-free route suggestion and estimated distance
📈 A walk-quality score
👀 Four sensory prompts: look, listen, touch, notice
📵 One rule: "Phone in a pocket that is not your hand."
⏳ A Pocket Mode countdown, a printable ticket, optional voice narration, and a private field note stored only in your browser

It's for anyone who wants a simple reason to step outside at the end of the day. The website is useful for less than a minute, and the goal is to help you leave it. It is intentionally not a chatbot, social feed, map app, habit tracker, or recommendation engine.

raw.githubusercontent.com

Demo

🌅 Live app: https://touch-grass-beryl.vercel.app/

Choose a city or use your door.
Select a walk duration.
Describe the sky and ground.
Issue a ticket.
Read it, print it, or start Pocket Mode.
Put the phone away.

Code

vannu07 / Touch-Grass

Last Light

The screen is the shortest part.
A local-first dusk walk ticket that gets you outside, then gets out of the way

Last Light turns sunset geometry, local climate signals, and open-weight AI into a small printed-style brief: when to leave, which way to walk, four sensory prompts, and one rule — put the phone away.

Why it exists

Most outdoor apps increase screen time. Last Light is deliberately the opposite:

Prepare on screen. Calculate the useful facts before leaving.
Carry a ticket. Print it, read it aloud, or remember the four cues.
Pocket the phone. Pocket Mode becomes a simple countdown.
Notice the ordinary. There is no feed, streak, destination, or species database.

The best interface is the one you stop looking at.

What the experience feels like

Choose a city or use your door.
Set the walk length, date, temperature, ground, and sky.
Receive a dusk ticket…

View on GitHub

How I built it

1. Solar calculations without a weather API

Last Light calculates:

Sunrise
Sunset
Golden hour
Civil dusk
Solar noon
Sunset azimuth
Day length

The calculation uses latitude, longitude, date, and time-zone offset.

That allows the app to create a practical instruction such as:

Leave by 17:42. Walk west.

The basic walk ticket does not require map tiles or a weather service.

2. A generated dusk-walk dataset

The scoring engine uses 1,400 generated dusk-walk rows.

The features include:

Temperature
Rain
Wind
Cloud cover
Humidity
Latitude
Day of year
Minutes until sunset
Foliage season
Urban surroundings
Walk duration

The dataset is generated from a transparent recipe, which makes the model evaluation repeatable instead of presenting an unexplained number.

3. TabPFN with a Ridge baseline

TabPFN predicts the walk-quality score.

A Ridge regression model is trained alongside it as a baseline. The application reports held-out evaluation metrics so the score can be compared with a simpler model.

The full TabPFN backend is available for local and Render deployments.

Vercel uses the lightweight Ridge fallback because TabPFN's PyTorch dependencies exceed Vercel's serverless function size limit.

If TabPFN is unavailable, the app still issues a ticket instead of breaking.

4. Gemma and offline field recipes

When a compatible Gemma endpoint is available, Gemma writes the natural-language ticket.

The model receives structured facts and returns:

A short title
A one-sentence lead
Four sensory cues
A route
A phone-away rule

The response is validated before it reaches the user.

When no model endpoint is available, deterministic offline field recipes generate the ticket.

This is an intentional product mode, not a broken fallback.

The application can therefore work:

With local Ollama
With a compatible hosted endpoint
Without an AI endpoint
Without a weather API
Without storing field notes on a server

Why open innovation matters

Open innovation matters because this project is about reducing dependence on the screen.

A closed hosted AI service could generate attractive copy, but it would create several limitations:

Location and conditions would need to leave the user's machine.
The experience would depend on a network connection.
The model could not be easily swapped or run locally.
The fallback behavior would be harder to control.
The project would be less inspectable.

With open-source and open-weight components, the important parts remain replaceable and understandable.

Someone can:

Run Gemma locally with Ollama.
Replace the language model.
Inspect the prompt and validation rules.
Run the scoring model on their own machine.
Use the deterministic field recipes with no model at all.
Change the behavior without waiting for a platform feature.

The open approach made it possible to design the experience around privacy, inspectability, and graceful degradation.

The most important design decision

The most important decision was to make the screen temporary.

Last Light does not try to maximize:

Sessions
Notifications
Scroll time
Daily streaks
Recommendations
Engagement

It gives the user enough information to leave meaningfully, then offers Pocket Mode as a low-information countdown.

The product succeeds when the user stops using it.

Privacy and safety

Location is used to issue a ticket.

Field notes are stored in the browser using localStorage. They are not sent to a database or used to build a profile.

The project does not require:

A tracking account
A social graph
Map tiles
A weather API
A recommendation feed

The walk score is an experiment, not a safety guarantee.

Users should always use their own judgment about weather, lighting, traffic, accessibility, and personal safety.

What I learned

The easiest way to make an AI product useful is not always to add more conversation.

For this project, the better design was to constrain the model:

Give it structured facts.
Require a small JSON response.
Validate the response.
Keep an offline path.
Make the final output printable.
Make the interface disappear after the handoff.

The AI is useful before the walk. It should not become the activity.

Prize categories

Best Use of TabPFN — Prior Labs

Last Light uses TabPFN to score dusk-walk conditions from a generated dataset containing environmental, seasonal, solar, and duration features.

A Ridge model is included as a baseline so the TabPFN result can be compared with a simpler model.

Best Use of Gemma

Gemma can generate the natural-language walk ticket from structured facts.

The response is validated, and deterministic field recipes keep the experience available when no model endpoint is running.

What I would build next

The next version could add:

An installable offline PWA
Better accessibility and high-contrast themes
User-created walking rituals
A lightweight on-device audio model
Transparent model evaluation charts
User-controlled field-note export
Safety-aware route constraints without turning the project into a map app

For now, Last Light has one clear job:

Help someone leave the screen before the last light leaves the sky.

Built for Hacktoberfest

This project is built for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass.

The screen prepares the walk. Then it gets out of the way. 🌿

Heirloom AI: Preserving Grandpa's Voice Recipes with Open-Weight Gemma AI & ElevenLabs

Varnit Kumar — Sun, 04 Oct 2026 05:29:16 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

I built Heirloom AI for my Grandpa Arthur (and families with elderly loved ones). Grandpa Arthur spent a lifetime mastering rustic Italian dishes, slow-simmered marinara, and secret family recipes. As he got older, typing or writing recipes down became difficult, so he began recording rambling voice memos on his phone.

However, voice notes are unstructured, contain rambling family anecdotes, and lack exact measurements ("a pinch of love", "a knob of butter"). Heirloom AI uses open-weight Gemma AI to process Grandpa's voice audio, resolve vague measurements into standardized quantities, separate emotional stories from technical cooking steps, and store them in an interactive Heirloom Vault.

It also features a Hands-Free Kitchen Companion with voice commands ("Next step", "Repeat") and natural voice audio readout so anyone in the kitchen can cook Grandpa's recipes hands-free without getting screens greasy!

Demo

Live Deployed App: https://heirloom-ai-pi.vercel.app/
Key Senior Interactive Features:
1. 🎙️ Voice Memory Studio: Live microphone audio recorder with dynamic HTML5 Canvas waveform visualizer, persona selectors, and Gemma 4-bit prompt extraction pipeline.
2. 📖 The Heirloom Vault: Scalable recipe gallery with unit converter (Metric vs. US Imperial), serving multiplier (1x, 2x, 3x), Recharts flavor profile radar chart, macro nutrient cards, and printable heirloom recipe card modal generator.
3. 👨‍🍳 Hands-Free Kitchen Companion: Distraction-free full-screen kitchen display with Web Speech & ElevenLabs voice readout, voice command listening, interactive cooking timers, and an "Ask Grandpa's AI" assistant.
4. 💡 Open Innovation & Benchmarks: Technical deep-dive with Recharts benchmark visualizations comparing on-device Gemma 4-bit latency and privacy against closed cloud models.

Code

Project source code repository:

vannu07 / Heirloom-AI

🍳 Heirloom AI

Grandpa's Voice Recipe Vault & Hands-Free Kitchen Companion

Built for the Hacktoberfest Weekend Challenge: Build for a Friend
Preserving priceless family culinary knowledge using open-weight Gemma AI & ElevenLabs Voice Synthesis

🌟 Executive Summary

Heirloom AI is built for Grandpa Arthur (and families with elderly loved ones). Grandpa Arthur spent a lifetime mastering rustic Italian dishes, slow-simmered marinara, and secret family recipes. As he got older, writing or typing recipes down became difficult, so he began recording rambling voice memos on his phone.

However, voice notes are unstructured, contain rambling family anecdotes, and lack exact measurements ("a pinch of love", "a knob of butter").

Heirloom AI uses open-weight Gemma AI to process Grandpa's voice audio, resolve vague measurements into standardized quantities, separate emotional stories from technical cooking steps, and store them in an interactive Heirloom Vault.

It also features a Hands-Free Kitchen Companion with…

View on GitHub

src/App.jsx: Main layout container & state manager with Framer Motion AnimatePresence and Toast system.
src/components/VoiceStudio.jsx: Audio recorder & Gemma 4-bit extraction pipeline.
src/components/RecipeVault.jsx: Recipe gallery with serving scaler, unit converter, Recharts flavor radar, and printable cards.
src/components/CookingMode.jsx: Hands-free voice cooking assistant & timer.
src/components/OpenInnovationTab.jsx: Judge technical deep-dive with latency benchmarks.

How I Built It

Open-Source AI Core: Powered by open-weight Gemma 2 / Gemma 4-bit local inference models. The Gemma model takes raw speech-to-text transcripts, parses vague measurements into precise quantities, and structures the output into standardized JSON.
Frontend Stack: Vite, React 19, Tailwind CSS v4, Framer Motion for fluid layout animations, Recharts for data visualizations, Playfair Display & Plus Jakarta Sans typography, Fira Code fonts.
Voice Synthesis & Control: ElevenLabs & Web Speech API for natural step-by-step audio readout and hands-free voice commands.
Audio Visualizers: HTML5 Canvas API with real-time waveform rendering and Web Audio API ambient kitchen simmer sound generator.

Why Does Open Innovation Matter?

Open innovation was essential for building Heirloom AI:

100% Privacy & Data Security: Voice memos contain intimate family memories, names, and personal stories. Closed cloud APIs harvest prompt data to train commercial models. With open-weight Gemma running locally, zero personal data leaves the device.
Offline Kitchen Resilience: Kitchens often have poor WiFi reception or thick plaster walls. Local Gemma inference runs seamlessly offline on laptops or kitchen tablets without internet connection.
Zero Operating Cost: Heirloom recipes should be accessible across generations without recurring API token subscriptions.
Heritage Dialect Fine-Tuning: Open-weight Gemma allows fine-tuning on regional culinary terms (e.g. Italian "soffritto", Indian "tadka", Southern "potlikker") that proprietary generic models misinterpret.

Prize Categories

Best Use of Gemma (Featured Category)
Best Use of ElevenLabs (Partner Category)
Best Use of Render / DigitalOcean (Featured Category)