LocalFind Gemma — AI-Powered Semantic Search and Chat for Your Local Files

Malik — Sat, 23 May 2026 19:29:07 +0000

This is a submission for the Gemma 4 Challenge: Build with Gemma 4

What I Built

LocalFind Gemma is a fully local, privacy-first semantic search engine for your own files — documents, images, and audio — powered by Gemma 4 running on Ollama.

Most search tools match filenames or keywords. LocalFind Gemma understands content:

Images indexed by what's in them — Gemma 4 captions every image at sync time so you can search "whiteboard with the system architecture diagram" or "receipt from the coffee shop" and actually find it.
Agent that reads images to answer questions — ask "how much does that invoice say?" and the agent finds the image, sends it to Gemma 4 vision, and gives you the number, not a file path.
Audio fully searchable — Whisper transcribes recordings at index time so you can search across hours of meetings by what was said.
Cross-lingual search — the nomic-embed-text-v2-moe embedding model supports ~100 languages in a shared vector space. Search in French, find English documents.

Supported file types: PDF, DOCX, TXT, MD, CSV, JPG, PNG, GIF, BMP, WEBP, MP3, WAV, FLAC, M4A.

Everything — Gemma 4, Whisper, the ChromaDB vector store — runs on your machine. No API keys, no cloud, no data leaving your device. There's also an optional Claude Desktop integration via MCP for files you're comfortable sharing with a third party.

Demo

Code

https://github.com/maliklovable1-spec/localfind-gemma

How I Used Gemma 4

Gemma 4 isn't just the chat model here — it's active at three distinct points in the pipeline:

1. Index time: captioning every image
When you sync a folder, each image is sent to Gemma 4 via Ollama's vision API. The caption is embedded and stored permanently in ChromaDB. Future searches use the stored caption; the model isn't called again unless you re-sync. This means fast search without repeated inference.

2. Agent reasoning and tool use
The conversational agent runs on gemma4:e4b (the recommended default). It decides when to search, what query to issue, and how to synthesise results into a direct answer rather than just returning file paths.

I chose e4b over e2b because it follows tool-use instructions more reliably — which matters a lot in an agentic loop where the model needs to decide between search, image reading, and response synthesis. e2b is also supported for users with less RAM (~12 GB vs 16 GB).

3. Live image reading
When the agent finds an image relevant to your question, it sends the image bytes directly to Ollama's native /api/chat API with your question as context. Gemma 4 reads the image and the agent uses that to answer you. The bytes go from your disk to your local Ollama process —nowhere else.

A note on audio
Gemma 4 E2B and E4B natively support audio transcription at the architecture level — multilingual, up to 30 seconds, built into the model. LocalFind Gemma currently uses Whisper for audio because
Ollama doesn't expose audio input via its API yet. Once Ollama ships that support
([issue #11798(https://github.com/ollama/ollama/issues/11798)), the transcription backend can
switch to Gemma 4 — the architecture is already designed with that transition in mind, though it will require some code changes depending on how Ollama exposes the audio API.

DEV Community: Malik

LocalFind Gemma — AI-Powered Semantic Search and Chat for Your Local Files

What I Built

Demo

Code

How I Used Gemma 4