The latest articles on DEV Community by Drishya Singhal (@drishya). https://dev.to/drishya https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4156763%2F1144cd29-00ce-462e-8b7b-851187544667.webp https://dev.to/drishya en Drishya Singhal Sun, 04 Oct 2026 19:42:23 +0000 https://dev.to/drishya/find-my-thingy-a-local-first-ai-memory-assistant-powered-by-gemma-3-4b-ifo https://dev.to/drishya/find-my-thingy-a-local-first-ai-memory-assistant-powered-by-gemma-3-4b-ifo <h1> Find My Thingy: A Local-First AI Memory Assistant Powered by Gemma 3 4B </h1> <p>Have you ever remembered saving something important in a document, but couldn't remember which file it was in?</p> <p>Maybe it was a piece of code, a project note, an explanation, or an idea you wrote down weeks ago.</p> <p>Searching through multiple files just to find one small detail can be frustrating.</p> <p>That's the problem behind <strong>Find My Thingy</strong>.</p> <h2> What is Find My Thingy? </h2> <p>Find My Thingy is a local-first personal memory assistant that helps you find information stored in your documents using natural-language questions.</p> <p>Instead of manually opening files and searching for keywords, you can add your documents to the application and ask questions about their contents.</p> <p>The application retrieves relevant passages and uses a locally running AI model to generate an answer with source citations.</p> <p><strong>The idea is simple: your information stays on your device, and you can ask questions about it whenever you need it.</strong></p> <h2> How does it work? </h2> <p>The workflow has four main steps:</p> <ol> <li> <strong>Add documents:</strong> Import PDFs, TXT files, and Markdown notes.</li> <li> <strong>Extract and index:</strong> The application extracts text, divides it into chunks, and creates embeddings.</li> <li> <strong>Retrieve relevant information:</strong> When you ask a question, the system searches for relevant document passages.</li> <li> <strong>Generate an answer:</strong> Gemma 3 4B, running through Ollama, generates an answer grounded in the retrieved information.</li> </ol> <p>Source citations help you trace an answer back to the document it came from.</p> <h2> Tech Stack </h2> <p>Here are the technologies used to build the project:</p> <ul> <li> <strong>Frontend:</strong> React, Vite, TypeScript, Tailwind CSS</li> <li> <strong>Backend:</strong> Python, FastAPI</li> <li> <strong>Database:</strong> SQLite</li> <li> <strong>Vector database:</strong> ChromaDB</li> <li> <strong>Embeddings:</strong> Sentence Transformers (<code>all-MiniLM-L6-v2</code>)</li> <li> <strong>Local AI:</strong> Gemma 3 4B through Ollama</li> <li> <strong>PDF processing:</strong> PyMuPDF</li> </ul> <h2> Why local-first? </h2> <p>I wanted the application to work without depending on a hosted AI API for everyday document retrieval and question answering.</p> <p>Running Gemma locally through Ollama means the application can process questions on the user's own machine once the required models and dependencies are installed.</p> <p>There is an initial setup requirement: the application needs its dependencies and models downloaded before it can operate fully offline.</p> <h2> Key features </h2> <ul> <li>Import PDFs, TXT files, and Markdown documents.</li> <li>Search document contents using natural-language questions.</li> <li>Generate answers grounded in retrieved passages.</li> <li>Display source citations.</li> <li>Store personal memories separately from document content.</li> <li>Use a locally running language model.</li> <li>Manage documents and saved memories through a dashboard.</li> </ul> <h2> Challenges during development </h2> <p>Building a local AI application involves more than connecting a model to a chat interface.</p> <p>Some of the important challenges include:</p> <ul> <li>Extracting useful text from different document formats.</li> <li>Creating meaningful chunks for retrieval.</li> <li>Avoiding answers when the available context does not support them.</li> <li>Connecting the frontend, backend, vector database, and local model.</li> <li>Managing model availability and local dependencies.</li> <li>Making the setup process understandable for Windows users.</li> </ul> <p>One important lesson was that a convincing AI response is not enough. The application also needs to show where the information came from and avoid making unsupported claims.</p> <h2> Current limitations </h2> <p>This is still a work in progress. Some limitations include:</p> <ul> <li>Scanned PDFs are not currently processed using OCR.</li> <li>Indexing is synchronous.</li> <li>Chat history is session-only.</li> <li>Memory extraction is best-effort.</li> <li>The application is designed for a trusted local device rather than multi-user deployment.</li> </ul> <h2> What's next? </h2> <p>Some areas I would like to improve:</p> <ul> <li>Better semantic retrieval for personal memories.</li> <li>More robust document processing.</li> <li>A smoother one-click Windows installation and launch experience.</li> <li>Improved retrieval evaluation and testing.</li> <li>Better support for larger document libraries.</li> </ul> <h2> Try it out </h2> <p><strong>GitHub Repository:</strong> <a href="https://github.com/Drishya-code/Find-My-Thingy" rel="noopener noreferrer">https://github.com/Drishya-code/Find-My-Thingy</a></p> <p>The repository contains the source code and setup instructions.</p> <p>If you explore the project, I'd especially appreciate feedback on the retrieval quality, local setup experience, and ways to make the assistant more useful.</p> <p><strong>The goal of Find My Thingy is straightforward: stop searching through everything you saved and start asking for what you remember.</strong></p> <p>Thanks for reading!<br> <a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwfsmkxsg18jw5r6ynwb8.png" class="article-body-image-wrapper"><img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwfsmkxsg18jw5r6ynwb8.png" alt=" " width="800" height="427"></a></p> devchallenge weekendchallenge ai hf26challenge