This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine
What I Built
I built Ghost in the Machine, a locally running AI assistant that can remember useful information across conversations.
The idea was to explore how an AI assistant could maintain long-term memory without relying on a fully cloud-hosted AI service.
It uses Qwen 2.5 3B through Ollama for local inference, Mem0 for memory extraction and management, and ChromaDB for vector storage.
One challenge I faced was response latency. Memory extraction was happening during the chat request, making responses slow. I improved this by moving memory extraction into a background worker backed by a persistent SQLite queue, while keeping memory retrieval synchronous.
This allows the assistant to respond without waiting for memory extraction to finish.
Demo
Ghost in the Machine currently runs locally and is not publicly deployed.
Screenshots of the running application and its memory-retrieval functionality are included in this article.
Source Code: https://github.com/nithyanandharajms138-debug/ghost-in-the-machine
Code
nithyanandharajms138-debug
/
ghost-in-the-machine
A spooky terminal-style web app with an AI ghost.
👻 Ghost in the Machine
A spooky, retro terminal-style AI companion that lives inside your computer. Powered by locally running AI models, Ghost communicates through a mysterious terminal interface.
✨ Features
- Retro terminal-style interface built with HTML, CSS and JavaScript.
- Interactive AI conversation with a ghost character.
- Local AI responses using Ollama and Qwen2.5 3B.
- Python Flask backend.
- Mem0 integration for extracting and retrieving user memories.
- ChromaDB as the persistent vector store.
- Local-first architecture — no paid AI API key required.
🛠️ Tech Stack
| Component | Technology |
|---|---|
| Frontend | HTML, CSS, JavaScript |
| Backend | Python, Flask |
| AI Runtime | Ollama |
| Language Model | Qwen2.5 3B |
| Memory Framework | Mem0 |
| Vector Database | ChromaDB |
| Embedding Model | nomic-embed-text |
📋 Prerequisites
- Python
- Git
- Ollama
- Sufficient local resources to run the selected AI models
🚀 Run Locally
1. Install Ollama
Download Ollama from ollama.com.
2. Download the required models
ollama pull qwen2.5:3b
ollama pull nomic-embed-text:latest
3. Clone the repository
git…How I Built It
I built Ghost in the Machine using Python and open-source AI tools.
The main components are:
- Python and Flask: Used to build the backend and handle chat requests.
- Ollama with Qwen 2.5 3B: Runs the language model locally, allowing conversations without depending on a cloud-based LLM API.
- Mem0: Handles extracting and managing long-term memories from conversations.
- ChromaDB: Stores vector embeddings so relevant memories can be retrieved later.
- nomic-embed-text: Converts information into embeddings for semantic search.
- SQLite: Powers a persistent background queue for memory extraction jobs.
- HTML, CSS and JavaScript: Build the frontend chat interface.
One of the biggest challenges was response latency. Initially, memory extraction happened as part of the chat request, making the assistant slow.
I redesigned this workflow by introducing a background worker and a SQLite-backed queue. Messages are queued for memory extraction after a short period of chat inactivity. The worker processes jobs asynchronously, with retry and recovery mechanisms.
Memory retrieval remains synchronous, so the assistant can use previously stored information while generating responses.
This project helped me understand how local LLMs, vector databases, memory systems and background processing can work together to create a more capable AI assistant.
Why Does Open Innovation Matter?
Open innovation matters because it allows developers to learn from, build upon, and contribute to work created by the wider community.
Building Ghost in the Machine helped me understand the value of open-source tools such as Ollama, Mem0 and ChromaDB. Instead of building every component from scratch, I could combine existing technologies, study how they work, and focus on solving a real engineering challenge.
It also showed me that open source is more than using freely available code. It is about sharing knowledge, documenting ideas, learning from other developers, and making improvements that others can build upon.
As someone beginning my open-source journey, this project has been a meaningful opportunity to learn by building and contributing.
Prize Categories
- GitHub — Best Use of GitHub Copilot
Top comments (1)
done....