This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass
Building a Local AI Assistant with Ollama
I've been experimenting with local LLMs and exploring how to build AI applications without depending entirely on hosted APIs.
For this challenge, I started building a Python-based RAG (Retrieval-Augmented Generation) pipeline using Ollama. This is an early-stage project, and I'm still exploring how to turn it into something people can use outside their screens.
What I Built
I built a basic RAG pipeline that processes documents, indexes their content, retrieves relevant passages, and uses a locally running language model to answer questions.
So far, I've implemented:
- Document processing for CSV and PDF files.
- Passage indexing for information retrieval.
- Retrieval-augmented question answering.
- Local LLM inference using Ollama.
I tested the pipeline with an exoplanet dataset containing 27,215 indexed passages. I was able to ask questions about the dataset and get answers based on the retrieved information.
This is currently a terminal-based experiment. I haven't built a frontend or completed an agent harness integration yet.
Connecting It to the Touch Grass Theme
My next goal is to explore how this local AI pipeline could support outdoor activities, such as helping people prepare for hikes, learn about nature, or access useful information when internet connectivity is limited.
These are planned features, not capabilities I've implemented yet.
Demo
Currently, I don't have a deployed demo or video demonstration.
I've tested the pipeline locally through my terminal, including indexing the dataset and querying it with Ollama.
I'll share a demonstration once I have a more complete version.
Code
The project is currently running locally on my Linux machine.
I haven't published a GitHub repository yet. I plan to organize the code and document the setup before making it available.
How I Built It
My current setup uses:
- Python: For document processing and application logic.
- Ollama: For running a language model locally.
- RAG: For retrieving relevant information from indexed documents before generating answers.
- CSV and PDF processing: For working with different document formats.
I wanted to understand how the different components fit together rather than simply sending prompts to a hosted AI API.
I'm also exploring open-source agent harnesses, although I haven't integrated one successfully yet.
Why Does Open Innovation Matter?
Open innovation gives developers the freedom to experiment, inspect implementations, change components, and build on existing work.
For a project like this, it means I can experiment with locally running models, choose different models, and explore how the retrieval pipeline behaves without tying the entire application to one hosted API.
Local inference can also help keep document contents on the user's machine, depending on how the application is designed. It gives me more control over the system and helps me learn how these applications work internally.
I'm still at the beginning of this journey, but I want to understand these technologies by building something myself.
My Agent Session
I haven't recorded an agent session yet.
Prize Categories
No partner categories selected yet.
I'll consider the relevant categories as the project develops.
This is a work in progress, and I'm sharing it at an early stage rather than waiting until everything is polished.
If you've built local AI applications or experimented with open-source agent harnesses, I'd appreciate your suggestions on what I should explore next.
Top comments (0)