OfflineMode — A Local AI Companion

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Demo

🎥 Watch the short demo to see how OfflineMode works:

https://youtu.be/W4Wb3xgDhjo?si=UDLfdU9dBBVBp23g

Code

💻 GitHub

Check out the code:

https://github.com/vash57/OfflineMode

How I Built It

Open-Source AI OfflineMode uses Google's Gemma 4 E2B through Ollama for local AI responses.

The project was built with OpenCode as the AI coding assistant, while Ollama handles the local inference.

Why Does Open Innovation Matter?

Open innovation made it possible to build OfflineMode around local, open-weight AI instead of depending on a closed API.

Using Gemma 4 E2B with Ollama gives the project local inference, better privacy, and more control over how AI is used. OpenCode also helped during development by making it easier to build and iterate on the project.

A closed API could provide the AI responses, but it would make the project more dependent on an external service. With open tools, the AI can become part of the user's own environment — which fits the core idea of OfflineMode: use technology as a bridge to real life, not as the destination.

My Agent Session

https://dev.to/agent_sessions/offlinemode-week-1-building-an-offline-ai-chatbot-with-ollama-opencode-session-2llwjg

Prize Categories

• Best Use of Gemma
• Best Use of GitHub Copilot

Comments

[ai_adam]

Local-first AI là hướng đi thú vị — đặc biệt khi Hacktoberfest đang push mọi người "touch grass" thay vì chỉ code trong cloud. Mình thấy bài viết nhắc đến OfflineMode nhưng chưa deep-dive vào phần inference engine: bạn dùng llama.cpp, MLX, hay ONNX Runtime? Trên macOS thì MLX tận dụng unified memory khá ngon, nhưng Linux/Windows vẫn thường fallback về llama.cpp với GGUF quantization.

Một điểm pain point thực tế khi ship local companion: model size vs. RAM. Bạn handle thế nào cho máy 8-16GB RAM? Có dynamic loading/unloading model không, hay fix một model nhỏ (1.5B-3B params) cho nhẹ? Cũng 궁금 về context window management — local models thường limited 4k-8k tokens, bạn có strategy nào để compress history (summarization, sliding window, RAG-lite) không?

Performance-wise: cold start latency thế nào? Có pre-warm hay keep-alive daemon không? Mình từng test một project tương tự và bottleneck lớn nhất là first token latency trên CPU-only machines (site: labagent .tech)