DEV Community

David Simões
David Simões

Posted on

I Built a Private AI Assistant That Runs 100% Locally — No Cloud, No Subscriptions

#opensource #ai #node #privacy

Every time I used ChatGPT or similar tools the same thought crossed my mind:

"Where is this conversation going? Who has access to it? What are they doing with my data?"

So I decided to build my own solution. Meet CrustAI 🦀 — a fully private, self-hosted AI assistant that runs entirely on your own machine.

No cloud. No subscriptions. No data leaving your computer. Ever.

🌐 Documentation | ⭐ GitHub

The Problem I Wanted to Solve

Most AI assistants have one fundamental issue — your data belongs to someone else.

When you ask ChatGPT something personal, that conversation is:

  • Stored on OpenAI's servers
  • Potentially used to train future models
  • Subject to their privacy policy (which can change)
  • Accessible in case of a data breach

I wanted an AI assistant that was genuinely mine — one that I could use daily through apps I already use (Telegram, WhatsApp) without worrying about privacy.

What is CrustAI?

CrustAI is a self-hosted AI assistant powered by Ollama that connects to your favorite messaging platforms and runs completely offline after setup.

Key features:

  • 🔒 100% Private — all processing happens on your hardware
  • 🧠 Local LLM — powered by Ollama (llama3.2, tinyllama, phi3, mistral...)
  • 📱 Multi-platform — Telegram, WhatsApp, Discord and Slack
  • 🧬 Long-term memory — remembers facts across conversations
  • 🗣️ Offline voice — speech-to-text and text-to-speech (pt-BR)
  • ⚡ REST API — built-in Fastify server for custom integrations
  • 🎭 Custom personality — fully configurable assistant behavior

Tech Stack

Technology Purpose
Node.js Runtime environment
Ollama Local LLM inference engine
node-telegram-bot-api Telegram integration
@whiskeysockets/baileys WhatsApp integration
discord.js Discord integration
@slack/bolt Slack integration
Fastify REST API server
sql.js Embedded SQLite for memory
yaml Configuration management

Architecture Overview

The architecture is straightforward — a central message handler receives messages from any platform and routes them through the same pipeline:

┌─────────────────────────────────────────────────────┐
│                     CrustAI Core                     │
│                                                      │
│  ┌──────────┐  ┌──────────┐  ┌──────────────────┐  │
│  │ Telegram │  │ Discord  │  │     WhatsApp     │  │
│  │  Adapter │  │  Adapter │  │      Adapter     │  │
│  └────┬─────┘  └────┬─────┘  └────────┬─────────┘  │
│       └──────────────┼──────────────────┘            │
│                      ▼                               │
│            ┌─────────────────┐                      │
│            │  Message Handler │                      │
│            └────────┬────────┘                      │
│         ┌───────────┼───────────┐                   │
│         ▼           ▼           ▼                   │
│    ┌─────────┐ ┌────────┐ ┌──────────┐             │
│    │  Ollama │ │ Memory │ │ REST API │             │
│    │  (LLM)  │ │  Store │ │  Server  │             │
│    └─────────┘ └────────┘ └──────────┘             │
└─────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

Each platform adapter is independent — you enable only what you need in config.yml.

Getting Started in 5 Steps

Prerequisites

Step 1 — Clone the repository 

git clone https://github.com/DaveSimoes/CrustAI.git
cd CrustAI
Enter fullscreen mode Exit fullscreen mode

Step 2 — Install dependencies 

npm install
Enter fullscreen mode Exit fullscreen mode

Step 3 — Start Ollama and pull a model 

# Start the Ollama server (keep this terminal open)
ollama serve

# In a new terminal — pull a lightweight model
ollama pull tinyllama   # 600MB — works on modest hardware
# or
ollama pull llama3.2    # 2GB — needs ~8GB RAM
Enter fullscreen mode Exit fullscreen mode

Step 4 — Configure the project 

cp config/config.example.yml config/config.yml
Enter fullscreen mode Exit fullscreen mode

Edit config/config.yml:

model: tinyllama
ollama_url: http://localhost:11434
language: pt-BR

telegram:
  enabled: true
  token: YOUR_BOT_TOKEN_HERE
  allowed_user_ids: []
Enter fullscreen mode Exit fullscreen mode

Step 5 — Launch CrustAI! 

npm start
Enter fullscreen mode Exit fullscreen mode

Expected output:

✓ Ollama connected     (tinyllama)
✓ Memory store ready   (./data/memory.db)
✓ Telegram ready
✓ REST API ready       (http://localhost:3000)

🦀 CrustAI is ready. Your shell awaits.
Enter fullscreen mode Exit fullscreen mode

Available Commands

Once running, use these commands directly in Telegram (or any connected platform):

Command Description
/ping Check if the bot is alive
/help Show all commands
/model Show which AI model is running
/remember <fact> Store a fact in long-term memory
/forget Erase all stored facts
/clear Clear conversation history

The Privacy-First Philosophy

CrustAI was built with privacy as its core principle, not an afterthought:

✅ All conversations processed locally — nothing leaves your hardware
✅ No calls to OpenAI, Anthropic, Google or any cloud AI service
✅ No telemetry or usage tracking
✅ Open source — audit every single line of code
✅ MIT license — use it however you want

Running on Modest Hardware

One concern I had was performance. I'm running CrustAI with tinyllama on a machine with limited RAM and it handles daily conversations well.

For basic Q&A and conversation, tinyllama is surprisingly capable. If you have more resources, llama3.2 or phi3 give significantly better results.

What's Next — Roadmap

  • [ ] Web UI dashboard
  • [ ] Image understanding (multimodal LLMs)
  • [ ] Plugin system for custom tools
  • [ ] Docker one-click deployment
  • [ ] Mobile app companion

Contribute!

CrustAI is open source and I'd love contributions from the community:

  • 🐛 Bug reports — open an issue
  • 💡 Feature ideas — let's discuss in issues
  • 🔧 Pull requests — always welcome
  • ⭐ Star the repo — helps others discover it!

GitHub: https://github.com/DaveSimoes/CrustAI
Documentation: https://documentcrustai.netlify.app

Final Thoughts

Building CrustAI taught me a lot about local LLM inference, multi-platform bot development, and the real meaning of privacy-first software.

If you're tired of sending your conversations to the cloud, give CrustAI a try. Your data deserves to stay yours.

Made with 🦀 and ❤️ by Dave Simoes

If you found this useful, consider starring the repo and sharing with others who care about privacy!

Top comments (1)

Collapse
 
zerocosttech profile image
ZeroCost Tech •

Really nice project. I went down a very similar path recently with Node.js + Ollama + WhatsApp, but used whatsapp-web.js instead of Baileys.

I like the adapter-based architecture here — especially keeping WhatsApp/Telegram/Discord behind the same message handler.

How are you handling WhatsApp session recovery after a restart or temporary disconnect? That was one of the trickier parts for me when trying to keep a local bot running continuously on Windows.