DEV Community

Martin Schenk
Martin Schenk

Posted on

Building an AI Chat Terminal That Routes Private Data to Local AI

#ai #privacy #opensource #terminal

I was tired of accidentally pasting API keys into ChatGPT, so I built a terminal that intelligently routes conversations.

The Problem

Every time you chat with ChatGPT, Claude, or any cloud AI:

  • Everything goes to their servers - passwords, API keys, emails, everything
  • It's all logged - no way to delete it
  • You can't verify what they do with your data

I kept cringing every time I accidentally pasted something sensitive.

The Solution

AI Chat Terminal - keyword detection that runs BEFORE anything touches the network:

You: "save my password SecretPass123"
→ Routes to LOCAL Qwen AI ✅
→ Encrypted SQLite (AES-256) ✅
→ NEVER sent to cloud ✅

You: "What's the capital of Spain?"
→ Routes to OpenAI ✅

You: "best food there?"
→ OpenAI understands context (Spain) ✅
Enter fullscreen mode Exit fullscreen mode

How It Works

1. Keyword Detection (<1ms)

Pattern matching for save, show, delete and 30+ synonyms:

if matches_keywords(input, ['save', 'note', 'record']):
    route_to_local()
else:
    route_to_cloud()
Enter fullscreen mode Exit fullscreen mode

Works in English, German, Spanish:

  • "save my email" → Local
  • "guarda mi contraseña" → Local
  • "speichere meine Telefonnummer" → Local

2. Local AI Processing

Uses Qwen 2.5 Coder (7B) via Ollama:

User: "save my email test@example.com"
→ Qwen generates SQL:
  INSERT INTO mydata (content, meta)
  VALUES ('test@example.com', 'email')
→ Encrypted with SQLCipher (AES-256)
Enter fullscreen mode Exit fullscreen mode

3. Cloud for Everything Else

OpenAI GPT-4o with full context:

You: "capital of France?"
AI: "Paris"

You: "best food there?"
AI: "Croissants..." (knows "there" = Paris)
Enter fullscreen mode Exit fullscreen mode

Tech Stack

  • Python 3.9+
  • Ollama - Local AI runtime
  • Qwen 2.5 Coder (7B) - SQL generation
  • SQLCipher - AES-256 encryption
  • OpenAI API - Cloud queries
  • ~5GB disk space for model

Installation (macOS)

One-line install:

curl -fsSL https://raw.githubusercontent.com/martinschenk/ai-chat-terminal/main/install.sh | zsh
Enter fullscreen mode Exit fullscreen mode

This automatically:

  1. Installs Ollama if needed
  2. Pulls Qwen 2.5 Coder model
  3. Sets up encrypted database
  4. Adds shell integration

Takes ~5 minutes (mostly downloading the 4.5GB model).

Usage Examples

Save Private Data 

$ chat
👤 You ▶ save my email test@example.com
🤖 AI    🗄️ Stored 🔒

👤 You ▶ save password SecretPass123
🤖 AI    🗄️ Stored 🔒

👤 You ▶ guarda mi dirección Calle Mayor 1
🤖 AI    🗄️ Guardado 🔒
Enter fullscreen mode Exit fullscreen mode

Retrieve Anywhere 

👤 You ▶ show my email
🤖 AI    🗄️🔍 test@example.com

👤 You ▶ list all
🤖 AI    🗄️🔍 Found 3 items:
           1. test@example.com (email)
           2. SecretPass123 (password)
           3. Calle Mayor 1 (dirección)
Enter fullscreen mode Exit fullscreen mode

General Questions (Cloud) 

👤 You ▶ capital of Spain?
🤖 AI    Madrid.

👤 You ▶ best food?
🤖 AI    Tapas. (remembers Spain!)
Enter fullscreen mode Exit fullscreen mode

Privacy Architecture 

┌─────────────────────────────────────────────┐
│ User Input                                  │
└────────────┬────────────────────────────────┘
             ↓
   ┌─────────────────────┐
   │ Keyword Detection   │  ← Fast (<1ms)
   │ save/show/delete    │
   └─────────┬───────────┘
             ↓
      ┌──────────────┐
      │  Detected?   │
      └──┬────────┬──┘
         │        │
    YES  │        │  NO
         ↓        ↓
  ┌──────────────┐  ┌──────────────┐
  │ Qwen 2.5     │  │   OpenAI     │
  │ Coder (7B)   │  │  GPT-4o      │
  │ SQL Direct   │  │  (Cloud)     │
  └─────┬────────┘  └──────┬───────┘
        ↓                   ↓
  ┌─────────────┐  ┌──────────────┐
  │ Encrypted   │  │  Response    │
  │ SQLite DB   │  │ with Context │
  │ (AES-256)   │  │              │
  └─────────────┘  └──────────────┘
Enter fullscreen mode Exit fullscreen mode

Key Features

🔒 Privacy First

  • Keyword detection before any network call
  • Local Qwen AI for sensitive data
  • AES-256 encrypted SQLite
  • Zero cloud exposure for private data

🧠 Smart Context

  • OpenAI maintains conversation history
  • Understands references ("there", "it", "that")
  • Natural follow-up questions

🌍 Multilingual

  • English, German, Spanish
  • 30+ action verbs per language
  • Flexible phrasing

⚡ Fast

  • Keyword matching: <1ms
  • Local operations: <1s
  • OpenAI queries: 5-7s
  • Always shows 🗄️ icon for local DB operations

System Requirements

  • macOS 12.0+ (Monterey or later)
  • Zsh shell
  • Python 3.9+
  • ~5GB disk space (for Qwen model)
  • 8GB RAM minimum (16GB recommended)
  • OpenAI API key

Why Qwen 2.5 Coder?

I tested several local models and Qwen 2.5 Coder was surprisingly good at:

  • Generating SQL from natural language
  • Understanding multilingual input
  • Pattern recognition for data extraction
  • Running efficiently on 7B parameters

Future Improvements

Looking for contributors! Especially for:

  • More languages (French, Italian, Portuguese)
  • Windows/Linux ports (currently macOS only)
  • Better error handling
  • Testing on different macOS versions

Open Source

Conclusion

This scratches an itch I had - wanting the power of GPT-4 without accidentally logging sensitive data. The keyword-based routing is simple but effective.

Would love feedback on the privacy model - I'm sure there are edge cases I haven't considered!

Links

Top comments (0)