DEV Community

Abiodun Paul Ogunnaike
Abiodun Paul Ogunnaike

Posted on

Built a Fully Offline AI Assistant on My Mac (Using Local LLMs)

#ai #llm #node #programming

Most AI apps today rely on APIs.

I wanted something different:

  • A fully offline AI assistant
  • Runs locally on my Mac
  • No API keys, no internet required

So I built one.

What I Built

A local AI assistant that:

  • Runs using a small language model (LLaMA / Mistral)
  • Works entirely offline
  • Supports chat + memory
  • Can be extended into a RAG system

Architecture 

[CLI / Terminal] 
      ↓ 
[Node.js App] 
      ↓ 
[Ollama (Local AI Engine)] 
      ↓ 
[Local LLM]
Enter fullscreen mode Exit fullscreen mode

Step 1 — Install Ollama on your local Machine 

brew install ollama
Enter fullscreen mode Exit fullscreen mode

Start the server:

ollama serve
Enter fullscreen mode Exit fullscreen mode

Pull a model:

ollama run qwen2.5:latest
Enter fullscreen mode Exit fullscreen mode

I choose qwen:2.5:latest because its just 4.7GB in size

Step 2 — Build CLI Assistant 

mkdir local-ai-assistant 
cd local-ai-assistant 
npm init -y 
npm install axios readline-sync
Enter fullscreen mode Exit fullscreen mode

Create a file in the local-ai-assistant folder

assistant.js

!/usr/bin/env node

const axios = require('axios');
const readline = require('readline-sync');

let history = [];

async function chat() {
    while (true) {
        const input = readline.question("You: ");

        history.push({ role: "user", content: input });

        const response = await axios.post('http://localhost:11434/api/chat', {
            model: 'qwen2.5:latest',
            messages: history,
            stream: false
        });

        const reply = response.data.message?.content || "No response";

        history.push({ role: "assistant", content: reply });

        console.log("AI:", reply);
    }
}

chat();
Enter fullscreen mode Exit fullscreen mode

Step 3 — Make It a Global CLI Tool 

Update package.json:

{
  "name": "local-ai",
  "bin": {
    "local-ai": "./assistant.js"
  }
}
Enter fullscreen mode Exit fullscreen mode

Run:

npm link
Enter fullscreen mode Exit fullscreen mode

Now you can run:

local-ai
Enter fullscreen mode Exit fullscreen mode

What I Learned

  • Local LLMs are powerful enough for real use cases
  • You don’t need Python to start — Node works fine
  • The real value is in architecture, not just models

What i intend to build next

  • Add file-based memory (RAG)
  • Add tools (agent behavior)
  • Connect to Laravel API
  • Deploy via Docker + AWS

Final Thought

Running AI locally changes how you think about systems.

You’re no longer dependent on external APIs — you own the stack.

If you’re building something similar, I’d love to connect

https://github.com/abbeymaniak
https://linkedin.com/in/abiodun-paul-ogunnaike

Top comments (0)