DEV Community

Arun Sai Veerisetty
Arun Sai Veerisetty

Posted on • Edited on

🚀 Chat With AI on Your Terminal – No API Key, No Limits, Just Python + Ollama

Why should I use a terminal-based AI chatbot when ChatGPT is just a browser tab away?

That’s a valid question — and this blog will give you a very good answer.

Imagine this:
• You’re working offline or with limited internet.
• You want to explore open-source models like LLaMA 3 without any API restrictions.
• You want full control over what happens under the hood.
• You want to avoid rate limits and costs associated with commercial APIs.

If any of that resonates, this project is for you.

💡 The Problem with Web-Based AI

As amazing as ChatGPT is, there are still a few limitations:
• ⚠️ API costs add up fast if you’re prototyping regularly
• 🔒 Privacy concerns — your data is processed in the cloud
• 🌐 Requires a constant internet connection
• 📈 Rate limits on free or developer-tier APIs
• 🧪 Difficult to experiment with custom models or prompts

Sometimes, all you need is a simple, local environment to build and test — without worrying about tokens, keys, or infrastructure.

🔥 The Solution: A Local ChatGPT-Style CLI Tool

This project lets you build and run your own ChatGPT-style chatbot in the terminal using:
• 🐍 Python
• 🤖 Ollama – an open-source local runtime for LLMs
• ✅ Free & offline usage after the initial setup

You’ll be chatting with a powerful AI model like LLaMA 3 without needing an OpenAI key or an internet connection.

🧠 What Is Ollama?

Ollama makes running large language models (LLMs) on your own device incredibly easy. It supports:
• ⚡ Running LLaMA 3, Mistral, Gemma, and more
• 🌐 Serving models via a local HTTP API
• 🖥️ Compatible with macOS, Windows, and Linux

With just one command, you can run models that used to require massive infrastructure.

ollama run llama3
Enter fullscreen mode Exit fullscreen mode

That’s it — it downloads and starts serving the model locally via http://localhost:11434.

🛠️ What We Built

We’ve built a cross-platform command-line chatbot with:
• 💬 Seamless terminal chat experience
• 🔁 Streaming responses like ChatGPT
• ⌛ A “thinking…” indicator for better UX
• 🧩 A pluggable design that can support any Ollama model
• 🚀 Designed to be beginner-friendly, open-source, and fast to set up

📦 GitHub Project

🧠 Repo: https://github.com/arunsaiv/ollama-cli-assistant

It’s open-source and beginner-friendly. You can clone it, run it, and tweak it.

🧰 Prerequisites

  1. Python 3.8+
  2. Ollama installed – Download Ollama
  3. A supported LLM installed (e.g. ollama run llama3)

That’s all!

🏃‍♂️ How to Run the Chatbot

Clone and install:

git clone https://github.com/arunsaiv/ollama-cli-assistant.git
cd ollama-cli-assistant
pip install -r requirements.txt
Enter fullscreen mode Exit fullscreen mode

make sure your model is running:

ollama run llama3
Enter fullscreen mode Exit fullscreen mode

The launch the assistant:

python main.py
Enter fullscreen mode Exit fullscreen mode

Start chatting!

🎥 What It Looks Like

You: What are embeddings?

💬 AI is thinking...

AI: Embeddings are numerical representations of data — such as words or sentences — that allow machines to understand and process language in a more meaningful way...
Enter fullscreen mode Exit fullscreen mode

The response is streamed word-by-word, just like ChatGPT.

🧠 Why This Matters

This isn’t just a neat little side project — it’s your gateway to:
• 🔍 Learning how LLMs work under the hood
• 🧪 Running experiments without worrying about billing
• 🧱 Building AI tools and agents on top of open-source models
• 💡 Understanding prompt engineering by tweaking things yourself
• 🛠️ Customizing it to support additional commands, context, or tools

🧭 What You Can Build Next

This CLI is just the beginning. From here, you can:
• Add conversation memory
• Enable different models via CLI arguments
• Use voice input/output (via speech_recognition or gTTS)
• Build desktop wrappers with tools like Tauri or Electron
• Integrate with your own documents (RAG-style)

🏁 Wrap-Up

This project was born out of curiosity — can we build a truly offline, fully functional chatbot using open tools?

Turns out: Yes, and it’s awesome.

This CLI might not replace ChatGPT for everyone, but it gives you freedom, control, and insight into how LLMs really work — all while avoiding cloud limits.

📣 Final Thoughts

If this project helped you learn something, feel free to:
• ⭐ Star the GitHub repo
• 🗨️ Leave a comment or question below
• 🔁 Share this with someone who loves open-source AI
• 📬 Follow for more hands-on projects!

Top comments (0)