Building a Local AI Chatbot: What I Learned Running LLMs on My Own Laptop (No GPU, No Cloud API)

Hi Codez,

Today I'm taking my first step toward building my own chatbot. Now, the obvious assumption is that you need a GPU like an NVIDIA CUDA-compatible card, plus a subscription to a high-level LLM like OpenAI's or Claude's models.

But here's the thing — as learners, we don't need any of that. You don't need a high-performance laptop with a beefy GPU, and you don't need to pay a monthly subscription in dollars just to experiment. Even with an old laptop — 16GB RAM (8GB works fine too) and no real GPU to speak of — you can still build something real, using open source tools.

Starting from nothing is actually the best way to learn exactly what you need. - My quote :)

The Goal

I wanted to build a working conversational chatbot — something with memory, a defined personality, and a real chat interface — but entirely offline. No API costs, no sending my conversations to a third-party server, and no dependency on an internet connection.

fig.1 Output of Chatbot

---

The Stack

• Miniconda — for setting up an isolated Python environment and managing libraries
• Ollama — to run a local LLM on your machine (no API keys, no cloud)
• LangChain — to handle prompts and conversation memory
• Streamlit — for a clean, simple chat interface in the browser

---

Step 1: Set Up Your Environment with Miniconda

If you don't already have Miniconda, download it from the official site and install it for your OS.

---

Step 2: Install and Run Ollama

Ollama is what makes this whole project possible without a GPU. It lets you download and run open source LLMs locally, with sensible defaults for CPU-only machines.

1. Download Ollama from ollama.com and install it.
2. Pull a lightweight model. For a machine with 8–16GB RAM, something like llama3.2 (3B) or phi3 is a good starting point — small enough to run smoothly, capable enough to hold a real conversation:

ollama pull llama3.2

1. Test that it works directly from the terminal:

ollama run llama3.2

If you get a response, your local LLM is live. No API key, no internet required after the download.

---

Step 3: Install the Python Dependencies

create a dedicated environment for this project, install the libraries we'll need:

I create yml file given below file named as environment.yml (save it in your project file path)

name: chatbot
channels:
  - conda-forge
  - defaults
dependencies:
  - python=3.11
  - pip
  - pip:
      - langchain
      - langchain-ollama
      - streamlit

• langchain-ollama gives LangChain a direct connector to your local Ollama server
• streamlit will handle the chat UI

In Terminal go to your project file path and run this command:

conda env create -f environment.yml

once its install all libraries and your activated conda environment.

conda activate chatbot

---

Step 4: Build the Chatbot Logic

Create a file called chatbot.py. This is where LangChain connects to your local model and keeps track of the conversation.

from langchain_ollama import ChatOllama
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.messages import HumanMessage , AIMessage
MODEL_NAME = 'llama3.2:3b'

ROLE = (
            "Daisy — a young, intelligent AI companion with the calm confidence of a "
            "world-class personal assistant. She's warm, quick-witted, emotionally aware, "
            "and speaks like a smart Indian friend: natural, concise, and never robotic."
        )

SYSTEM_PROMPT  = f"""
    You are Daisy.

    ## Personality
    - Calm, confident, and highly capable.
    - Friendly without being overly casual.
    - Uses light, clever humour when it fits naturally.
    - Loyal and genuinely looks out for the user's best interests.
    - Takes initiative by suggesting better ideas, but never becomes pushy.

    ## Speaking style
    - Use fluent, grammatically correct Indian English.
    - Sound like a young professional (22–30), not formal or corporate.
    - Keep conversations natural and conversational.
    - Avoid American slang unless the user uses it first.
    - Don't overuse emojis or exclamation marks.

    ## Behaviour
    - Be proactive, practical, and honest.
    - Explain complex topics simply.
    - If the user is stressed, stay reassuring and solution-focused.
    - Admit uncertainty instead of making things up.

    Your name is Daisy.
"""
class DaisyChatbot:
    def __init__(self, model_name = MODEL_NAME , role = ROLE,  temperature=0.4 , max_tokens = 300):
        self.model_name = model_name
        self.role = role
        self.history = []
        self.llm = ChatOllama(
                    model = model_name,
                    temperature = temperature,
                    num_predict = max_tokens
        )


        self.prompt = ChatPromptTemplate.from_messages([
            ("system", SYSTEM_PROMPT),
            MessagesPlaceholder(variable_name="history"),
            ("human", "{question}")
        ])

        self.chain = self.prompt | self.llm

    def ask(self, question: str) -> str:
        """Send a question to Daisy and get a reply, updating memory."""
        response = self.chain.invoke({
            "role": self.role,
            "question": question,
            "history": self.history
        })

        reply = response.content

        self.history.append(HumanMessage(content=question))
        self.history.append(AIMessage(content=reply))

        return reply

    def reset_memory(self):
        """Clear the conversation history."""
        self.history = []


    def run_cli(self):
        """Start an interactive terminal chat loop."""
        print(f"Daisy is ready . Type 'quit' to exit.\n")

        while True:
            user_input = input("You: ")
            if user_input.strip().lower() in ("quit", "exit", "bye"):
                print("Daisy: Goodbye! Talk soon.")
                break

            reply = self.ask(user_input)
            print(f"Daisy: {reply}\n")    

In this code, I use the method format, including Prompt Template, to define how I want my chatbot to work. (If you need any specific requirements, just edit your prompt accordingly.)

---

Step 5: Build the Streamlit Interface

Create a second file, app.py, in the same folder:

import streamlit as st
from chatbot import DaisyChatbot

st.set_page_config(page_title="Daisy Chatbot", page_icon="🌼")

# ---- WhatsApp-style chat bubble CSS ----
st.markdown("""
<style>
.chat-row {
    display: flex;
    margin: 8px 0;
}
.chat-row.user {
    justify-content: flex-end;
}
.chat-row.assistant {
    justify-content: flex-start;
}
.bubble {
    max-width: 70%;
    padding: 10px 14px;
    border-radius: 14px;
    font-size: 15px;
    line-height: 1.4;
    word-wrap: break-word;
}
.bubble.user {
    background-color: #DCF8C6;
    color: #111;
    border-bottom-right-radius: 4px;
}
.bubble.assistant {
    background-color: #2A2F32;
    color: #E9EDEF;
    border-bottom-left-radius: 4px;
}
</style>
""", unsafe_allow_html=True)

st.markdown(
    """
    <div style="text-align: center;">
        <h1>🌼 Daisy Chatbot</h1>
        <p style="color: gray; margin-top: -10px;">
            Hi, my name is Daisy, and I'm here to help you with anything you need.
        </p>
    </div>
    """,
    unsafe_allow_html=True
)

# Initialize Daisy once per session, not on every rerun
if "daisy" not in st.session_state:
    st.session_state.daisy = DaisyChatbot(model_name="llama3.2:3b")

# Track chat messages for display (separate from Daisy's internal memory)
if "messages" not in st.session_state:
    st.session_state.messages = []

# Sidebar controls
with st.sidebar:
    st.header("Settings")
    if st.button("🔄 New Conversation"):
        st.session_state.daisy.reset_memory()
        st.session_state.messages = []
        st.rerun()


def render_bubble(role: str, content: str):
    st.markdown(
        f'<div class="chat-row {role}"><div class="bubble {role}">{content}</div></div>',
        unsafe_allow_html=True
    )


# Display past messages
for msg in st.session_state.messages:
    render_bubble(msg["role"], msg["content"])

# Chat input box
user_input = st.chat_input("Type your message to Daisy...")

if user_input:
    # Show user message immediately
    st.session_state.messages.append({"role": "user", "content": user_input})
    render_bubble("user", user_input)

    # Get Daisy's reply
    with st.spinner("Daisy is thinking..."):
        reply = st.session_state.daisy.ask(user_input)

    render_bubble("assistant", reply)
    st.session_state.messages.append({"role": "assistant", "content": reply})

---

Step 6: Run It

Make sure Ollama is running in the background (it usually starts automatically after install, or run ollama serve), then launch the app:

streamlit run app.py

Your browser will open a clean chat interface, and every response is generated locally on your own CPU — no cloud calls, no billing dashboard, no GPU required. (Shown in fig.1)

---

What You Actually Learned

This isn't just "installed some tools and it worked." Along the way you touched:

• How local LLM inference works without cloud APIs
• How LangChain structures conversation memory
• How to wire a model up to a real, usable frontend

That's the whole point of starting from nothing — every piece you added, you understand why it's there.

---

Where to Go From Here

• Swap llama3.2 for other Ollama models (mistral, phi3, gemma2) and compare responses
• Add a system prompt to give your chatbot a personality
• Add ConversationBufferMemory from LangChain for more advanced memory handling
• Try streaming responses token-by-token instead of waiting for the full reply

Next post, I'll share my next learning. Until then — happy building, my codez!