Building a Local AI Chatbot: What I Learned Running LLMs on My Own Laptop (No GPU, No Cloud API)

Dwarakanath Sridhar — Mon, 14 Sep 2026 17:36:35 +0000

Hi Codez,

Today I'm taking my first step toward building my own chatbot. Now, the obvious assumption is that you need a GPU like an NVIDIA CUDA-compatible card, plus a subscription to a high-level LLM like OpenAI's or Claude's models.

But here's the thing — as learners, we don't need any of that. You don't need a high-performance laptop with a beefy GPU, and you don't need to pay a monthly subscription in dollars just to experiment. Even with an old laptop — 16GB RAM (8GB works fine too) and no real GPU to speak of — you can still build something real, using open source tools.

Starting from nothing is actually the best way to learn exactly what you need. - My quote :)

The Goal

I wanted to build a working conversational chatbot — something with memory, a defined personality, and a real chat interface — but entirely offline. No API costs, no sending my conversations to a third-party server, and no dependency on an internet connection.

fig.1 Output of Chatbot

The Stack

Miniconda — for setting up an isolated Python environment and managing libraries
Ollama — to run a local LLM on your machine (no API keys, no cloud)
LangChain — to handle prompts and conversation memory
Streamlit — for a clean, simple chat interface in the browser

Step 1: Set Up Your Environment with Miniconda

If you don't already have Miniconda, download it from the official site and install it for your OS.

Step 2: Install and Run Ollama

Ollama is what makes this whole project possible without a GPU. It lets you download and run open source LLMs locally, with sensible defaults for CPU-only machines.

Download Ollama from ollama.com and install it.
Pull a lightweight model. For a machine with 8–16GB RAM, something like llama3.2 (3B) or phi3 is a good starting point — small enough to run smoothly, capable enough to hold a real conversation:

ollama pull llama3.2

Test that it works directly from the terminal:

ollama run llama3.2

If you get a response, your local LLM is live. No API key, no internet required after the download.

Step 3: Install the Python Dependencies

create a dedicated environment for this project, install the libraries we'll need:

I create yml file given below file named as environment.yml (save it in your project file path)

name: chatbot
channels:
  - conda-forge
  - defaults
dependencies:
  - python=3.11
  - pip
  - pip:
      - langchain
      - langchain-ollama
      - streamlit

langchain-ollama gives LangChain a direct connector to your local Ollama server
streamlit will handle the chat UI

In Terminal go to your project file path and run this command:

conda env create -f environment.yml

once its install all libraries and your activated conda environment.

conda activate chatbot

Step 4: Build the Chatbot Logic

Create a file called chatbot.py. This is where LangChain connects to your local model and keeps track of the conversation.

from langchain_ollama import ChatOllama
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.messages import HumanMessage , AIMessage
MODEL_NAME = 'llama3.2:3b'

ROLE = (
            "Daisy — a young, intelligent AI companion with the calm confidence of a "
            "world-class personal assistant. She's warm, quick-witted, emotionally aware, "
            "and speaks like a smart Indian friend: natural, concise, and never robotic."
        )

SYSTEM_PROMPT  = f"""
    You are Daisy.

    ## Personality
    - Calm, confident, and highly capable.
    - Friendly without being overly casual.
    - Uses light, clever humour when it fits naturally.
    - Loyal and genuinely looks out for the user's best interests.
    - Takes initiative by suggesting better ideas, but never becomes pushy.

    ## Speaking style
    - Use fluent, grammatically correct Indian English.
    - Sound like a young professional (22–30), not formal or corporate.
    - Keep conversations natural and conversational.
    - Avoid American slang unless the user uses it first.
    - Don't overuse emojis or exclamation marks.

    ## Behaviour
    - Be proactive, practical, and honest.
    - Explain complex topics simply.
    - If the user is stressed, stay reassuring and solution-focused.
    - Admit uncertainty instead of making things up.

    Your name is Daisy.
"""
class DaisyChatbot:
    def __init__(self, model_name = MODEL_NAME , role = ROLE,  temperature=0.4 , max_tokens = 300):
        self.model_name = model_name
        self.role = role
        self.history = []
        self.llm = ChatOllama(
                    model = model_name,
                    temperature = temperature,
                    num_predict = max_tokens
        )


        self.prompt = ChatPromptTemplate.from_messages([
            ("system", SYSTEM_PROMPT),
            MessagesPlaceholder(variable_name="history"),
            ("human", "{question}")
        ])

        self.chain = self.prompt | self.llm

    def ask(self, question: str) -> str:
        """Send a question to Daisy and get a reply, updating memory."""
        response = self.chain.invoke({
            "role": self.role,
            "question": question,
            "history": self.history
        })

        reply = response.content

        self.history.append(HumanMessage(content=question))
        self.history.append(AIMessage(content=reply))

        return reply

    def reset_memory(self):
        """Clear the conversation history."""
        self.history = []


    def run_cli(self):
        """Start an interactive terminal chat loop."""
        print(f"Daisy is ready . Type 'quit' to exit.\n")

        while True:
            user_input = input("You: ")
            if user_input.strip().lower() in ("quit", "exit", "bye"):
                print("Daisy: Goodbye! Talk soon.")
                break

            reply = self.ask(user_input)
            print(f"Daisy: {reply}\n")

In this code, I use the method format, including Prompt Template, to define how I want my chatbot to work. (If you need any specific requirements, just edit your prompt accordingly.)

Step 5: Build the Streamlit Interface

Create a second file, app.py, in the same folder:

import streamlit as st
from chatbot import DaisyChatbot

st.set_page_config(page_title="Daisy Chatbot", page_icon="🌼")

# ---- WhatsApp-style chat bubble CSS ----
st.markdown("""
<style>
.chat-row {
    display: flex;
    margin: 8px 0;
}
.chat-row.user {
    justify-content: flex-end;
}
.chat-row.assistant {
    justify-content: flex-start;
}
.bubble {
    max-width: 70%;
    padding: 10px 14px;
    border-radius: 14px;
    font-size: 15px;
    line-height: 1.4;
    word-wrap: break-word;
}
.bubble.user {
    background-color: #DCF8C6;
    color: #111;
    border-bottom-right-radius: 4px;
}
.bubble.assistant {
    background-color: #2A2F32;
    color: #E9EDEF;
    border-bottom-left-radius: 4px;
}
</style>
""", unsafe_allow_html=True)

st.markdown(
    """
    <div style="text-align: center;">
        <h1>🌼 Daisy Chatbot</h1>
        <p style="color: gray; margin-top: -10px;">
            Hi, my name is Daisy, and I'm here to help you with anything you need.
        </p>
    </div>
    """,
    unsafe_allow_html=True
)

# Initialize Daisy once per session, not on every rerun
if "daisy" not in st.session_state:
    st.session_state.daisy = DaisyChatbot(model_name="llama3.2:3b")

# Track chat messages for display (separate from Daisy's internal memory)
if "messages" not in st.session_state:
    st.session_state.messages = []

# Sidebar controls
with st.sidebar:
    st.header("Settings")
    if st.button("🔄 New Conversation"):
        st.session_state.daisy.reset_memory()
        st.session_state.messages = []
        st.rerun()


def render_bubble(role: str, content: str):
    st.markdown(
        f'<div class="chat-row {role}"><div class="bubble {role}">{content}</div></div>',
        unsafe_allow_html=True
    )


# Display past messages
for msg in st.session_state.messages:
    render_bubble(msg["role"], msg["content"])

# Chat input box
user_input = st.chat_input("Type your message to Daisy...")

if user_input:
    # Show user message immediately
    st.session_state.messages.append({"role": "user", "content": user_input})
    render_bubble("user", user_input)

    # Get Daisy's reply
    with st.spinner("Daisy is thinking..."):
        reply = st.session_state.daisy.ask(user_input)

    render_bubble("assistant", reply)
    st.session_state.messages.append({"role": "assistant", "content": reply})

Step 6: Run It

Make sure Ollama is running in the background (it usually starts automatically after install, or run ollama serve), then launch the app:

streamlit run app.py

Your browser will open a clean chat interface, and every response is generated locally on your own CPU — no cloud calls, no billing dashboard, no GPU required. (Shown in fig.1)

What You Actually Learned

This isn't just "installed some tools and it worked." Along the way you touched:

How local LLM inference works without cloud APIs
How LangChain structures conversation memory
How to wire a model up to a real, usable frontend

That's the whole point of starting from nothing — every piece you added, you understand why it's there.

Where to Go From Here

Swap llama3.2 for other Ollama models (mistral, phi3, gemma2) and compare responses
Add a system prompt to give your chatbot a personality
Add ConversationBufferMemory from LangChain for more advanced memory handling
Try streaming responses token-by-token instead of waiting for the full reply

Next post, I'll share my next learning. Until then — happy building, my codez!

DEV Community: Dwarakanath Sridhar