Fine-tuning Qwen as my personal assistant

Lakshmi Shankar — Tue, 09 Dec 2025 11:27:29 +0000

I Built a Local “Remembering” AI Assistant Using Qwen 2.5 — In Under 50 Lines

I wanted my own tiny AI that runs locally, remembers things only when I tell it to, and doesn’t burn my GPU alive.
So I grabbed Qwen 2.5 (1.5B) and wrote this little assistant.

It talks.
It thinks.
It remembers on command.
And it fits in one Python file.

The Code (Yes, it’s this small)

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "Qwen/Qwen2.5-1.5B-Instruct"
tokenizeer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    dtype=torch.float16,
    device_map="auto",
    local_files_only=True,
)

system_prompt = "You are a helpful assistant that helps as a software developer."
current_prompt = [{"role": "system", "content": system_prompt}]
chat_history = []

while True:
    userInput = input("User:\n")
    current_prompt.append({"role": "user", "content": userInput})

    if userInput.lower() in ["bye", "exit", "quit"]:
        print("\nExiting model...")
        break

    if any(word in userInput.lower() for word in ["remember", "log"]):
        chat_history.append({"role": "user", "content": userInput})

    final_prompt = chat_history + current_prompt
    text = tokenizeer.apply_chat_template(final_prompt, tokenize=False, add_generation_prompt=True)
    inputs = tokenizeer(text, return_tensors="pt")

    outputs = model.generate(**inputs, max_new_tokens=180)
    response = tokenizeer.decode(outputs[0], skip_special_tokens=True)
    reply = response.split("assistant")[-1].strip()

    if "remember" in userInput.lower():
        chat_history.append({"role": "assistant", "content": reply})

    print("\nAssistant:\n" + reply + "\n")

What It Can Do

Talks like a normal chat model
Keeps track of conversation
Remembers things only when I say “remember” or “log”
Everything runs offline with Qwen 2.5

Example moment:

User: Remember my project name is SkillGapMatcherAi
Assistant: Noted! I’ll remember that.

Later…

User: What's my project name?
Assistant: You said your project name is Flexy.

Feels like magic, but it’s just Python + Qwen.

DEV Community: Lakshmi Shankar

Fine-tuning Qwen as my personal assistant

I Built a Local “Remembering” AI Assistant Using Qwen 2.5 — In Under 50 Lines

The Code (Yes, it’s this small)

What It Can Do