DEV Community

KAMAL KISHOR
KAMAL KISHOR

Posted on

Smollm3 + Ollama: Running a Local LLM That Feels Like Magic 🚀

#webdev #programming #ai #javascript

Large Language Models (LLMs) are transforming how we code, write, and interact with machines. But many devs still think LLMs are only for big players like OpenAI or Anthropic. That’s not true anymore.

With Smollm3 (a super lightweight LLM) + Ollama (a local runtime for LLMs), you can run AI models on your own laptop—no API bills, no vendor lock-in, no internet dependency.

Let’s explore how to do this with real-world examples that go beyond just saying “it works.”

⚡ Why Smollm3?

  • Tiny but powerful – designed for local use
  • Fast – runs even on consumer laptops
  • Privacy-first – no data leaves your machine
  • Flexible – you can fine-tune or extend for your needs

Perfect for devs who want hands-on AI without GPU farms.

🛠️ Step 1: Install Ollama

Ollama is like Docker, but for AI models.

curl -fsSL https://ollama.com/install.sh | sh
Enter fullscreen mode Exit fullscreen mode

Once installed, test with:

ollama run llama3.2
Enter fullscreen mode Exit fullscreen mode

Boom—you’ve got a working local LLM.

🧠 Step 2: Pull Smollm3 Model

Smollm3 is small enough for everyday use.

ollama pull smollm3
ollama run smollm3
Enter fullscreen mode Exit fullscreen mode

Now you’re chatting with a local AI model. 🎉

💻 Real-Life Examples

Here’s where it gets interesting—running Smollm3 for everyday dev and life tasks.

1. 🚀 As a Coding Assistant 

ollama run smollm3
Enter fullscreen mode Exit fullscreen mode

Prompt:

Write a Python script to monitor a folder and print any new files in real time.
Enter fullscreen mode Exit fullscreen mode

Output:

import os
import time

folder = "./watch_folder"
seen = set(os.listdir(folder))

while True:
    current = set(os.listdir(folder))
    new_files = current - seen
    if new_files:
        for f in new_files:
            print(f"New file detected: {f}")
    seen = current
    time.sleep(2)
Enter fullscreen mode Exit fullscreen mode

✅ Run instantly on your machine. No cloud latency.

2. 📝 Summarizing Research Papers Offline

Let’s say you downloaded a PDF from arXiv.

Prompt:

Summarize this research paper in 5 bullet points: [paste abstract]
Enter fullscreen mode Exit fullscreen mode

Smollm3 gives a digestible version—no internet required.

3. 🛍️ Personal Shopping Assistant

Imagine you copy-paste Amazon product descriptions.

Prompt:

Compare these 3 headphones and tell me which is best for bass lovers.
Enter fullscreen mode Exit fullscreen mode

Smollm3 instantly gives pros/cons breakdowns. Perfect offline shopping buddy.

4. 📅 Meeting Notes Summarizer

Paste your Zoom transcript into Smollm3:

Prompt:

Summarize key decisions and action items from this transcript.
Enter fullscreen mode Exit fullscreen mode

Now you’ve got meeting minutes—no Notion AI subscription needed.

5. 📚 Learning Aid

Students can run:

Prompt:

Explain quantum entanglement as if I’m 10 years old.
Enter fullscreen mode Exit fullscreen mode

Or even:

Generate 10 practice questions for Python list comprehensions.
Enter fullscreen mode Exit fullscreen mode

6. 🛡️ Privacy-Preserving Journal

If you keep a private journal:

Prompt:

Rewrite this journal entry in a positive, motivating way: [paste text]
Enter fullscreen mode Exit fullscreen mode

No servers. 100% private.

🧑‍💻 Step 3: Build Custom Workflows

With Ollama, you can integrate Smollm3 into apps:

Example: Local API Server 

ollama serve
Enter fullscreen mode Exit fullscreen mode

Send requests with curl:

curl http://localhost:11434/api/generate -d '{
  "model": "smollm3",
  "prompt": "Write a haiku about DevOps"
}'
Enter fullscreen mode Exit fullscreen mode

✅ Local AI endpoint for your apps.

🔮 Final Thoughts

Running Smollm3 + Ollama makes AI feel:

  • Personal → no one else sees your data
  • Accessible → no expensive GPU cloud bills
  • Hackable → integrate into your apps, workflows, or scripts

LLMs don’t need to live in a datacenter anymore. They can live on your laptop, right beside VS Code, Chrome, or Spotify.

🔥 If you found this useful, drop a comment with how you’d use a local LLM—I might build a follow-up with custom workflows for developers, students, and everyday creators.

Top comments (0)