While building my ConversaAI web app, I started experimenting with a local AI model using Ollama, running the Gemma 3 (1B) model, a "lightweight" 815 MB model.
According to the documentation, a system with 8 GB RAM can handle models up to 7B parameters, so I expected a smooth run since my system has:
• CPU: Intel i7-8650U @ 1.9 GHz
• RAM: 16 GB
• OS: Windows 11 Pro
• GPU: NVIDIA MX130
But in practice… it’s a different story.
Every time model started generating a longer response,
• CPU jumped from 10 % to 60 %
• GPU (NVIDIA MX130) hit 95% utilization.
• Memory climbed to 8.5 GB / 16 GB
• Fans roared like a jet engine. 🌀
…and then the model simply stopped generating mid-response. No error. No crash message.
A few seconds later, the fan also slowed down again.
👇 Here's the before-and-during-generating-longer-response comparison of hardware utilization.
At first, I thought something broke.
But the real reason was far more interesting.
🧠 What Was Really Happening
Even at 1B parameters, the model performs billions of operations per token.
Every generated word triggers massive matrix multiplications. The GPU handles the heavy computation, while the CPU manages data flow, scheduling, and memory movement. Meanwhile, RAM and VRAM temporarily hold weights, activations, and intermediate states.
In simple terms:
• The GPU does intense math.
• The CPU coordinates everything.
• RAM/VRAM holds model weights and activations.
• The fan is the system’s way of saying, “I’m working really hard.”
When GPU utilization peaks (95%), it produces heat rapidly.
Once temperature crosses a safe limit, thermal throttling activates, a safety mechanism that slows performance to protect the hardware.
This throttling explains why the AI response halted and fan speed decreased shortly after.
📽️ Watch the video below that explains this situation.
Hardware strain during model inference
💡 Why This Is Normal
"Small" in AI doesn't mean "light" for consumer hardware.
A quick comparison:
• My GPU (MX130) -> ~0.4 TFLOPS
• Modern AI GPU (RTX 4090) -> ~82 TFLOPS
A gap of nearly 200x.
So, even though my laptop is capable of running local inference, it has trouble maintaining it, particularly when the model needs to continue executing billions of operations per second for longer outputs.
🔚 Final Thought
Running AI locally isn't just about generating text. It teaches you how your CPU, GPU, RAM, and thermal system manage the workload of modern AI models in real time.
Now, every time my laptop fan spins up, I know it's just trying to think really hard. 😅
Did you ever run into a similar issue? If yes, how did you tackle it?
If you found this post useful or learned something new, drop a ❤️ and share your thoughts in the comments. I'd love to hear your experience.
Feel free to reach out on
Top comments (8)
Hi, this looks like a great article but just letting you know those small bold mathematical symbols don't present well on all devices; e.g. some machines using system fonts just see the unicode missing glyph symbols.
Thanks for pointing that out! I didn’t realize some of the math symbols don’t render correctly on certain devices. I’ll update the post to use more compatible formatting. Appreciate you taking the time to mention it.
Thanks! Was able to read it now. I'd be curious if either 1) an even smaller model or 2) a quanitized model would work for you.
Yeah, I am planning to test both options.
1) Light model like phi-2, mistral:7b-q4
2) Quantized models such as gemma3:1b-q4_K_M
I totally get that! Even on mobile, just running heavier apps or games can push the limits of my device’s CPU and RAM. Can’t imagine how crazy it is with AI models!
Yes, absolutely. Thanks for sharing your thoughts.
Interesting read! It’s wild how demanding AI models are on regular hardware. Definitely learned something new here.
Thanks for appreciating it! Glad you find this useful.