I Ran Gemma 4 on 10,000 Linux Security Logs Locally — Here's the Real Cost

I'm building SystemGuard, an open-source HIDS to replace $35/month CrowdStrike for freelancers. My biggest problem isn't detection — it's explaining alerts without sending logs to OpenAI.

When Google dropped Gemma 4 with a 128K context window, I tested it for one job: summarizing security events on-prem.

This is what actually worked, what didn't, and why I chose the 4B model.

Which Gemma 4 Model I Chose (And Why)

Gemma 4 comes in three flavors:

• 2B/4B — runs on Raspberry Pi / phone
• 31B Dense — server-grade
• 26B MoE — high-throughput reasoning

I picked Gemma 4 4B Instruct. Not 2B (too many hallucinations on Linux paths), not 31B (needs 24GB VRAM, my test server has 8GB), not MoE (I need low latency, not batch reasoning).

For a security agent that runs on a $20 VPS, 4B is the sweet spot: 3.5GB RAM, ~4 seconds per batch, Apache 2.0 license.

How I Set It Up (Free, Local, No API Keys)

bash
# 1. Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# 2. Pull Gemma 4
ollama pull gemma3:4b  # Ollama uses gemma3 tag for Gemma 4 family

# 3. Test
ollama run gemma3:4b "Explain this log: nginx uid=33 opened /etc/shadow"