I built a QEMU sandbox to test if a 4B model (Gemma) can act as an autonomous Linux sysadmin — here are the results (2/3 pass rate)

What happens when you give an open-weights 4B parameter language model root access to a broken Linux server and tell it to fix the problem?

I built local-agent-sandbox, a lightweight evaluation harness running on pure QEMU and llama.cpp. Here is how the experiment was structured, how the sandboxing works, and how the model performed.

The Architecture: Why Not Docker?

Most LLM agent sandboxes default to Docker. While containers are great for application packaging, they are not security boundaries:

1. The container shares the host Linux kernel.
2. If an agent runs with root privileges or encounters prompt injection executing kernel exploits, container escape is a real concern.
3. If an agent accidentally breaks networking or DNS inside the container, external API communication dies.

Instead, I used pure QEMU with KVM and QCOW2 Copy-On-Write Overlays:

• A base image (ubuntu-base.qcow2) contains a clean headless Ubuntu 24.04 install.
• Before each test, the script creates a 100KB overlay file: qemu-img create -f qcow2 -b ubuntu-base.qcow2 -F qcow2 sandbox.qcow2
• The VM boots off the overlay in ~6 seconds.
• When the test concludes, we delete the overlay and recreate it in 50ms. The base image never changes, and your host is 100% isolated.

The Test Model

• Model: gemma-4-E4B-it-qat-UD-Q4_K_XL (Google Gemma 4B quantized)
• Engine: llama.cpp (llama-server with --jinja tool calling)
• Harness: Python script executing commands via SSH on 127.0.0.1:2222

The Benchmark & Results

We tested 3 distinct DevOps failure modes:

Scenario 1: Rogue Process & Port Collision (PASSED in 60.5s)

• Fault: Nginx was stopped, and a background Python script was spun up on port 80.
• Trace: Gemma ran systemctl status nginx, noticed the socket couldn't bind, checked lsof -i :80, extracted the PID (892), ran kill -9 892, and started Nginx cleanly.
• Verdict: Flawless multi-step diagnosis.

Scenario 2: Permission Lockout / HTTP 403 (FAILED in 196.2s)

• Fault: /var/www/html/index.nginx-debian.html was set to chmod 000.
• Trace: Gemma ran ls -la /var/www/html and accurately printed ---------- 1 root root. But instead of modifying permissions (chmod 644), it ran chown -R www-data:www-data. When curl -I still returned 403, it assumed Nginx routing was broken and spent the next 5 turns rewriting virtual hosts with sed until it ran out of turns.
• Verdict: Classic human-like pitfall—misidentifying root cause and over-engineering the fix.

Scenario 3: Configuration Syntax & Health (PASSED in 76.4s)

• Fault: Service restart and syntax health check.
• Trace: Ran nginx -t, inspected configuration, checked status, and ensured traffic on port 80 was restored.

Key Takeaways

1. Small models can do real system work: A 4B quantized model is capable of parsing Linux CLI stdout and chaining commands logically.
2. Error recovery needs guardrails: Once Gemma made a wrong hypothesis in Scenario 2, it doubled down instead of questioning its initial assumption.
3. QCOW2 overlays are the ultimate agent sandbox: Fast, zero-cost, disposable, and secure.

Full code, setup instructions, and raw execution logs are open source:
👉 https://github.com/Chetan0246/local-agent-sandbox