Scaling agents locally with Incus, Ansible, Tailscale

I regularly worry about agents performing risky tool calls on my machine. In particular, one of the concerns I've been thinking about is filesystem access:

• Secrets on disk getting leaked
• Destructive deletions and unexpected updates
• Surprise package installations and system configuration changes

I have other concerns too (network, database, identity), but for this article I'll describe how I view the problem of filesystem isolation, and the experiment I'm running to solve it.

I describe my ongoing experiment to use an old desktop as a local Debian server that runs an Incus service to spin up VMs to run agents and untrusted code.

Why worry about agents accessing my filesystem?

I've seen agents actually make the mistakes that fit the three categories above: reading secrets (and repeating them into chat logs), file edits, and system config changes.

For the most part, agents are very safe. Auto-accept permission mode can operate a very long time on most problems and enabling it makes me faster for very low risk.

For context, It's worth noting that I'm intentionally using agents in this way:

1. Auto-accepting tool calls using "auto" permission modes (Claude, Codex)
2. Using them for local/remote system administration
3. Running them in parallel (more tool calls, less direct supervision)

This is the time-worn tradeoff: increasing velocity (by granting more permissions) at the expense of security.

For the most part, I haven't spent too much time worrying about isolation. I use Branchmore to monitor and catch unsafe tool calls. But a few months ago, something forced my hand: my primary computer was running out of memory.

Design criteria: running out of memory for parallel agents

My experimentation with Incus started when my desktop ran out of memory. It had 64GB of memory, which I thought would be enough. But I run multiple workloads on it--engineering, business, and personal--and each includes its own set of applications and browser tabs.

This was fine, but as I ran more parallel agents, I was running Docker Compose clusters, one per worktree. Each of these added around 500MB of memory.

Another factor was RAM prices. I thought I could wait out the RAM shortage, but running agents in parallel, their applications, and a general proliferation of experiments.

A 32GB pair of DDR5 RAM sticks cost me $189 on Newegg in January 2025:

2 x 32gb memory in January 2025 for $189.99 on Newegg

But the same exact product on Newegg is now $1259 (6.7x) in September 2026:

2 x 32gb memory in September 2026 for $1,259 (6.7x vs Jan 2025) on Newegg

I could spend time to optimize the worktrees and be more disciplined about cleaning up apps. But I felt an acceleration of researching, building and running. I had other objectives worth thinking about too.

A few other design criteria and considerations

A cheap place to deploy development applications. Things like small web applications with a database. I was previously running them on my computer, but doing so eats into the memory I have. Local hardware is cheaper than cloud, but I didn't want background jobs lingering on my desktop.

A safe place to deploy untrusted code, especially AI-enabled ones. In particular, I wanted to let Openclaw and Hermes run for longer without monitoring them as closely as I had done. Friends have told me about worrying incidents of prolonged hallucinations.

I've also been worried about malicious agent skills, hallucinated or malicious packages, and I have some general concern about AI accelerating supply chain attacks. I realized that fear was holding me back, and I just wanted to reduce risks.

I also happened to have a desktop, about three years old, that I was no longer using. After realizing that I couldn't buy cheaper DDR5 memory sticks because they don't mix well, I also realized that I could buy 64GB of DDR4 memory more cheaply.

I bought my 2 x 32gb DDR4 on Newegg for $504 in August 2026, which was 60% cheaper than DDR5. Some examples:

DDR4 memory prices are 60% cheaper than DDR5 in September 2026

To summarize: I could sense that memory and caution was slowing me down, and I wanted to find a middle ground to be more productive while being cautious.

Landing on Incus, Ansible, and Tailscale to scale agents locally

How I set up my local incus server

First, Incus to run VM's. I landed on Incus primarily because it gives me the option to launch VMs and containers. I still like to SSH into virtual machines; they have a statefulness that I find easy to reason about. They let me store files, install applications, and accumulate data. They're heavier than container workloads, but they're also simpler. There's less wiring for filesystems and networking.

Why not Kubernetes? I think Kubernetes is a great tool, but I'm not an expert. I worry about its learning curve, and I want to spend more time building than operating it. I also don't need granular isolation or control, horizontal scalability, and I don't need rapid provisioning. I'm looking for basic, stable, coarse-grained isolation that's drop-dead simple to set up.

Next, Ansible to provision repeatably. Then I used my agents to write and run Ansible and some Incus management scripts. I wanted some light automation, so I could spin up consistent VMs using a base image. I wanted the provisioning in code, so I could create different variations with a standard set of tools and configurations.

After I had a few machines that kept changing IP addresses, I ran into an old problem of needing to remember whether my Openclaw instance was hosted at 10.0.0.47 or at 10.0.0.213. I've run some local DNS servers before, but found them to be annoying.

So lastly, Tailscale for easy routing and name resolution. I realized that adding Tailscale agents to my automated provisioning allowed me to have easy names like claw.mytailnet12345.ts.net. Not only would I get dynamic DNS resolution to my machines, I'd also get out-of-network VPN access.

How I run the VM's

I keep Openclaw and Hermes each running on their own VM. They manage their own state, and they won't conflict with any other running agents.

Sometimes I deploy applications on their own boxes, other times Hermes/Openclaw to let them manage/edit them. I try not to be a purist, but separate it if there's a reason to worry about agents' messing up the service.

Overflow Claude agents get their own boxes and their own Docker clusters.

I've also been starting to use herdr as a nicer terminal interface to view all my running agents on my local and remote machines.

Identity, secret management and least privilege is also loose concern. But this is a topic for another day.

Reflection: low-effort scaling

The platform project was small: perhaps 1–2 days of work spread out over a few weeks. My agents were mostly building and spinning up the VMs to test the applications. Incus was easy and robust for my management needs, Ansible worked well, and Tailscale simplified my setup.

It solved my immediate problems: being able to scale up beyond my computer. My specific issue was running out of DDR5 memory, but I'd like the option to scale up beyond two computers if I wanted to. I could also apply my Ansible playbooks to any remote server with an SSH connection.

I'm also more comfortable knowing that Openclaw and Hermes are given more autonomy. Even if they download a malicious package, it will not be as easy for them to reach into my primary computers to destroy or steal data.

But I will point out that this isn't for everyone. It will still be easier to code on my immediate computer without SSH, and to only think about one machine. This is an extra management layer, and I sometimes have to remember where files are and what the hostnames are.

If you don't have memory or isolation problems, and you have a stable setup, then you don't have this type of obstacle. But given that I was being slowed down in a few areas, I'm glad I put the time into it.

Appendix: high-level setup

For technical details, here are high-level instructions for my setup (including test runs). You can paste this blog article into your agent for wider context, or just the following list. It can do everything after Debian is installed.

In this setup, I assume: your main computer is the client, and the server will run Incus. Technically, you don't need to run Incus on a separate system. You could just run Incus on your immediate computer.

1. Set up desktop
   1. Install Debian
2. Install Incus server
   1. You can run management commands on the Incus server either over SSH, or via an Incus client on your primary computer
3. Test drive Incus with a test VM
   1. Use agent (e.g. Claude, Codex) to manually spin up a VM
   2. SSH into the VM
   3. Tear down the VM
4. Create a new repo with management scripts to make VM spin up/down consistent
   1. List all VMs
   2. Create single VM
   3. Delete single VM
5. Create an Ansible subdirectory, install Ansible, and test drive provisioning
   1. Provision a simple test VM - e.g. to install Emacs
   2. Define an Ansible playbook that installs the tools you need: e.g. a database, programming language, agents, dev tools, dotfiles
6. (Optional) Create management script to publish images - layers so we don't have to repeat slow setup common to multiple VMs
   1. Build VM image
   2. List VM image
   3. Delete VM image
7. (Optional) Register and install Tailscale on your server, your primary workstation
   1. Bake Tailscale into your common Ansible provisioning layer for all machines
   2. Use OAuth credentials so your VMs automatically bring up Tailscale. You can put the OAuth credentials in a password manager on your primary computer, or in an env file. Then Ansible can inject the value into your client machine without reading it.

Appendix: why not use a Sandbox service?

I was asked after I posted this – why not use a Sandbox service like Daytona, Fly.io or Modal? These cost is minor (can be < $1/mo, as I understand it) because they run for as long as you need them.

I have nothing against sandbox services! For starters, I haven't had time to try them out yet. In addition, I wanted something to host an Openclaw and Hermes agent, each in isolation - so that requires some persistent memory.

Then, I wanted to actually run a Docker Compose application that I can interact with in the browser. Screenshots through headless browser services like Browser Use make sense, and are also easy. I want something stateful that I can SSH into, run agents on, and have some flexibility in deploying applications. I may deploy dev instances within each agent's VM, or I may create a VM to deploy persistent applications on, independent of each agent.

There are some shortcomings with this Incus setup, but I don't think it's a big issue. Provisioning may be a few minutes, and spin up may be a few seconds (as opposed to fractions of a second for a microVM or container). But most of my agent interactions are long running jobs, and ideally I can trigger the jobs in the background so a long startup time is invisible to me.

Technically, monthly electricity cost of a second server at home is higher than the cost these short-lived sandboxes. But a few differences: (1) I already mentioned my desire to deploy long-lived services. (2) I plan to run more agents and builds in parallel, and (3) technically if I wanted something short-lived, I could use the same Ansible provisioning infrastructure to spin up in AWS EC2 as well.