DEV Community

Cover image for My AI Agent's VM Gets Replaced Every ~2 Hours — Measured, Then Designed For
mryo0826
mryo0826

Posted on Edited on Originally published at qiita.com AI-assisted

My AI Agent's VM Gets Replaced Every ~2 Hours — Measured, Then Designed For

Three notes before we start

  • "Muse" in this article is not Anthropic's Claude. Muse is the name of a personal AI agent product, and this article is about the Linux VM that comes with it.
  • The AIDD practice guide is written in Japanese. The canonical text of the guide mentioned below is in Japanese. If you want to read it, please run it through a translation into your own language: https://github.com/mryo0826/aidd-practice-guide
  • This article was written with an AI collaborator. It was produced in collaboration with a generative AI agent (Muse), and this English version was also translated with AI assistance.

0. Background

The environment in this article is the VM of Muse, which I use every day. I practice a way of working with generative AI agents that I call AIDD, and I have published a practice guide for it.

One of AIDD's premises is: keep continuity on the file side, and treat sessions and execution environments as disposable. The countermeasures in this article apply that same premise to the VM side.

In the overall AIDD setup, Opus (Claude Code, on the Claude Pro plan), who handles internal affairs, owns the canonical documents, while this Muse handles external affairs — posting to outside venues and dealing with platforms. The two communicate by letters (round trips through a GitHub mailbox repository). Inbox checks are run as follows: on the Muse side, scheduled checks twice a day plus ad hoc checks as needed; on the Opus side, a check at session start plus ad hoc checks when the human asks. Muse itself runs on a free account; the only actual payment is Opus's Claude Pro plan.

The overall AIDD setup

And here is this Muse's home structure. What this article calls "what survives" is this structure as a whole.

Muse's home structure

1. Introduction

Muse is a personal AI agent that comes with one Linux virtual machine (VM) per user. The agent creates files, runs programs, and operates a browser on that VM. "The AI has its own computer" is a defining feature of this product.

But that computer had no fixed address. My Muse's VM is replaced by a different instance roughly once every two hours, running processes and all. Files survive, but work in progress vanishes. I first noticed this when a video render disappeared in the middle of rendering.

This article covers the results of actually measuring how often the replacement happens, and how I redesigned the work around that premise. I do not know why it gets replaced — there is no information from the platform side. Every number in this article is limited to what I measured in my own environment.

2. How I Measured It

"It gets replaced sometimes" is not enough to design against, so I decided to measure it first.

The method is simple: periodically check the kernel's boot ID. On Linux, every boot assigns a new UUID to /proc/sys/kernel/random/boot_id. I read it every five minutes, and if it differs from the previous value, I append the detection time and the old and new IDs to a log file. The boot time is an estimate back-calculated from the uptime at the moment of detection.

Flowchart of the boot-watch hook: poll boot_id every five minutes and log any change

Here is the actual script (the parts that follow the platform's hook conventions are partly omitted).

#!/usr/bin/env bash
set -euo pipefail

STATE="$HOME/hooks/state/vm-boot-watch.boot_id"   # where the previous boot_id is stored
LOG="$HOME/workspace/worklog/vm-boots.md"          # the observation log
CUR="$(cat /proc/sys/kernel/random/boot_id)"
UP="$(cut -d' ' -f1 /proc/uptime | cut -d. -f1)"
NOW="$(TZ=Asia/Tokyo date '+%Y-%m-%d %H:%M:%S')"
BOOT_TS="$(TZ=Asia/Tokyo date -d "@$(( $(date +%s) - UP ))" '+%Y-%m-%d %H:%M:%S')"

PREV="$(cat "$STATE")"
if [[ "$CUR" == "$PREV" ]]; then
  exit 0   # no change
fi

# generation change detected: append to the log and update the saved ID
printf -- "- %s JST detected: VM generation change boot_id %s -> %s (new generation estimated boot %s JST)\n" \
  "$NOW" "$PREV" "$CUR" "$BOOT_TS" >> "$LOG"
printf '%s' "$CUR" > "$STATE"
Enter fullscreen mode Exit fullscreen mode

I started observing at 22:39 on 2026-11-17. Since the detection interval is five minutes, a generation that is replaced within less than five minutes may be missed. With that caveat, here are the results.

3. What I Measured

Between 22:28 on 2026-11-17 and 23:06 on 2026-11-18 — about 24.6 hours — 14 generations appeared. The median lifetime of a generation was about 116 minutes; the shortest was 33 minutes and the longest 165 minutes.

The replacements happen regardless of user activity. They also occurred at 02:10 and 04:55, while I was asleep. Whether I am working or not, and whether the machine is under load or not, a generation reaches the end of its life in roughly two hours — that is the picture the measurements give.

(Update) I kept observing, and as of 17:13 on 2026-11-19 the total reached 28 generations over 42.8 hours. The median lifetime is 113 minutes and the trend is unchanged, but on the morning of 11-19 there was a stretch where extremely short generations — lifetimes of 2, 6, and 9 minutes — appeared in a cluster. I do not yet know what governs when the short ones appear. The distribution of lifetimes across 27 generations is as follows.

xychart-beta
    title "Distribution of generation lifetimes (27 generations, in minutes)"
    x-axis ["0-30", "30-60", "60-90", "90-120", "120-150", "150-180"]
    y-axis "Generations" 0 --> 10
    bar [4, 2, 3, 7, 10, 1]

4. What Breaks and What Survives

What survives a generation change and what vanishes is cleanly split.

What survives is every file under the home directory. Work files, settings, records, and the tools and libraries installed on the home side are all still usable in the next generation, exactly as they were.

Three kinds of things vanish: running processes, the contents of /tmp, and changes to the system area (packages installed with apt, writes under /usr, and the like).

What survives in the home directory and what vanishes outside it

The concrete damage was a video render. What vanished was the drawing of a vertical short video (the Merrow version) of "The Pencil That Gets Shorter," a song I make together with Muse. At frame 960 of 1,857, the whole VM disappeared and the process was nowhere to be found. With the output file only half-written, I could not even call it a "failure" — left alone, you cannot tell whether it finished or vanished.

Here is the full version of that song.

Merrow version:

Reina version:

5. Designing Around It

With the premise set to "the VM disappears in about two hours," the countermeasures narrowed down to three.

The first is to launch long-running work in a form where its state is visible. In my environment I built a small mechanism called job-runner, which manages job states in four values: DONE (completed), FAILED (failed), RUNNING (in progress), and LOST (vanished without a trace). A job with no exit-code record and no live lock is LOST; I treat it as "consumed by a VM generation change" and re-run it idempotently. Heavy jobs are serialized with a lock, to at most one at a time.

Sequence diagram of the job runner: a lost job is detected and re-run

Here is the actual launcher.

#!/bin/bash
# job-runner.sh <job_id> <expected_out|-> <cmd...>
set -u
JOBS="$HOME/workspace/.jobs"
JOB_ID="$1"; EXPECTED="$2"; shift 2
JD="$JOBS/$JOB_ID"
mkdir -p "$JD"
echo "$EXPECTED" > "$JD/expected.out"
# global lock: only one heavy job at a time (BUSY if it cannot be acquired)
exec 9>"$JOBS/global.lock"
if ! flock -n 9; then echo "BUSY"; exit 75; fi
# per-job lock: prevents the same job from being launched twice
exec 8>"$JD/job.lock"
if ! flock -n 8; then echo "ALREADY_RUNNING"; exit 74; fi
rm -f "$JD/job.exit"
"$@" > "$JD/job.log" 2>&1
CODE=$?
# write the exit code atomically, via a tmp file
echo "$CODE" > "$JD/job.exit.tmp"
mv "$JD/job.exit.tmp" "$JD/job.exit"
exit "$CODE"
Enter fullscreen mode Exit fullscreen mode

State is determined only by the combination of whether an exit record exists and whether the lock is alive.

Exit record Lock State
Present (0, and the output exists) — DONE
Present (non-zero) — FAILED
Absent Alive RUNNING
Absent Gone LOST

The second is to split work into small chunks. Video rendering is split into 10-second chunks; a finished chunk is verified by its length and skipped, only the remainder is drawn, and everything is joined at the end. Even if a generation change hits mid-way, at most one chunk is lost. This turned out to be exactly the same shape as the Scrum discipline of splitting tasks into predictable units and producing outputs that are in a releasable state.

Flowchart of segmented rendering: completed segments are skipped

The third is to keep continuity on the file side. The current state of the work is recorded as small, timestamped entries appended to a work log. VMs and sessions are both disposable; when resuming, you read that file first and then start. If files are the only thing that survives, putting the state in files too was the natural conclusion.

One more note: I do not send notifications for generation changes themselves. Being told about something that happens every day would just be noise, so notifications happen only when a job is suspected to be LOST.

6. What I Don't Know

I don't know why it gets replaced. It could be periodic deployments, or resource reclamation; the platform side publishes no information, and I have no way to check. I also formed a hypothesis — "maybe memory usage crossed a threshold" — but I could not verify that one either.

Three hypotheses for the replacements, all unconfirmed

As for frequency, the numbers in this article cover only the observation window from 2026-11-17 to 19. The interval could grow longer or shorter in the future. Leaving what I don't know as unknown, if I design on the premise "it disappears in about two hours," the design holds even if the frequency shifts somewhat.

7. A Second Issue: Environment Freshness

The fact that "changes to the system area vanish" was directly connected to security updates.

When I took inventory on the night of 2026-11-18, the VM's Node.js was still at 24.22.1 and had not reached 24.22.3, the release that addresses 16 vulnerabilities. The Python libraries were also still on old versions. Even if you upgrade with apt, the next generation change rolls everything back. The practice of "keeping the system side up to date" does not hold in this environment in the first place.

The freshness cycle: system updates vanish at the next generation change

Security freshness is not something you can afford to look at leisurely, once a month. In the field, responses run on weekends too; it is a day-to-day problem. So I made the countermeasure two-layered: move the tools and libraries I depend on over to the home side, and check freshness automatically every week, receiving a report only when there is a diff.

8. Putting It into Practice

This inventory was a countermeasure against generation changes, and at the same time an opportunity to review and optimize the architecture on my Muse side. I moved what had depended on the system area over to the home side, manage the locations and versions of tools in a registry, and measure freshness weekly. The overall home structure is the diagram in section 0; here I will list only what changed.

Here is what I actually did.

Item What I did Measurements / notes
Node.js Placed 24.22.3 on the home side and switched to it Track the latest patches in the 24 line. The decision on moving to the next major version will be made within 2027
Python libraries Placed updated versions on the home side Pillow 13.0.0, numpy 2.5.5, and others. They survive generation changes
Python itself Installed 3.15.2 side by side using uv, and made it the default for the drawing pipeline About 18% faster on a 240-frame render. Output matched pixel for pixel
Pillow Switched to the SIMD-optimized build (pillow-allsimd) Rendering about 1.73x faster. Output completely identical
FFmpeg Kept the platform-custom 8.1.2 as is Watching the 8.1 line for new releases and security information
Freshness check Runs automatically every week Reports only when there is a diff, with security diffs placed first

I also added a practice: whenever a tool is added to or updated on the VM, record it in the registry. In an environment you cannot see into, a tool with no record is the same as a tool that does not exist.

Rendering speed changed with this review, too. These are the measured seconds it took to draw the same 240 frames.

Bar chart of rendering time for 240 frames: Python 3.12 + Pillow, Python 3.15 + Pillow, and Python 3.15 + pillow-allsimd

Because the CPU is shared, the absolute values swing with the load at the time. Comparing ratios under the same conditions, moving to Python 3.15 gave about 0.82x, pillow-allsimd about 0.58x, and combined, rendering took roughly half the time or less. In every case, I switched only after confirming that the output matched byte for byte and pixel for pixel.

I measured one more thing: Python 3.15's experimental JIT compiler. In my build it can be enabled with PYTHON_JIT=1, but the same 240-frame render took about 24.3 seconds with it enabled and about 24.2 seconds with it disabled — within the margin of error. The output matched as well. The weight of the drawing sits on Pillow's native processing side, so this was not a place where a JIT, which speeds up Python bytecode, could take effect — that is my reading of the measurement.

Even in the official numbers, the 3.15 JIT's improvement is stated as a geometric mean of 7–8% on pyperformance (x86-64 Linux), which is not something that changes the order of magnitude. Furthermore, PEP 836, which proposed the JIT's survival, was withdrawn in October 2026, and the JIT is expected to be removed from the core from 3.16 onward. What I measured and found ineffective, I did not adopt.

Sources I consulted:

For this review, Gemini (Hydrogen, in thinking mode) took on the role of bringing in the general knowledge about tools and libraries. I did not adopt the general claims as they were; I used them by separating, against the measurements on my own setup, where they applied and where they did not. My thanks for the cooperation.

9. Summary

  • I don't know why it gets replaced. But if you measure the frequency, it can become a premise for design
  • Do not depend on what vanishes (processes, /tmp, the system area); put state and tools in what survives (files under home)
  • Split long work into chunks that can be detected when lost and resumed from the middle

As long as it remains a vague feeling of "it gets replaced sometimes," you are surprised every time it vanishes and lose time on recovery. Measure it and turn it into a premise, and a generation change is no longer an incident — it is simply a property of the environment.

Top comments (0)