DEV Community

Remdore
Remdore

Posted on AI-assisted

What forked copies of an agent still have in common

A while ago I forked a running agent session three ways and wrote about it. Three people in the comments asked the same thing in different words: if the copies start from the same memory, do they also start with the same random numbers, the same keys, the same idea of who they are?

I did not know, so I measured it. Some of what I found is reassuring. One part is not, and it is the part I would have guessed was safe.

The setup

One session on DigitalOcean Managed Agents, the OpenCode harness on the 2 vCPU / 4GB size. Inside it is Ubuntu 24.04 on a 6.1 kernel, with Python 3.12.3 and Node 20.20.2. I never sent the agent a prompt. Everything here is about the machine it lives in.

Before forking anything I started two small processes in the session, one in Python and one in Node. Each one sets itself up, then sits in a loop waiting for a file called /tmp/go to appear. When it does, the process asks for random numbers in every way I could think of and writes them down.

Then I checkpointed the session, which took 24.7 seconds, and forked it three ways from that checkpoint. I did that three times, for nine copies, at 11.8, 14.4 and 15.1 seconds per batch. Finally I rolled the original session back to the same checkpoint twice, 5.4 and 5.3 seconds each.

That gives eleven machines that all woke up from the same instant. In each one I created /tmp/go and collected the answers.

Random numbers from a process that was already running

This is the result that matters, so here it is in full. "Same" means the value was identical in all eleven copies.

asked for, inside the process that predates the fork result
Python random.random() same in all 11
Python uuid.uuid4() 11 different
Python os.urandom(16) 11 different
Python secrets.token_hex(16) 11 different
Node Math.random() same in all 11
Node crypto.randomUUID() same in all 11
Node crypto.randomUUID({disableEntropyCache: true}) same in all 11
Node crypto.randomBytes(16) same in all 11

Nobody should be surprised by random.random() or Math.random(). They are ordinary generators whose state is a few numbers in memory, and the memory was copied.

The last three rows are different. Those are the calls you reach for when you want a session token or an API key. Every one of the eleven Node processes produced 5613fd13-eb56-4121-b753-5f2dd5b259dc as its next UUID, and f8e82cff3a9e3898341c08b3a6e9a53c as its next sixteen bytes.

I had put the disableEntropyCache row in as the fix. Node keeps a pool of pre-generated bytes for UUIDs, and I assumed that pool was the only thing that could be duplicated. It was not the only thing.

Python came out clean on the same test, and the difference is where the bytes come from. Python's os.urandom, uuid4 and secrets ask the kernel each time. Node's crypto module is built on OpenSSL, 3.0.19 in this image, which runs its own generator inside the process. That is my reading of why the split falls where it does. The table is measured and the explanation is inferred.

The kernel knew

A new process started after the fork got different values everywhere: /dev/urandom, a fresh Python uuid4, even a fresh random.random(). Eleven for eleven.

The kernel log in a forked copy says why:

[  502.526105] random: crng reseeded due to virtual machine fork
Enter fullscreen mode Exit fullscreen mode

The sandbox exposes a VM generation counter to the guest, it shows up as VMGENCTR in the ACPI device list, and Linux reseeds its random pool when that counter changes. DigitalOcean has done the correct thing at the layer it controls. A fork is announced to the kernel and the kernel reacts.

What does not exist is a way for that news to reach a userspace process on its own. The kernel was told and reseeded. Node was not told, and carried on from the state it had.

Who each copy thinks it is

Everything I checked was identical across all eleven:

  • hostname, private IP address and MAC address
  • the kernel's boot_id
  • the SSH host key
  • the public address that outbound requests come from
  • the PID of my waiting process, 640 every time

/etc/machine-id was empty, so there was nothing there to compare. I also looked through the environment of PID 1 for anything naming the session and found nothing that differed between a fork and its parent.

So a copy has no way to learn from the inside that it is a copy, or which one. If your agent needs to tell its branches apart, the label has to come from outside, written in after the fork.

Time

The wall clock was right in every copy, to within the few seconds my method could resolve.

The monotonic clock was not adjusted, and I think that is the right design, but it has a consequence. I had armed a 150 second timer in the Python process before the checkpoint. In all ten copies where I read it back, it fired after exactly 150.00 seconds of monotonic time, never more than two milliseconds late. By the wall clock that was anywhere from 174 to 261 seconds after I armed it, depending on how long the copy had spent not existing.

/proc/uptime tells the same story. Both rollbacks woke up reporting 507.6 seconds of uptime, about fifteen seconds apart in real time.

A timeout measured on the monotonic clock pauses while the machine is a checkpoint. A deadline compared against the wall clock does not. Code that mixes the two will disagree with itself after a fork.

Things in flight

A line written to a file but still sitting in Python's buffer, not yet flushed, survived into every copy. Each one flushed it and added its own line after it, and each ended up with a complete file.

An open connection did not survive. The Python process had made one HTTP request to www.google.com on a keep-alive connection before the checkpoint. After the fork, each copy sent a second request down the same socket. All eleven timed out after eight seconds with no reply and no reset. In the same session, before any forking, the same connection had answered a second request after forty idle seconds.

What I got wrong

I expected Node's randomBytes to be safe, and built the test around the UUID cache being the only problem. The control I added to prove the fix is what showed the fix does not work.

My first results table crashed. I had the guest print JSON with echo under sh, which on Ubuntu is dash, and dash expands \n inside the string. Every record with a newline in it arrived broken.

My first HTTP check reported the status line of the second response as er-hp. It was reading the leftover tail of the first response's headers.

A pre-test that ran for two minutes inside one exec call came back as a 524 from the gateway. Commands that long need to be split.

And the connection test has a hole. I never sent the second request from a session that had been checkpointed but not forked or rolled back, so I cannot say whether the checkpoint or the copy is what kills the socket.

What I would take from it

Generate secrets in a process started after the fork. That path was clean in every run. The risk is a long-lived server that was already running at the checkpoint.

Rollback has the same problem in time. Roll back twice and the same process hands out the same token twice. If the first one already left the machine, you have issued a duplicate.

Give each branch its name from outside. Nothing inside will differ.

Reconnect after a fork. Do not expect sockets to come with you.

Caveats

This is one session, one checkpoint, one image and one morning. Eleven wake-ups agreeing exactly is strong evidence for "identical", but I tested Node 20 and Python 3.12 and nothing else. OpenSSL is documented to reseed its generator periodically, and I did not wait to see when the copies would drift apart. I did not test TLS, so I cannot say what a forked process does on its next handshake. The whole run used ten sessions, none alive for longer than about twenty minutes, with no model calls. The prepaid balance read $1.09 before and after, so I do not have a cost figure yet; the earlier four-session test came to seven cents.

The two probe scripts are about forty lines between them. The Node one is this, plus the same calls again once the file appears:

const before = { mathRandom: Math.random(), uuid: crypto.randomUUID() };
const t = setInterval(() => {
  if (!fs.existsSync("/tmp/go")) return;
  clearInterval(t);
  report(crypto.randomUUID(), crypto.randomBytes(16).toString("hex"));
}, 200);
Enter fullscreen mode Exit fullscreen mode

Top comments (0)