DEV Community: Alex Ialanji

Six AI agents froze my production server. Each one did the right thing

Alex Ialanji — Thu, 29 Oct 2026 14:07:34 +0000

This article was created with the help of AI. The incident, the logs and the numbers are from my own server.

On September 25 at 19:01 UTC my server stopped responding. The site, the admin panel and the publishing queue all froze at once. Seventeen minutes later I hard-rebooted it.

There was no attack and no traffic spike. I had given Claude Code a list of six tasks, and it started six agents in parallel. Each one dutifully installed dependencies, ran the type checker and the tests, and built the project. On the same machine that serves the site.

The arithmetic nobody did

Take round numbers close to mine. The server has 10 GB of RAM, and the live services use about 3. That leaves 7.

One typecheck or build of a project this size takes 1.5 to 3 GB. Six at once need 9 to 18. That doesn't fit into 7 however you slice it.

Anyone could have added this up. Nobody did, not me and not the model.

Why the server didn't save itself

I expected Linux to kill the hungriest process and let the rest carry on. It didn't. The journal has no out-of-memory kills at all.

What it does have: system services reporting they were running "under memory pressure", and Docker saying it could not send a kill signal to a container process. Then nothing until the reboot.

Swap did it. When RAM runs out, the kernel starts moving pages to disk and back, and that eats all the CPU time. On paper there is still memory, so the kernel sees no reason to kill anything. In practice the machine is busy only with itself. The server didn't crash, it hung, and that's worse. A crashed service restarts on its own. A hung one waits for a human.

Why the agent did it

For agents, parallelism is a feature. Six independent tasks go to six workers, each with its own copy of the code in a separate git worktree, nobody in anyone's way. In the code, that held.

Where they got in each other's way was memory, and nobody had told them about it. A model doesn't know how much RAM the machine has or that a live site runs next to it, unless it checks or someone tells it. And "finish the job and make sure it builds" is usually a good habit. Run six times over on a small server, that habit took the server down.

What I changed

No more than two heavy jobs at once. Heavy means a build, a typecheck of the whole project, or a Docker image build.
Each agent checks only its own part. Types and tests for the package it changed, no full build.
Full builds run one at a time, from the main session, not inside every agent.
The rule lives in CLAUDE.md, the project notes Claude reads at the start of every session. Since then it brings the rule up on its own when the task list gets long.

Doing the math for your machine

Take total memory, subtract what's already in use, keep a gigabyte in reserve, and divide by the peak of one job. With my round numbers that's (10 - 3 - 1) / 3, so two jobs.

You don't have to guess the peak. Measure it:

free -h
/usr/bin/time -v npm run build 2>&1 | grep 'Maximum resident'

The first command shows how much memory is free right now. The second runs the build and prints the peak of its largest process, in kilobytes. A build that forks parallel workers uses more than that in total, so round up.

Two safeguards for when the math lets you down. First, don't build on the production server at all if you have a separate machine or CI. Second, install something like earlyoom, which kills the hungriest process early instead of waiting for the system to drown in swap.

I put in the second one late, and I shouldn't have. Three weeks later the server hung again, this time with no agents involved: a nightly job that checks for dependency updates built a Docker image and started a second copy of a service next to the live ones. Same arithmetic, only nobody was around to do it.

This is what /etc/default/earlyoom looks like on my box now:

EARLYOOM_ARGS="-m 5 -s 100 --prefer ^(chrome|chromium|tsc|next-server)$ --avoid ^(postgres|dockerd|containerd|containerd-shim|systemd|sshd|journald)$"

-s 100 is the part that matters here. By default earlyoom acts only when both RAM and swap are nearly gone, and my server was already thrashing while swap was still free. With -s 100 it acts on low RAM alone. Builds and the browser go first, and the database and Docker are never touched.

Each of the six agents did the right thing. The only thing wrong was how many there were.

I collect findings like this about Claude and AI on Promtime, the news site this server runs.

Claude Code quietly deletes your session history after 30 days. I noticed three and a half months in

Alex Ialanji — Thu, 22 Oct 2026 14:07:28 +0000

This article was created with the help of AI. The numbers come from my own server.

My first prompt to Claude Code on my work server is dated May 26. The oldest session transcript still on disk is from September 11. Everything in between is gone, and I never saw a warning.

It isn't a bug. Claude Code keeps session transcripts for 30 days and then deletes them. The docs say so, on a page few people open. One line in the settings changes that, but only going forward.

What disappears

Every Claude Code session is written to a file under ~/.claude/projects/. It holds the whole conversation: your messages, the model's replies, every tool call and its result. /resume reads these files to take you back into an old session.

After 30 days the file is deleted. Nothing tells you. The session simply stops showing up in /resume, and its subagent transcripts and any orphaned git worktrees from parallel tasks go with it.

Some things survive:

Prompt history. history.jsonl keeps what you typed, unless HIPAA settings apply to your account. So my May questions are still there. The answers are not.
Auto memory. The notes Claude keeps about a project are not part of the sweep.
Claude Desktop and Cowork sessions. Since v2.1.248 they are kept at any age, unless you set desktopSessionCleanupPeriodDays.

I checked my own server

On October 11 I had 724 transcript files taking 861 MB. By age: 181 from the last week, 62 from the week before, 338 from the third week and 143 older than that. Not one older than 30 days.

I can't check what was in the deleted ones. Probably bug hunts and notes on why something was built a certain way. Exactly what you open an old session for.

Has it already happened to you?

Three commands:

grep cleanupPeriodDays ~/.claude/settings.json
find ~/.claude/projects -name '*.jsonl' | wc -l
find ~/.claude/projects -name '*.jsonl' -mtime +30 | wc -l

The first shows whether you set a retention period. If it prints nothing, you're on the 30-day default. The second counts all transcripts, the third only those older than 30 days. If the third prints zero and you've used Claude Code for more than a month, the sweep has already run.

One line in settings

Add the key to ~/.claude/settings.json:

{
  "cleanupPeriodDays": 3650
}

That's about ten years. Three things to watch:

Zero doesn't mean never. 0 fails validation, and the minimum is 1. To keep things long, pick a big number.
It only works forward. Deleted files don't come back.
Disk adds up. A month of my work took 861 MB, roughly 10 GB a year. I run a lot of agents, so yours may be far less.

The catch: those files hold everything

Transcripts are plain text, unencrypted. The docs are blunt about it: if a tool reads a .env file or a command prints a credential, that value lands in the session file. That is why the default is short.

The longer you keep history, the more secrets pile up on disk. If you raise the limit, deny reads of credential files with permission rules, and keep ~/.claude/projects/ out of unencrypted backups.

And if you'd rather keep nothing, CLAUDE_CODE_SKIP_PROMPT_HISTORY stops writing transcripts and prompt history, --no-session-persistence does the same for a single -p run, and claude purge wipes what is stored for one project.

What I picked

365 days. A year covers nearly every time I go back to an old decision, and the disk use and the secrets stay manageable.

Funny that a tool which holds your whole project in its head forgets its own history after a month.

The details are in the Claude Code docs. I collect findings like this on Promtime, the news site I edit.