The Memory Limit That Didn't Kill Anything

The process that runs my language model has a habit of growing. Not dramatically — it starts at a reasonable size and drifts upward over a shift, and the cause is somewhere in code I don't own. I spent a day trying to configure the leak away, failed, and did the sensible thing instead: stopped trying to fix it and put a ceiling on it, so the damage would be confined to one service instead of the whole box.

I gave it a soft ceiling. MemoryHigh in the unit file, a bit under what I'd measured the process to need.

That is how I turned a leak into an outage.

Everything was green

The symptom, when it arrived, was that nothing was wrong. The service was active (running). Its PID was current, its log had no errors, and top showed a process of unremarkable size. Every dashboard I had said the system was up, and it was up. It was just doing a batch of work — the kind that normally takes minutes — for most of a day.

The thing you have to understand about a soft limit is what the kernel does when you cross it. It doesn't kill the process. It doesn't return an allocation failure. It applies back pressure: it reclaims pages, pushes what it can to swap, and throttles the process's allocations until it fits back under the line. From inside the process, memory still works. Every allocation succeeds. Nothing raises. The program is simply slower, by a factor that has no upper bound, because its working set now lives on a disk.

A hard limit produces an event. A soft limit produces a symptom. MemoryMax kills the process, systemd restarts it, and you lose one unit of work loudly. MemoryHigh keeps it alive and makes it useless, and there is no line in any log that says so.

Where the truth was

Not in top. Resident size looked fine — that was the point, the kernel was keeping it fine.

The truth lives in the cgroup's own counters, which nobody looks at because nothing points you there:

cat /sys/fs/cgroup/system.slice/<service>/memory.events
cat /sys/fs/cgroup/system.slice/<service>/memory.swap.current

memory.events has a high counter — the number of times the process was throttled at the soft ceiling — and an oom_kill counter. Mine read a throttle count in the millions against zero kills. That pair is the entire diagnosis in two numbers: nothing died, everything was strangled. And memory.swap.current said that essentially all the swap on the machine belonged to this one service.

When a process is slow for no reason, check whether something is holding it under water on purpose. A limit you set yourself is the easiest cause to overlook, because you remember configuring it as a safety measure and safety measures aren't suspects.

The measurement that caused it

The ceiling wasn't arbitrary. I measured the process's working size and set the limit above it.

I measured it at the wrong moment. I took the number shortly after the model loaded, which is the calmest instant in that process's life: weights in memory, nothing computed yet, none of the caches and scratch buffers that only exist once real work is running. The number I recorded was a startup value wearing the label "working set". The real figure under load was meaningfully higher, and my ceiling landed underneath it.

So the process spent its life a few percent over a line it could never get back under, being throttled continuously, on every batch, forever.

A limit derived from a measurement inherits every flaw in how you measured. If the measurement was taken when the system was idle, the limit is a limit on idleness. Sample under load, or don't sample.

Slowness is not a local property

Here's the part that turned a slow service into a dead website.

The worker reads its queue from Postgres and then, in the same transaction, goes off to the model. That was already sloppy, and for a batch that takes minutes it was survivable sloppiness. With the batch now taking most of a day, the worker sat in idle in transaction for most of a day, holding a perfectly ordinary read lock on one table.

Then a deploy landed. Startup ran a migration, the migration wanted ALTER TABLE, and the ALTER queued behind the worker's read lock. Fine so far — a migration waiting is not an outage.

Except for the Postgres behaviour that everyone learns exactly once: a waiting AccessExclusiveLock blocks everything that queues up behind it. The migration wasn't just waiting, it was a wall. Every subsequent query against that table joined a line behind a lock that would not be granted until a language model finished a job it was never going to finish on time. The API never reached listen. Its last log line was Waiting for application startup, and systemctl reported it active (running), which was true and worthless. nginx returned 502 for everything except static images.

The whole cascade fits in one query, which is the one thing I'd want anyone to take from this:

SELECT pid, state, wait_event_type, left(query, 60) FROM pg_stat_activity;

An active | Lock | ALTER TABLE … sitting next to an idle in transaction | SELECT … is not a clue. It's the answer.

Both sides are fixed now, because both sides were wrong: DDL runs with a lock_timeout and logs who was holding, and the worker closes its transaction before it goes anywhere near the model. Never hold a database transaction across a call to something whose duration you don't control, and treat a lock without a timeout as a promise to hang the table rather than a promise to wait.

The alarm that had been ringing so long it was quiet

I found all of this by accident, which was its own lesson.

There was an hourly health check, and it had been finding problems the entire time. It said nothing, because alert state was stored as a single boolean for the whole system, and the rule was to notify on change. Chronically elevated swap had set that flag days earlier and it never came back down — so when new failures appeared underneath it, first the worker falling behind, then the API not answering, the state didn't change, and not one message was sent. Meanwhile the morning summary printed the numbers with no verdict attached and signed off with "all services running."

An alarm that stays silent because the system was already unhealthy is worse than no alarm, because it manufactures the feeling of being watched. State has to be tracked per check, not per system; unresolved problems have to keep repeating rather than being deduplicated into silence; and no report gets to print an all-clear until every check has actually passed.

What I'd take from it

I set the soft limit because a hard one felt violent — killing a process over a few megabytes seemed like an overreaction, and back pressure sounded like the gentler, more grown-up choice.

It's the opposite. The hard limit fails: the process dies, systemd brings it back, one batch is lost, and the log says exactly what happened. The soft limit degrades: nothing fails, nothing is logged, no counter you routinely look at moves, and the system quietly stops doing its job while continuing to report that it is doing its job. One of those is an event you can build on. The other is a fact your monitoring is structurally unable to notice.

Prefer the failure you can see to the degradation you can't. It's the same rule I keep arriving at from other directions — an absent value beats a wrong one — because the common thread isn't memory or validation. It's that a system which fails loudly is one you can operate, and a system which merely gets worse is one you find out about from your users.

The limit is gone now. In its place: a hard cap, and swap turned off entirely for that service. If it grows past what the machine can give it, it dies, and I hear about it.

Both of those are better than what I had, which was a process that was alive the entire time.