There are two ways to suspend a process tree. The classic one is kill -STOP: a job-control signal goes out, the process stops, and kill -CONT brings it back. The newer one is the cgroup v2 freezer: you write 1 into the cgroup.freeze file in a cgroup directory, and that cgroup plus everything under it stops.
Both arrive at "this process is not running right now", but they are not the same mechanism. Here is the part worth knowing: both can be in effect at the same time. Lift one and the process does not start running — the other one takes over. I walked through that handover step by step inside a container, and the resulting picture holds both a monitoring trap and a recovery trap.
The setup
The lab is the linuxkit virtual machine of Docker Desktop 4.37.2 (Engine 27.4.0) on a Mac: kernel 6.10.14-linuxkit, a single cgroup v2 hierarchy, and because the container runs --privileged, /sys/fs/cgroup is writable. In an ordinary container that directory arrives read-only; every write below came from a privileged shell. The cgroup root the container sees belongs to its own namespace, so even there a cgroup.freeze file exists — which does not contradict the documentation's "exists on non-root cgroups", because that root is not the real root.
The measurement method is crude but hard to argue with: every worker process appends one line to its own file every 100 ms. The answer to "is it running" is whether that line count grows. I read the state letter from the third field of /proc/<pid>/stat and the sleep site from /proc/<pid>/wchan. The kernel-side counterparts come from the source on the master branch; I separately checked that the two lines quoted below are byte-identical at the 6.10 tag.
One detail up front: this kernel has no cgroup.stat.local file. The frozen_usec field that reports time spent frozen is in today's documentation, but on my kernel that file never appears. The docs describe the current development series (7.4.0-rc3) while the lab runs 6.10.
One says T, the other says S
Two separate cgroups, one worker in each. I froze one with cgroup.freeze and sent kill -STOP to the other:
FROZEN pid=8 stat=S State: S (sleeping) wchan=get_signal
STOPPED pid=9 stat=T State: T (stopped) wchan=do_signal_stop
f1 events: populated 1 frozen 1
f2 events: populated 1 frozen 0
The stopped process is T (stopped). The frozen one is S (sleeping) — an ordinary sleep. Neither one makes progress, neither tick file grows, yet a human or a monitoring plugin reading the process list sees only one of them as "halted".
The kernel spells out why. do_signal_stop() sets the state directly:
current->jobctl |= JOBCTL_STOPPED;
set_special_state(TASK_STOPPED);
The freezer lands in do_freezer_trap(), and the line there is:
__set_current_state(TASK_INTERRUPTIBLE|TASK_FREEZABLE);
The letters in /proc come from the task_state_array table in fs/proc/array.c, and in that table TASK_INTERRUPTIBLE renders as S (sleeping) while TASK_STOPPED renders as T (stopped). So the freezer takes no process state of its own; it goes to sleep interruptibly. The TASK_FREEZABLE in that line is not a state but a modifier bit attached to the sleep, and it does not reach the reported letter.
The wchan values separate the two paths as well: get_signal for the frozen process, do_signal_stop for the stopped one. That first value also tells you what the freezer really is: do_freezer_trap() is called from inside get_signal(), so the freezer is a trap living in the signal path.
Watch the cgroup's own ledger too: for the group I stopped with SIGSTOP, cgroup.events reads frozen 0. The processes are halted and the cgroup says it is not frozen. It is telling the truth, because nobody froze it.
The pid list you read is already stale
The practical difference starts here. kill -STOP needs a pid list. The time between reading that list and sending the signal is time during which the target keeps forking.
I put a process tree that constantly spawns children into a cgroup, read cgroup.procs, waited 0.3 seconds (about what it takes a script to filter the list and decide), then sent SIGSTOP to the list I had read:
snapshot: 38 pids
afterwards in cgroup: 49 pids
in the snapshot: T=37 S=0
NOT in the snapshot: T=0 S=12
The 37 processes on the list are T, genuinely stopped (one had exited in between). The 12 processes that were not on the list are S and none of them stopped. In an earlier run of the same experiment the numbers were 10 escapees against a 40-pid list; in both runs the escapees landed between a quarter and a third of the processes actually stopped. The only reason those escapees did not grow the tree further is that the forking parent happened to be on the list; had a forker been among them, the list would have fallen further behind.
Same tree, same delay, with the freezer:
before freeze: 38 pids
2 seconds later: 38 pids (events: populated 1 frozen 1)
state spread: T=0 S=38
No new pids, nothing escaped. The freezer does not work through a pid list; it works through cgroup membership. The documentation puts it in two sentences: a process moved into a frozen cgroup stops, and a process moved out of one starts running. I ran the first — a worker I moved into a frozen cgroup wrote one more tick and then went quiet; after thawing the cgroup its counter climbed from 5 to 13. The second sentence, moving a task out of a frozen cgroup into a running one, is not something this run tried; that half rests on the documentation alone.
It receives signals, it does not act on them
Sending a signal to a frozen process is not wasted; it just waits. I froze a worker that installs its own SIGTERM handler and sent SIGTERM:
SigPnd: 0000000000000000
ShdPnd: 0000000000014000
handler ran (while frozen): NO
The 0x14000 in the ShdPnd mask lights bits 14 and 16; since signal n sits at bit n−1 in that mask, those are 15 and 17 — SIGTERM and SIGCHLD. The signal sits in the thread group's pending set and the handler does not run. The moment I wrote 0 into cgroup.freeze, a HANDLER_RAN line appeared in the file. Freezing does not cancel signal delivery; it defers it.
SIGCONT, on the other hand, does nothing here. I sent SIGCONT to the frozen group: cgroup.events still said frozen 1, the tick counter still sat at 5. Only writing 0 into cgroup.freeze releases the freezer. That feels right to me: job control is a contract belonging to the user's shell, while the freezer is the cgroup owner's policy; if one could override the other, the policy would not be a policy.
Fatal signals do get through. The documentation says in a single sentence that processes in a frozen cgroup can be killed by a fatal signal, and the lab confirms it: I sent SIGKILL to a lone frozen process and it died, while the cgroup stayed at frozen 1 and only populated dropped to 0. A frozen cgroup stays frozen even after it empties out.
In October's memory.max and oom.group article I left cgroup.kill as "not something this run touched". This run touched it, in the shape I like least — on top of a frozen cgroup:
before: lab=3 pids, sub=1 pid states: S S S S
cgroup.kill written (rc=0)
after: lab=0 pids, sub=0 pids states: GONE GONE GONE GONE
lab/cgroup.events: populated 0 frozen 1
All four went, including the one in the child cgroup. Freezing does not stand in the way of a group kill. If you froze a group for inspection and the situation got away from you, you do not have to thaw it first.
When the two layers stack
Here is the heart of it. Because SIGSTOP and freezing are not the same mechanism, they can stack — and the order changes the result.
First I stopped two workers with SIGSTOP (T, cgroup frozen 0), then wrote 1 into cgroup.freeze:
after writing freeze=1, frozen=1
states: S1=T wchan=do_signal_stop
cgroup.events says frozen 1, yet the process is still T and still parked in do_signal_stop; it never entered the freezer's trap. (The second process in the group was T as well; the script line probing it was written wrong, so it landed separately in the log and only the first one made it into the block above.)
The kernel source gives the reason in a single line:
frozen = test_bit(CGRP_FREEZE, &cgrp->flags) &&
cgrp->freezer.nr_frozen_tasks == __cgroup_task_count(cgrp);
The comment above it reads: if the cgroup has to be frozen (the CGRP_FREEZE bit is set) and all tasks are frozen and/or stopped, the cgroup is considered frozen. Both conditions are required. So the frozen 1 line states two things at once: a freeze order is in effect and nothing in the group is running. What it does not state is which layer is holding the processes. The frozen 0 of the SIGSTOP-ed group in the first measurement is the same formula seen from the other side: no order was given, so the first condition fails.
Then I sent SIGCONT to the same group. The processes moved to S and to get_signal, frozen stayed at 1, and the ticks did not advance: job control let go, the freezer took over. Releasing them needed a separate cgroup.freeze=0, and only then did the counter climb from 5 to 15.
The reverse order is the same logic from the other side. I sent SIGSTOP to a frozen process: it stayed S, because the signal was waiting. Then I thawed the cgroup — the process turned T and the ticks did not move. The pending SIGSTOP took its turn once the freeze lifted. After SIGCONT the counter ran again.
Practical takeaway: when you thaw a frozen group, do not assume its processes start running. There may be another layer in between, and cgroup.events will not show it to you.
A child cgroup cannot release itself
On the hierarchy side the documentation is clear: a cgroup can be frozen by its own setting or by the setting of any ancestor, and if any ancestor is frozen it stays frozen. In the lab the user-interface side of that turned out to be the more interesting part.
I froze the parent and looked at the child:
h2/cgroup.freeze = 0
h2/cgroup.events: populated 1 frozen 1
The child says "I issued no freeze order" (cgroup.freeze = 0) and "but I am frozen" (frozen 1); read together, that makes sense. Then I wrote 0 into the child:
exit code of writing 0 to h2: 0
1 s later tick: before=4 now=4
h2 events: populated 1 frozen 1
The write returned success. Nothing changed. I would have expected EBUSY or EPERM here; the documentation defines no error code for that write and the file quietly accepts the value — because what you write is that cgroup's own setting, while the effective state is decided by the whole tree. That is exactly the trap when scripting: exit code 0, nothing done. The right thing to check is not the exit code but cgroup.events.
The converse holds too: writing 1 to the child and 0 to the parent left the parent at frozen 0, the child at frozen 1, and the process frozen. Its own setting stays put.
One last detail: under a frozen cgroup, mkdir works and rmdir on an empty child works. The documentation already says so, and the lab matches — freezing holds the processes, it does not lock the tree.
What I could not measure
The documentation says freezing "may take some time"; the design is two-stage, the write returns immediately and frozen 1 arrives later. I tried to time that gap and the numbers are not good enough to publish.
As medians of five runs: empty cgroup 0.87 ms, three sleeping worker shells 1.87 ms, fifty sleeping processes 1.73 ms, two CPU-burning processes 2.01 ms. The floor of the rig itself — a single cgroup.events read — is 0.68 ms. In other words, all my "freeze duration" numbers sit in the same range as the noise of a read that freezes nothing. There is no visible growth with process count either; the fifty-process run came out below the three-worker one, and one run jumped to 16.3 ms. The real task count is looser than the labels too: each worker shell's transient sleep child is born into the same cgroup, and one of the fifty-process runs measured 99 processes in the group.
A cleaner piece of evidence: across the 26 runs that printed the polling counter, the counter came out 0 every single time. The first read after the write already saw frozen 1. At shell level I could not catch that transition window even once. This does not mean freezing always takes microseconds — if a task is not sleeping but parked in a long uninterruptible wait, a longer transition is exactly what you would expect, and the D state from the khungtaskd article is that very family. My lab had no such task, so this article claims no number for freeze duration. The thaw side was never timed at all.
Three more things went unmeasured, and all three are among the first questions production asks. First, systemd: the systemctl freeze documentation says it uses the same cgroup freezer, that it works only on the unified hierarchy, and that a unit is thawed automatically right before a job runs against it — how closely that automatic thaw tracks cgroup.events is not something this run tried. Second, supervisor notification: the probe I wrote — to check whether a parent sees SIGSTOP but not a freeze — returned an inconsistent status value, so this article makes no claim about that difference. Third, what a frozen process still holds: file locks, open TCP connections, half-finished database transactions. Freeze a group "for inspection" and all of that stays exactly where it is, while the other end of each connection reads it as an outage.
One thing also stays open: the cgroup.stat.local / frozen_usec field from the docs does not exist on 6.10, and in the source the freezer's own counter is named frozen_nsec and is updated only on thaw. You cannot read the duration of an ongoing freeze from that counter.
Which one, when
The short list the measurements left me with:
-
To halt a whole tree completely, use the freezer: there is no pid-enumeration race, and members that arrive later freeze too. But writing to
cgroup.freezetakes privilege, and from inside an ordinary container that file may not be yours to touch. -
For a single process by hand, SIGSTOP is enough; every tool reads the letter
T. -
If you write monitoring, do not trust the state letter for "frozen". The right source is the
frozenfield incgroup.events, and per the documentation that file can be watched withpoll/inotify— you do not need the busy loop my lab script used. On the process side the distinguishing mark iswchan. -
If you write scripts, do not read a successful write to
cgroup.freezeas success: if an ancestor is frozen, the write returns fine and does nothing. -
When releasing, do not assume a single layer. After you see
frozen 0, a process can still beTbecause of SIGSTOP, andwchanis what tells you which layer holds it. -
In an emergency you do not need to thaw a frozen group; SIGKILL and
cgroup.killboth work on one.
Conclusion
The story of these two mechanisms is about the same fact being kept in two different places. SIGSTOP changes the process's own state; the freezer parks the task at a sleep site inside the signal path and keeps the record in the cgroup. Because neither cancels the other, they can stack: lift one and the process does not start running, it simply hands its turn to the other.
Stopping a process is easy. Knowing that you really stopped it, and what is doing the stopping, takes reading two different files.
Top comments (0)