DEV Community

Naz Quadri
Naz Quadri

Posted on • Originally published at nazquadri.dev

Your Process Doesn't Exist Alone

#linux #programming #systems #tutorial

Your Process Doesn't Exist Alone

Sessions, Process Groups, and Why Ctrl-C Kills the Right Thing

Reading time: ~13 minutes

You pressed Ctrl-C and the program stopped. Exactly the right program. Not its parent. Not the shell you typed from. The one you were running.

That probably felt unremarkable. It shouldn't. The kernel had to figure out which processes — out of a structured hierarchy on your machine — deserved that signal. It got it right every time you've ever tried. There is infrastructure specifically designed to make that work, and it involves three layers of grouping you've never had to think about.

Let's look at what's actually happening.

The Problem Ctrl-C Has to Solve

Here's a scenario. You type this:

tar czf archive.tgz big_directory/ | pv | gpg --encrypt > archive.tgz.gpg
Enter fullscreen mode Exit fullscreen mode

Three processes. A pipeline. You get bored waiting and press Ctrl-C.

Which one should die? All three. They're one operation from your perspective. The terminal agrees.

Consider: you've spawned those three processes from a bash shell. Bash itself is running. Maybe there are other background jobs. Maybe there's a long-running process you started earlier with &. When you press Ctrl-C, none of those should die.

The kernel needs a way to know which processes are "the foreground thing you're doing right now" and which are everything else. The mechanism it uses is the process group.

Process Groups: The First Layer

Every process belongs to a process group, identified by a PGID (process group ID). When you run a pipeline like tar | pv | gpg, the shell puts all three into the same process group. When you run a single command, that command becomes a process group of one.

The PGID is inherited through fork(). When a process forks, the child starts in the same process group as the parent. To create a new process group, a process calls setpgid().

A pipeline works like this:

Shell (PGID=1000)
    │
    ├─ fork() → tar   (PGID=1001, also leader since tar's PID=1001)
    ├─ fork() → pv    (PGID=1001, setpgid'd to join tar's group)
    └─ fork() → gpg   (PGID=1001, setpgid'd to join tar's group)
Enter fullscreen mode Exit fullscreen mode

The shell then tells the terminal: "the foreground process group is now 1001." When you press Ctrl-C, the kernel sees that sequence, generates SIGINT, and delivers it to every process in group 1001. All three die. The shell — in group 1000 — is unaffected.

That's why Ctrl-C kills the right thing. And that's why Ctrl-C sometimes doesn't work — if a program installs a SIGINT handler and doesn't propagate the signal to its children, or puts its children in a different process group, Ctrl-C kills the parent but leaves the children running. You've seen this: the prompt comes back, but there are still processes chewing through CPU.

Process groups and the terminal — SIGINT hits the pipeline, not the shell

There's one process in each group that the kernel considers the group leader — the one whose PID equals the PGID. For a pipeline, bash usually makes the first process in the pipeline the leader. If the leader dies, the group doesn't disappear; the other members keep running. It's just a label.

Sessions: The Second Layer

Process groups answer "which processes are this foreground job." But there's a bigger question: which foreground job is active right now, and who owns the terminal?

That's where sessions come in.

A session is a collection of process groups. All the process groups you create during a login session belong to the same session. When you open a terminal and start typing, every command you run — every pipeline, every background job, everything — is in the same session.

Sessions have an ID too: the SID. The first process to call setsid() creates a new session and becomes the session leader. Its PID becomes the SID.

The session has one more piece: the controlling terminal. This is the terminal device that delivers job control signals — SIGINT from Ctrl-C, SIGTSTP from Ctrl-Z, SIGHUP when the terminal closes. Exactly one session owns a given terminal at a time, and the terminal knows which process group is in the foreground.

Session (SID=999, controlling terminal: /dev/pts/3)
    │
    ├── Process Group 999 (shell — the session leader's group)
    │       └── bash (PID=999, PGID=999)
    │
    ├── Process Group 1001 (foreground: tar | pv | gpg)
    │       ├── tar
    │       ├── pv
    │       └── gpg
    │
    └── Process Group 1002 (background job: long-running-thing &)
            └── long-running-thing
Enter fullscreen mode Exit fullscreen mode

The terminal keeps a pointer to the foreground process group. SIGINT goes there. SIGTSTP goes there. When you run a command in the foreground, the shell calls tcsetpgrp() to tell the terminal which group has the focus. When the command finishes, the shell takes it back.

Run ps -ej and look at the SID and PGID columns. Every process is accounted for — including daemons, which show ? in the TTY column because they have no controlling terminal.

The Controlling Terminal: What It Actually Is

"Controlling terminal" sounds abstract. It's not.

A controlling terminal is a file descriptor — specifically a TTY or PTY device — that a session is attached to. It's the device that knows how to translate "the user pressed Ctrl-C" into a signal, and "the user pressed Ctrl-Z" into a different signal.

When a session leader opens a TTY device for the first time, that TTY becomes the controlling terminal for the session. The kernel records this association in both directions: the TTY knows its session, and the session knows its TTY.

Here's what "controlling terminal" buys you:

  1. Job control signals. Ctrl-C, Ctrl-Z, Ctrl-\ all generate signals via the controlling terminal's line discipline.
  2. SIGHUP on terminal close. When the controlling terminal's last master side is closed, the kernel sends SIGHUP to the session leader and the foreground process group.
  3. Background I/O protection. If a background process tries to read from or write to the controlling terminal without being in the foreground group, it gets SIGTTIN or SIGTTOU. The process stops, you see "Stopped" in the shell, and you have to foreground it. Programs that reach for /dev/tty directly (rather than stdout) are explicitly targeting the controlling terminal — and the kernel enforces access control on it.

PTY device as the hub — sessions, process groups, and signals

nohup and disown: Why They Exist

Two commands make more sense once you know what a session is.

When your SSH connection drops, the terminal closes. The kernel notices: the master end of the PTY is gone. It sends SIGHUP to the session leader and the foreground process group. If your long-running job is in that session, it gets SIGHUP. The default action for SIGHUP is: die.

This is not a bug. It's the intended behavior. The terminal is gone, so the processes that depended on it should clean up. The problem is that sometimes you want a process to keep running after you log out.

nohup solves this by making the child process ignore SIGHUP before exec. It also redirects stdout and stderr to nohup.out since the terminal won't exist to receive output. The process is still in your session, still has the same controlling terminal, it just won't die when SIGHUP arrives.

# nohup sets SIGHUP to SIG_IGN before exec, and redirects output
nohup long-running-job &
Enter fullscreen mode Exit fullscreen mode

disown is a bash shell builtin that removes the job from the shell's job table. When the terminal closes and the shell receives SIGHUP, it re-delivers SIGHUP to its job groups. disown removes the job from that list, so the shell won't SIGHUP it on terminal close.

Neither of these is a clean solution. The process is still in the session. If the session leader exits and the kernel sends SIGHUP through the normal terminal-close path, your ignored-SIGHUP process might still get caught. And the process still has the closed terminal as its controlling terminal, which causes problems if it ever tries to read from stdin.

These are bandaids. The real solution is what comes next.

That's why ssh remote-host some-command & is dangerous. The some-command process starts on the remote host inside your SSH session. When your SSH connection drops, SIGHUP fires. If some-command doesn't handle SIGHUP, it dies. The solution is either nohup, or — better — put it inside a tmux session on the remote host.

setsid(): Cutting the Cord

The real way to detach a process from a terminal is setsid().

setsid() creates a new session. The calling process becomes the session leader of a brand new, empty session. This new session has no controlling terminal — and importantly, a session only gets a controlling terminal when the session leader explicitly opens a TTY. Without O_NOCTTY, opening a TTY device gives you a controlling terminal whether you want one or not. So if you never open a TTY, you never get one. No controlling terminal means no SIGHUP, no job control signals, no SIGTTIN/SIGTTOU.

One constraint: setsid() fails if the calling process is already a process group leader. This is why the daemonization recipe forks first — the child is not a group leader, so setsid() succeeds.

This is what daemonization does. The classic daemon recipe:

// Parent forks, then exits — child is now orphaned
// (child is not a process group leader, so setsid() will work)
if (fork() > 0) exit(0);

// Child calls setsid() — new session, no controlling terminal
setsid();

// Fork again so we're NOT the session leader
// (without O_NOCTTY, opening a TTY would give us a controlling terminal)
if (fork() > 0) exit(0);

// Now we're a non-session-leader process in a session with no
// controlling terminal. SIGHUP cannot reach us through the terminal.
// We are truly detached.
Enter fullscreen mode Exit fullscreen mode

The double-fork: by forking a second time, we become a child of the session leader — still in the new session, but not the leader. A non-leader can never become a session leader, so it can never acquire a controlling terminal by opening a TTY.

That's why daemonization requires a double fork. You can't get to truly "no controlling terminal" in a single step if you're a session leader. The second fork is what closes that loophole.

setsid() is also what PTY supervisors call in the child process before exec. When a supervisor spawns a child with a PTY, the child calls setsid() first, then opens the PTY slave. That open() call is what gives the child its controlling terminal — the PTY slave. This is intentional. We want the child to have a controlling terminal. We just want that terminal to be the PTY we control, not whatever terminal the supervisor was launched from.

Why tmux and screen Survive Terminal Closure

Now all the pieces are in place to understand something that confused most of us for years.

You connect to a server over SSH. You start tmux. You run a bunch of stuff inside tmux. Your SSH connection drops. You reconnect and attach to the same tmux session. Everything is still there.

How?

When you start tmux, it forks a tmux server. That server calls setsid(). It is now the session leader of its own session with no controlling terminal. It is not attached to your SSH terminal at all. When your SSH session dies, SIGHUP goes to your SSH client's session and the processes in it. The tmux server is not in that session. It's untouched.

The tmux server holds the master ends of PTYs for all your "windows." Your shells and programs run on the slave ends. Those PTY slaves are controlled terminals for their sessions — not for yours.

When you reconnect and run tmux attach, a new tmux client process starts. It connects to the tmux server over a Unix socket. The server starts forwarding the PTY master output to the new client, and the client starts forwarding your keystrokes to the server. From your perspective, it looks like you reconnected. From the shell inside tmux's perspective, absolutely nothing changed.

Your SSH session (terminal = /dev/pts/0)
    │
    └── tmux client ─────────Unix socket────► tmux server (setsid, no ctrl terminal)
                                                    │
                                              PTY master ─────► /dev/pts/7
                                                                      │
                                               bash (ctrl terminal = /dev/pts/7)
                                                                      │
                                               your stuff, running fine
Enter fullscreen mode Exit fullscreen mode

When your SSH terminal closes, the left side of this diagram disappears. The right side doesn't care.

That's why the right workflow on a remote server is tmux first, then work inside tmux — not nohup or disown. The session architecture guarantees survival. The workarounds don't. tmux isn't magic. It's setsid() and a Unix socket.

The Signal Routing Map

Let's put the whole picture together. When you press a key combination in your terminal:

Ctrl-C → The terminal line discipline intercepts it. It sends SIGINT to the foreground process group of the terminal's session.

Ctrl-Z → The terminal line discipline intercepts it. It sends SIGTSTP to the foreground process group.

*Ctrl-\* → Same routing, different signal: SIGQUIT. Kills with a core dump.

Terminal close → The kernel sends SIGHUP to the session leader and the foreground process group.

Background process tries to read from terminal → Kernel sends SIGTTIN to that process's process group.

Signal delivery reference — keys, signals, and targets

Notice what's absent: there is no "kill just this one process from the terminal" mechanism. That's why daemons show ? in the TTY column of ps aux — a daemon has no controlling terminal, and the kernel records that as ?. The mysterious ? that used to look like noise is a direct signal: this process is properly detached. The terminal deals in groups. If you want to kill exactly one process, you use kill(pid, signal) directly — by PID. The terminal doesn't do individual targeting.

This is why kill -9 $$ in a subshell does what you expect but kill -9 $(jobs -p) might not — jobs -p returns PGIDs, not PIDs, and if you're not careful you're sending signals to entire groups.

What This Means for Supervising Processes

A PTY supervisor is a process that holds the master end of a PTY and manages a child process's terminal lifecycle. When it forks the child, the child calls setsid(). This creates a new session. The child then opens the PTY slave — which becomes its controlling terminal. The child and everything it forks lives in that session. Job control signals go to that session's process groups. SIGHUP will go to that session if the supervisor closes the PTY master — and we can use that deliberately to tell the supervised process to clean up.

The supervisor itself is in its own session, not the child's. The child's terminal lifecycle is entirely under the supervisor's control.

This is exactly why nohup and disown feel janky — they're trying to get the benefits of this architecture without actually building it. They leave the process in the wrong session and just ask it to ignore signals. The proper solution is to put the process in its own session from the start, owned by a supervisor that manages its lifecycle explicitly.

Quick Recap

  • Every process belongs to a process group (PGID). Shells put pipeline members in the same group.
  • Ctrl-C and Ctrl-Z deliver signals to the foreground process group, not just one process.
  • Every process group belongs to a session (SID). A terminal is the controlling terminal of exactly one session.
  • When a terminal closes, SIGHUP goes to the session leader and foreground process group.
  • nohup ignores SIGHUP. disown removes the job from the shell's cleanup list. Both are workarounds.
  • setsid() creates a new session with no controlling terminal. This is how daemons, tmux, and PTY supervisors properly detach.
  • tmux survives disconnection because the server calls setsid() at startup and communicates via a Unix socket — it was never in your terminal's session.

Further Reading

  • man 2 setsid, man 2 setpgid, man 3 tcsetpgrp — the system calls that manage all of this directly
  • man 7 credentials — Linux's full documentation on PID, PGID, SID and how they interact
  • ps -ej — run this in a terminal with a few jobs running and watch the SID/PGID columns. Everything above becomes concrete immediately.
  • The TTY Demystified — Linus Akesson's deep dive, already recommended in earlier posts. The section on job control is directly relevant here.

I'm writing a book about what makes developers irreplaceable in the age of AI. Join the early access list →

Naz Quadri once killed a production job by closing his laptop. He blogs at nazquadri.dev. Rabbit holes all the way down 🐇🕳️.

Top comments (0)