This blog is for personal reference of CSAPP Ch 10 Reference • System-Level I/O
1. The Core Idea - Everything is a File
Unix's most powerful design decision: everything is a file. Regular files, directories, sockets, pipes, terminals, devices - all of them are represented through the same interface: the file descriptor. The same read() and write() calls work on all of them. One interface for all I/O.
| CORE IDEA | A file descriptor is just an integer - a handle the kernel gives you when you open any I/O resource. Behind that integer, the kernel maintains all the actual state. Your program only ever sees the integer and uses it to tell the kernel what to operate on. |
|---|
2. File Descriptors - The Foundation of Unix I/O
2.1 What a File Descriptor Actually Is
A file descriptor (fd) is a small non-negative integer (0, 1, 2, 3...) that serves as an index into the kernel's per-process descriptor table. When you call open(), the kernel: finds or creates the resource, sets up internal tracking state, assigns the next available integer slot in your process's table, and returns that integer to you.
The integer itself contains no information - it's just a key. All the real state lives in the kernel.
2.2 The Three-Table Model - What the Kernel Maintains
The kernel maintains three separate tables to track open files. Understanding this model is essential for understanding fork(), dup(), and shared file positions.
Key detail: the open file table entry stores the current file position (offset). This is why two processes sharing the same open file table entry share the same position - and this is exactly what happens after fork().
2.3 The Three Pre-Opened File Descriptors
Every process is born with three file descriptors already open, set up by the shell before your program starts:
| fd | Name | Connected to | Purpose |
|---|---|---|---|
| 0 | stdin | Keyboard (or pipe input) | Standard input - where programs read user input from |
| 1 | stdout | Terminal screen (or pipe output) | Standard output - where printf/print writes to |
| 2 | stderr | Terminal screen | Standard error - always unbuffered, for error messages |
Why this matters: shell redirection (ls > output.txt) works by closing fd 1 and reopening it pointing to output.txt before exec-ing ls. The program writes to fd 1 as always - it never knows fd 1 was redirected. The abstraction is perfect.
2.4 File Descriptor Lifecycle
| CRITICAL BUG | Using a file descriptor after closing it (use-after-close) is one of the nastiest bugs in systems code. The integer may have been reused by the kernel for a completely different resource - reads/writes silently operate on the wrong thing. Always close fds exactly once, and never use them after closing. |
|---|
2.5 File Descriptor Limits
File descriptors are a finite OS resource. Each process has a per-process limit (RLIMIT_NOFILE, typically 1024 by default, often raised to 65536+ in production). The system also has a total limit across all processes.
What happens when you hit the limit: open() and accept() start returning -1 with errno = EMFILE (too many open files). Your server can no longer accept new connections. This is a real production failure mode.
Common cause: a server that opens connections or files and never closes them - fd leak. The server works fine for hours, then suddenly starts failing all new connections once it hits the fd limit.
3. Unix I/O - The Raw Syscall Interface
3.1 open() - Opening or Creating a File
Always check the return value: if open() returns -1, errno tells you why (file not found, permission denied, fd limit reached, etc.). Never assume open() succeeded.
3.2 read() - Reading Bytes
Critical: read() is NOT guaranteed to return n bytes. It returns however many bytes are available up to n. This is called a short count. Correct code must loop until all bytes are received.
3.3 Short Counts - The Most Important Detail About read()
Short counts (read() returning less than n bytes) happen in all of these cases:
- EOF on a regular file: fewer than n bytes remain before end of file
- Network sockets: TCP delivers data in chunks. One send() on the sender side may result in multiple read() calls needed on the receiver side. This is the most common networking bug.
- Interrupted by a signal: a signal arrives while read() is blocking. Returns early with errno = EINTR. Must retry.
- Pipe or terminal: kernel returns as soon as any data is available, not when n bytes are ready
- Kernel buffer boundary: kernel's internal buffer has less than n bytes available right now
Correct handling - the robust read loop:
| REAL WORLD | Every production network library implements this loop internally - Go's net package, Rust's TcpStream, Tokio's AsyncReadExt. When you call Read() in Go or read() in Rust on a TcpStream, the library handles short counts for you. But understanding why this loop is necessary is essential for understanding what those libraries are protecting you from. |
|---|
3.4 write() - Writing Bytes
write() can also return short counts - less than n bytes written - especially on sockets when the kernel's send buffer is full. Same loop pattern required for robust writing.
write() returning success ≠ data received by other end. On sockets, write() returning means data is in the kernel's socket send buffer - it has NOT been sent over the network yet. The TCP stack sends it when it decides to.
3.5 close() - Releasing a File Descriptor
Always close() every fd you open. On network servers especially: failing to close connected socket fds causes fd leaks that eventually exhaust the process's fd limit and prevent new connections.
3.6 lseek() - Repositioning the File Offset
lseek() is not valid on sockets or pipes - they are sequential streams with no seekable position. Calling lseek() on a socket returns -1.
4. Buffers - What They Are and Why They Exist
4.1 What a Buffer Is
A buffer is just a chunk of memory - an array of bytes - used as a temporary holding area to smooth out mismatches between how fast data is produced and how fast it is consumed, or between different chunk sizes on each side.
Example: keyboard input arrives one keypress at a time. Your program wants to read whole lines. The buffer accumulates individual keypresses until a newline arrives, then your program reads the whole line at once.
4.2 Buffers at Multiple Levels
Data passes through several buffer layers on the way to disk or network. Each layer is independently managed:
What each flush call does:
- fflush(fp) - flushes libc's user-space buffer → sends data to kernel via syscall
- fsync(fd) - tells kernel to flush page cache → forces data to physical disk hardware
- Neither guarantees survival of a power failure unless the hardware buffer also flushes (requires specific disk config)
4.3 The Three Buffering Modes
| Mode | When it flushes | Used for | Risk |
|---|---|---|---|
| Fully buffered | When buffer is full (typically 8KB) | Regular files, stdout redirected to file | Data in buffer lost if process crashes before flush |
| Line buffered | When newline \n is seen | stdout when connected to a terminal | Partial lines not seen until newline |
| Unbuffered | Immediately on every write | stderr (always), raw Unix I/O | More syscalls, but data never stuck in buffer |
| WHY stderr IS UNBUFFERED | Error messages need to appear immediately - especially right before a crash. If stderr were buffered, the error message that explains the crash might be lost in an unflushed buffer when the process dies. Unbuffered stderr guarantees error output always appears, regardless of what happens next. |
|---|
4.4 The Partial Buffer Problem
If your buffer is 10KB and you write 25KB of data:
| PRODUCTION CONCERN | Databases and write-ahead logs explicitly call fsync() after critical writes, not trusting buffering to flush at the right time. Log libraries in production often flush after every line. The performance cost is real but acceptable - losing the last 100 log lines before a crash is unacceptable for debugging. |
|---|
5. Standard I/O vs Unix I/O - The Critical Comparison
5.1 Why Standard I/O Exists
Every read() and write() syscall crosses the user→kernel boundary - a mode switch costing ~1-10 microseconds. Reading a 1MB file one byte at a time with raw read() = 1,000,000 syscalls. Standard I/O's user-space buffer reduces this to ~122 syscalls for the same file (1MB / 8KB buffer size = ~122 refills).
| Property | Unix I/O (read/write) | Standard I/O (fread/fwrite/FILE*) |
|---|---|---|
| Buffering | None - every call = syscall | User-space buffer (~8KB), batches syscalls |
| Performance (files) | Slow for small frequent reads | Fast - minimizes syscall count |
| Performance (sockets) | Correct and safe | DANGEROUS - buffer may delay sends |
| Control over when I/O happens | Precise - you control every syscall | Indirect - libc decides when to flush |
| Thread safety | fd has no state - safe | FILE* has internal locks - overhead |
| Use for sockets | Always use this | Never mix with sockets directly |
| Use for regular files | Fine, but more verbose | Preferred for text/line I/O |
5.2 Why Standard I/O Is Dangerous on Sockets
This is the most important practical distinction for network programming:
Problem 1 - Buffered writes may never send: if you fprintf() to a socket's FILE*, the data sits in libc's buffer. It only gets sent when the buffer fills up or you explicitly fflush(). Meanwhile the other end is waiting. This causes deadlocks where both sides are waiting for data that's stuck in an unflushed buffer.
Problem 2 - Two buffers on the same fd: if you mix read() and fread() on the same fd, Standard I/O's internal buffer may have consumed bytes that your read() call doesn't know about. The two buffers get out of sync. This causes data corruption that is very hard to debug.
| RULE | On network sockets: always use raw Unix I/O (read/write syscalls) or a library that implements its own socket-aware buffering (like Go's bufio or Rust's BufReader on TcpStream). Never use FILE* / Standard I/O directly on sockets. |
|---|
6. The RIO Package - Robust I/O
RIO is CSAPP's teaching library that solves two problems simultaneously: short count handling (robust reads/writes that loop until done) and socket-safe buffering (buffered line reading without using FILE*).
Important context: RIO is a teaching library, not a production library. You won't use csapp.h in real projects. But the concepts RIO implements are exactly what Go's bufio and Rust's BufReader implement. Understanding RIO means understanding those.
| RIO Function | What it does | Go equivalent | Rust equivalent |
|---|---|---|---|
| rio_readn(fd, buf, n) | Read exactly n bytes, loop on short counts | io.ReadFull(conn, buf) | read_exact() on TcpStream |
| rio_writen(fd, buf, n) | Write exactly n bytes, loop on short counts | conn.Write(buf) | write_all() on TcpStream |
| rio_readlineb(rp, buf, n) | Read one line (until \n), buffered, socket-safe | bufio.ReadString('\n') | BufReader::read_line() |
| rio_readnb(rp, buf, n) | Read n bytes from internal buffer, refill as needed | bufio.Read(buf) | BufReader::read_exact() |
| KEY INSIGHT | rio_readlineb reads one byte at a time from its internal buffer, refilling from the socket via one read() syscall when the buffer runs low. This is how you efficiently read line-delimited protocols (like HTTP headers) over a TCP socket without using FILE* or doing a syscall per byte. |
|---|
7. File Metadata - stat()
The stat() syscall retrieves metadata about a file - information stored in the file's inode on disk, not in its content.
File type from st_mode: use macros to check - S_ISREG() (regular file), S_ISDIR() (directory), S_ISSOCK() (socket), S_ISFIFO() (pipe)
| REAL WORLD | stat() is how tools like ls -l get file sizes and permissions, how web servers determine Content-Length headers, and how build systems like make decide if a file has changed (by comparing st_mtime). File size from stat() is also how you know how many bytes to read from a file before seeing EOF. |
|---|
8. File Sharing - fork() and Descriptor Inheritance
8.1 What fork() Does to File Descriptors
When fork() is called, the child gets its own COPY of the parent's descriptor table. But both copies point to the SAME entries in the kernel's open file table - they share file positions and flags.
Consequence - shared file position: if the parent reads 100 bytes (advancing shared position from 0→100), the child's next read starts at byte 100, not 0. They share one position counter in the open file table entry.
8.2 The Fork-Per-Connection Pattern - Close the Right fds
In a fork-per-connection server, two file descriptors exist at fork() time:
- listenfd: the listening socket (parent's job - accept new connections)
- connfd: the just-accepted connection socket (child's job - handle this client)
Why each close is mandatory:
- Child closes listenfd: child has no need for the listening socket. Holding it open inflates the ref count unnecessarily. If enough children hold it, the listening socket's resources are never freed even if the parent closes its copy.
- Parent closes connfd: parent won't talk to this client - the child will. If parent doesn't close it, the connection's ref count stays at 2. Even after the child finishes and closes its copy (ref count→1), the TCP connection CANNOT close (no FIN sent to client) because the parent still holds a reference. Connection hangs open indefinitely.
| REAL WORLD | Forgetting to close unused inherited fds is the #1 cause of 'too many open files' errors in fork-based servers. A server handling thousands of connections per hour that doesn't close inherited fds accumulates one leaked fd per connection, eventually exhausting the fd limit and crashing. Always close every fd your process inherits but doesn't need. |
|---|
8.3 dup2() - Redirecting File Descriptors
dup2() is also how pipes work (ls | grep foo) - the shell creates a pipe, uses dup2() to connect ls's stdout to the pipe's write end and grep's stdin to the pipe's read end, then forks both processes. Each writes/reads fd 1/0 as always.
9. I/O Redirection and Pipes
9.1 How Shell Redirection Works - Step by Step
9.2 How Pipes Work - Step by Step
| KEY INSIGHT | The parent must close both ends of the pipe after forking. If the parent keeps the write end open, grep (reading the read end) will never see EOF - it blocks forever waiting for more input that the parent might send, even though ls has already finished. This is a real deadlock that happens when you forget this close. |
|---|
10. How Ch 10 Connects to Ch 11 and Ch 12
| Ch 10 Concept | How it appears in Ch 11 (Networking) and Ch 12 (Concurrency) |
|---|---|
| File descriptors for everything | Ch 11: sockets are file descriptors. accept() returns a fd. read()/write() on socket fds works identically to files - same interface, different behavior underneath. |
| Short count handling / robust read loop | Ch 11: essential for every network server. TCP delivers data in chunks - raw read() on a socket almost never returns exactly what you asked for. The robust read loop is required code in every correct network server. |
| Standard I/O dangerous on sockets | Ch 11: never use FILE* on sockets. Use raw read()/write() or a socket-aware buffering layer (bufio in Go, BufReader in Rust). |
| Fork + fd inheritance + close unused fds | Ch 11: fork-per-connection server pattern. Child closes listenfd, parent closes connfd - both mandatory to avoid fd leaks and hung connections. |
| dup2() for redirection | Ch 11: used internally by shells and process supervisors to connect processes via pipes. Foundation of how data flows between processes in Unix. |
| fd limit (EMFILE) | Ch 12: a concurrent server handling many connections simultaneously uses one fd per connection. fd limits define the maximum concurrent connections your server can hold open at once. |
| Kernel socket send/receive buffers | Ch 12: when a send buffer is full (slow receiver), write() blocks - this is backpressure. Event-driven servers (epoll/kqueue, Ch 12) use non-blocking I/O specifically to avoid blocking here. |
11. Quick Reference - Things to Remember Cold
File descriptor facts
- fd = small non-negative integer, index into per-process descriptor table
- fd 0/1/2 = stdin / stdout / stderr - pre-opened by shell
- open() = creates open file table entry, returns fd integer
- close() = removes descriptor table entry, decrements ref count
- Use-after-close = undefined behavior - fd may be reused for different resource
- fd limit = 1024 default (RLIMIT_NOFILE), raise to 65536+ in production
read() / write() guarantees
- NOT guaranteed to return n bytes - always loop for correct behavior
- Returns 0 = EOF (or connection closed on socket)
- Returns -1 = error, check errno. EINTR = signal interrupted, retry
- write() returning = data in kernel buffer, NOT data received by other end
Buffering modes one-liner each
- Fully buffered: flushes when buffer full. Regular files, redirected stdout.
- Line buffered: flushes on newline. stdout to terminal.
- Unbuffered: every write = immediate syscall. stderr, raw Unix I/O.
Buffer layers
- fflush() → flushes user-space libc buffer → kernel
- fsync() → flushes kernel page cache → physical disk
- Crash before fflush = data in user buffer is lost forever
Standard I/O vs Unix I/O - one rule
- Files: prefer Standard I/O for text/line work (efficient, convenient)
- Sockets: always raw Unix I/O - never Standard I/O directly on sockets
fork() and file descriptors - two rules
- Parent and child share the same open file table entry → shared file position
- Always close every fd your process inherits but doesn't use → prevents fd leaks, hung connections, and zombie resources
dup2() in one line
- dup2(oldfd, newfd) = make newfd point to same resource as oldfd. Foundation of shell redirection and pipes.
CSAPP Ch 10 Reference • System-Level I/O
Top comments (0)