DEV Community: Valentyn Kit

Rust's compile errors are other languages' production bugs

Valentyn Kit — Wed, 08 Jul 2026 12:51:35 +0000

Rust refused to compile two lines of mine last week: a Vec of floats and a call to .sort(). Run the equivalent in Python with one NaN in the list and sorted() returns normally, having produced a list that is not sorted. Nothing crashes, nothing warns; the wrong answer comes back looking like a right one.

I've been writing Rust daily for a few weeks now, rebuilding a C Redis clone in it and keeping notes on every fight I lose to the compiler. A pattern keeps repeating: I hit a refusal, decide the compiler is being difficult, dig into the why, and find a bug I have almost certainly shipped in another language.

This piece walks through four of those refusals, with the receipts, then closes with the place Rust goes quiet instead. Everything below was run on rustc 1.95.0, Python 3.14.3, and Node 25.9.0 on 2026-07-08; the outputs are pasted, not paraphrased.

The refusals come in grades, and I'll be honest about which is which: three are hard compile errors, the fourth is a warning you learn to stop scrolling past.

Floats have no total order

The two lines:

let mut v: Vec<f64> = vec![3.0, 1.0, 2.0];
v.sort();

error[E0277]: the trait bound `f64: Ord` is not satisfied

sort() requires Ord, a total order: for any two values, exactly one of less-than, equal, greater-than holds. Sorting algorithms are built on that contract. A merge step trusts its comparisons; a partition trusts that every element lands on one side of the pivot. Integers can keep that promise. Floats carry one value that can't:

let nan = f64::NAN;
nan < 1.0    // false
nan > 1.0    // false
nan == nan   // false

Rust encodes "this comparison can fail" in the type: partial_cmp returns an Option<Ordering>, and for NaN it returns None. So f64 gets PartialOrd and not Ord, and the sort refuses to build.

Here is the bug that refusal prevents. Python, same values, three input orders (Python 3.14.3):

>>> nan = float("nan")
>>> sorted([3.0, 1.0, 2.0, nan])
[1.0, 2.0, 3.0, nan]
>>> sorted([nan, 3.0, 1.0, 2.0])
[1.0, nan, 2.0, 3.0]
>>> sorted([3.0, nan, 1.0, 2.0])
[3.0, nan, 1.0, 2.0]

The first result looks perfect. The second has nan in the middle and 3.0 after it. The third came back in its original, unsorted order. sorted() returned normally all three times.

The mechanism is the broken contract: Python's sort trusts its comparisons, every comparison against nan answers false, and the damage depends on where nan happens to sit in the input. A test suite without a NaN in an unlucky position passes forever. Production data eventually finds the unlucky position.

JavaScript behaves the same way once you hand sort the numeric comparator everyone writes. (x, y) => x - y returns NaN for NaN pairs, which makes the comparator inconsistent, and ECMAScript makes the resulting order implementation-defined. On Node 25.9.0, [3, 1, 2, NaN] comes out [1, 2, 3, NaN] and [NaN, 3, 1, 2] comes out [NaN, 1, 2, 3]: NaN stays wherever it started.

C is the bluntest of the three: qsort with a comparator that doesn't impose a total order is undefined behavior outright (C11 7.22.5).

The fix costs one comparator, and it's what the type system was asking for all along:

v.sort_by(|a, b| a.total_cmp(b));

total_cmp is IEEE 754's totalOrder predicate, a genuine total order over every float bit pattern:

let mut v = vec![1.0, f64::NAN, f64::NEG_INFINITY, -f64::NAN, f64::INFINITY, -0.0, 0.0];
v.sort_by(|a, b| a.total_cmp(b));
// [NaN, -inf, -0.0, 0.0, 1.0, inf, NaN]

Negative NaN first, positive NaN last, negative zero before positive zero. You don't need to memorize the order; you need it to exist. If NaN in your data is itself a bug, v.retain(|x| !x.is_nan()) before sorting is the more honest fix. Either way, you decided.

Strings have no [0]

let s = String::from("café");
let c = s[0];

error[E0277]: the type `str` cannot be indexed by `{integer}`

Ask what s[0] should return and the refusal starts making sense. Rust strings are UTF-8 bytes; the é in "café" takes two of them, so the string is five bytes and four characters. "Element zero" has three defensible meanings: a byte, a codepoint, or the grapheme a reader would circle on paper. They agree on ASCII and diverge on everything else.

Other languages resolve the ambiguity by picking silently. JavaScript indexes UTF-16 code units, so "🦀"[0] is "\ud83e", half a surrogate pair that renders as garbage (Node 25.9.0; the string's .length is 2 for one visible crab). C hands you a raw byte. Python gives you a whole character in O(1) and pays in memory: since PEP 393, each string is stored at the width of its widest codepoint.

Rust makes you spell out which one you meant: s.as_bytes()[0] for the byte (99), s.chars().nth(0) for the character, and range slicing like &s[0..1] for bytes with a boundary check; &s[4..5] panics with "byte index 4 is not a char boundary; it is inside 'é'" instead of handing you half a character.

The error message even suggests .chars().nth() and .bytes().nth(). rustc can see perfectly well what you're after; it wants the ambiguity resolved in your code rather than in its defaults.

255 + 1 has three answers

This one is a ladder, not a single error. When the overflow is provable at compile time, it refuses outright:

let x: u8 = 255;
let y = x + 1;

error: this arithmetic operation will overflow
 = note: `#[deny(arithmetic_overflow)]` on by default

Make the value opaque at runtime (parse it from a string) and the same addition panics in a debug build, "attempt to add with overflow," and prints 0 in release. That release wrap is defined two's-complement behavior, written down in RFC 560, never undefined.

The C comparison matters here. Signed overflow in C isn't a wrap, it's undefined behavior (C11 6.5p5): the optimizer may assume it cannot happen and fold away the very check you wrote to catch it. Rust's split is a cost decision made honestly: checking every add costs a branch, so debug builds check everything and release builds wrap deterministically.

If wrap is wrong for your domain, set overflow-checks = true in the release profile, or name the semantics per call: wrapping_add, checked_add, saturating_add.

Eight bits genuinely cannot hold 256, so something has to give. Rust's stance is that the choice belongs in your code, and the debug panic is the reminder that you haven't written it down yet.

The map that never ran

This one is a warning, not an error, and it still cost me an afternoon: a println! I'd tucked inside a map for debugging never fired. The program ran clean, printed everything after it, and exited zero.

let v = vec![1, 2, 3];
v.iter().map(|x| {
    println!("ran {x}");
    x * 2
});
println!("done");
// prints only: done

Rust iterator adapters are lazy. map doesn't run the closure; it builds a small struct that waits for a consumer, a for loop or collect or sum. Mine had no consumer, so the work never existed.

The answer had been sitting in my terminal the whole time: "unused Map that must be used: iterators are lazy and do nothing unless consumed." I'd scrolled past it the way JavaScript trained me to, because there map runs eagerly and hands back a whole new array per stage.

Rust's laziness is why a five-adapter chain compiles into one loop with no intermediate allocations. The price is that describing the work and doing the work are two separate moments, and this warning fires when your program only ever did the first.

Where Rust goes quiet

I'd be selling you something if I stopped there, so here's the counterexample I hit the same week, in my Redis clone's append-only file.

BufWriter does flush its buffer on scope exit; Drop takes care of it. Drop, though, has no way to return a Result, so a failure in that final flush has nowhere to go and gets discarded. The standard library documents this openly and tells you to flush by hand first if you care. No compile error, no warning, nothing.

Even a successful flush() has only reached the kernel's page cache. Getting bytes onto the actual disk is sync_all(), fsync under the hood, and nothing calls it for you.

The type system polices what it can model, comparability and ownership above all. Durability policy, what your program is allowed to lose when the power dies, isn't in the types.

Rust erases most of the C discipline I needed for the same file (my own buffering, ? on every write, close on scope exit) and leaves the two decisions that determine what a crash can take from you: when to flush, and when to sync. Those stay yours, unannounced.

The shape of the trade

Every refusal above is one question in different clothing: what should happen on the case you didn't think you had? Python, JavaScript, and C settle it for you at runtime, each in its own accidental way. Rust asks before the program exists and won't move until you answer. The friction is real; it also lands at the exact moment the bug would otherwise have shipped.

Most of these came out of the Redis rebuild, and that project's durability layer, the append-only file where the quiet BufWriter almost bit me, is the next writeup.

Rebuilding my C Redis clone in Rust taught me more Rust than any tutorial

Valentyn Kit — Mon, 06 Jul 2026 18:32:20 +0000

I built a small Redis clone in C: a RESP parser, a command table, an append-only file for persistence. Recently I started building the same thing again in Rust, and rebuilding a project I had already finished has taught me more Rust than any from-scratch tutorial.

The reason is simple. The second time, the design is already solved. I know what the AOF has to guarantee, what the command table dispatches, what the parser must reject. So none of my attention goes to what to build. All of it goes to how Rust wants it built.

That turns the domain into a constant and the language into the only variable. Every difference I hit is pure signal about Rust, not noise about key-value stores.

The first difference shows up before any logic runs. In C, I built the substrate first: my own dynamic strings, my own hashmap, my own linked list. Hundreds of lines before a single command worked. In Rust, Vec, String, and HashMap are just there, so that whole layer disappears and I start at the actual command logic. A standard library quietly decides where your project even begins.

The sharper difference is in dispatch. In C it is a switch with argument counts I check by hand:

if (argc != 3) return err("wrong arg count");
switch (cmd) {
    case CMD_SET: return do_set(argv[1], argv[2]);
    case CMD_GET: return do_get(argv[1]);
    /* forget a case and it is a runtime bug */
}

In Rust the same dispatch is an enum and a match, and the compiler will not build until every case is handled:

match cmd {
    Command::Set { key, val } => self.set(key, val),
    Command::Get { key }      => self.get(key),
}

Same dispatch. One version cannot ship the missing-case bug I actually shipped in C.

If you already know a project cold, rebuild it in the language you are learning. You stop thinking about the problem and start feeling the language.

From one blocking accept() to epoll: a C TCP server up the I/O ladder, measured

Valentyn Kit — Tue, 30 Jun 2026 12:36:20 +0000

I connected one client to a blocking TCP server and held the socket open without sending a single byte. Then I connected a second client and sent it a line of text. The second client sat there for 1.51 seconds with no reply. It got its echo back one millisecond after I closed the first connection.

That 1.51 seconds is the reason the other six versions of this server exist.

Last week I wrote up why I rebuilt this server seven times: framework knowledge resets every few years, the layer underneath it compounds. That piece stayed at the level of outcomes. This one goes the other way, down into the code and the numbers.

The claims that matter here are the kind you can read a hundred times without being able to derive them. "select is O(n)." "epoll only hands you the ready fds." I had read both for years. I wanted to make my own machine say them out loud.

The target the whole exercise is built around is Dan Kegel's old C10K problem: how do you serve ten thousand clients at once on one server? Each of the seven versions hits a wall, and the wall is what names the next one.

The whole thing is one echo server written seven times, no libraries beyond libc, on GitHub. Every number below is from running it on macOS (Apple clang 21, darwin 25.4) on 2026-06-29. The binaries are built with AddressSanitizer and UBSan on, so read the absolute microseconds loosely. The structure is what holds.

Phase 01: blocking, and the 1.5 second stall

The first server is the one everybody writes first. Accept a connection, talk to it, close it, accept the next.

for (;;) {
    int client_fd = accept(server_fd, NULL, NULL);
    if (client_fd == -1) { perror("accept"); continue; }
    handle_client(client_fd);
    close(client_fd);
}

handle_client loops on read until the client hangs up. Both accept and read block: when there is nothing to do, the thread sleeps in the kernel. That is good for idle cost and fatal for everything else. While the server is parked in read waiting on client A, client B is sitting in the kernel's backlog queue, accepted by TCP but not yet by your code. The server never gets to it.

That is the stall I measured at the top. Client B waited 1.51 seconds, the exact length of time I held client A open, and was served the instant A closed. One idle connection freezes every other connection behind it. The file comment in the repo says it plainly: "Open two clients at once and watch the second one hang." I wanted the number, so I held the first one open on purpose.

The good news is the idle cost. With no clients connected, this server sits at 0.0% CPU. It is asleep in accept. Hold onto that, because the next version throws it away.

Phase 02: non-blocking, and a pegged core

The obvious fix: stop blocking. Put every socket in O_NONBLOCK mode and the syscalls return immediately with EAGAIN instead of sleeping. Now one thread can walk all its clients, try each one, and skip the ones with nothing ready.

for (;;) {
    accept_client(&s);
    for (int i = 0; i < s.count; i++) {
        if (handle_client(&s, i))
            i--; /* a client was swapped into i, re-check it */
    }
}

This works. No client can freeze the others anymore. It also has a problem you can hear with a fan. The loop never sleeps.

The two calls hide the busy-work: accept_client fires a non-blocking accept, and handle_client a non-blocking read. When every client is idle both return EAGAIN, the outer loop runs them again, and with nothing to do it spins as fast as the core allows, millions of EAGAINs a second.

I measured the same idle server two ways. Blocking, phase 01, no clients: 0.0% CPU. Non-blocking, phase 02, no clients: 98 to 99% CPU. A whole core burned to accomplish nothing. That is not a tuning problem, it is the design. Polling means asking "anything yet?" forever, and the answer is almost always no.

(The i-- in that loop is a small trap worth pointing at. When a client disconnects, the code swaps the last client into its slot to keep the array dense, then decrements i so the loop re-checks the slot the swap just refilled. Miss that and you skip a client every time one leaves.)

Phase 03: select, and two scars from the 1980s

Phase 02 had the right shape and the wrong instruction. We do not want to ask "anything yet?" in a loop. We want to tell the kernel "wake me when one of these is ready" and go to sleep until it is. That instruction is select.

/* select() overwrites the set, so hand it a fresh copy each loop. */
fd_set readfds = s.active_fds;
int ready = select(s.max_fd + 1, &readfds, NULL, NULL, NULL);

The idle core comes back instantly. The server sleeps in select until a client actually does something, exactly like phase 01 slept in accept, except now it is waiting on the whole set at once. This is the real idea in the whole ladder: readiness notification. One wait, one wakeup, for many connections.

select carries the idea wrapped in two pieces of 1980s baggage. The first is in that comment: select overwrites the set you give it with the subset that is ready, so you keep a master copy and hand it a throwaway duplicate every single loop. The second is harder:

if (fd >= FD_SETSIZE) {
    fprintf(stderr, "fd %d >= FD_SETSIZE, rejecting\n", fd);
    close(fd);
    return;
}

The fd set is a fixed-size bitmask. On this machine FD_SETSIZE is 1024, which I confirmed by compiling a one-line program that prints it. A connection whose file descriptor lands at or above 1024 cannot be represented in the set at all, so the server has to refuse it.

On top of the cap, every wakeup scans the whole bitmap up to max_fd, so the cost is O(n) in the number of connections you are watching, whether one of them is active or a thousand.

That is the end of the road for C10K on select. The bitmask caps you at 1024, a tenth of the way to the goal, and the per-wakeup scan would tax you even if it did not.

Phase 04: poll, one scar gone

poll is select with the bitmask swapped for an array of struct pollfd, one entry per connection. That removes the FD_SETSIZE cap: the array is as big as you make it.

It does not remove the other scar. The kernel still walks every entry on every call, and you still hand it the full list each time. Better ceiling, same complexity: O(n) per wakeup, still proportional to the connections you hold, not the ones doing anything.

To actually fix the cost, the kernel has to remember what you care about between calls.

Phases 05 and 06: kqueue and epoll, register once

This is the jump. Instead of handing the kernel the entire list of connections on every wakeup, you register each connection once, and from then on the kernel hands you back only the ones that are ready.

struct kevent events[MAX_EVENTS];
for (;;) {
    int n = kevent(kq, NULL, 0, events, MAX_EVENTS, NULL);
    for (int i = 0; i < n; i++) {
        int fd = (int)events[i].ident;
        if (fd == server_fd) accept_client(kq, server_fd);
        else                 handle_client(kq, fd);
    }
}

kevent returns n, the number of ready events, and you loop over exactly those. The repo comment is the clearest one-line statement of the payoff in the whole project: "with 10,000 idle connections and one active, kevent() hands you exactly that one." The work per wakeup is O(ready), not O(registered). The fd cap is gone too.

I measured the readiness behavior, not the throughput. Three clients connected at once, each sending a line, each got its echo back in 0.1 to 0.2 milliseconds, overlapping, not serialized. That is the property phase 01 could not have at any speed: progress on many connections inside one loop, with the loop asleep whenever nothing is happening.

kqueue is the BSD and macOS interface. Linux spells the identical idea with different syscalls, which is epoll. They are not two steps, they are one idea on two kernels, and the repo lays the mapping out side by side:

kqueue()                     ->  epoll_create1()
EV_SET(... EVFILT_READ ...)  ->  struct epoll_event { .events=EPOLLIN }
kevent(... EV_ADD ...)       ->  epoll_ctl(EPOLL_CTL_ADD)
kevent(... EV_DELETE ...)    ->  epoll_ctl(EPOLL_CTL_DEL)
kevent(... &events ...)      ->  epoll_wait()

The epoll version does not compile on macOS, there is no <sys/epoll.h>, so I built and ran it in a Linux container (gcc:14) to confirm it is the same program. It is. The loop is byte-for-byte the kqueue loop with the five calls above renamed.

Once you have written one, you have written both. You have also written the thing every async runtime is sitting on. Go's netpoller, Node's libuv, tokio through mio: a loop that asks the kernel what is ready, handles exactly that, and goes back to sleep.

Phase 07: the byte stream that isn't messages

The first six servers all echo raw bytes, which lets them dodge the problem that makes a real protocol hard: TCP is a stream, not a sequence of messages. One read can return half of what the client sent, or three messages glued together. If your server assumes one read equals one message, it works on localhost and falls apart the first time a packet boundary lands somewhere inconvenient.

Phase 07 puts a length prefix on every message and extracts whole frames out of a per-client buffer:

while (c->len >= FRAME_HDR_SIZE) {
    uint32_t net_len;
    memcpy(&net_len, c->buf, FRAME_HDR_SIZE);
    uint32_t payload_len = ntohl(net_len);

    if (payload_len > MSG_MAX) { remove_client(s, fd); return; }
    if (c->len < FRAME_HDR_SIZE + payload_len)
        return;                 /* frame not fully arrived yet */

    size_t frame_size = FRAME_HDR_SIZE + payload_len;
    if (!queue_write(s, fd, c->buf, frame_size))
        return;                 /* write buffer full, resume after drain */

    memmove(c->buf, c->buf + frame_size, c->len - frame_size);
    c->len -= frame_size;
}

Two things in there are the difference between a toy and a server.

The if (c->len < FRAME_HDR_SIZE + payload_len) return; is framing: a partial message is not an error, it is just not here yet, so you keep the bytes and wait for the rest.

And if (!queue_write(...)) return; is backpressure: when a client reads slowly and its write buffer fills, the server stops pulling new frames off the read side instead of buffering without bound. A slow client throttles itself rather than growing the server's memory until it dies.

I did not eyeball this one. The repo has a Python client that hammers the framing rules, and it passes: a single message, three pipelined into one send, a 4000-byte payload, and an oversized one that the server correctly drops the connection over. That last test is the protocol defending itself, which is the part you only get once you stop pretending the stream is already messages.

What I got wrong

I expected the headline jump to be select to epoll, the efficiency one, the thing the blog posts are about. It was not the one that reorganized my head.

The jump that did was phase 02 to phase 03, non-blocking to select. It is the move from "I ask the world, constantly, whether anything has happened" to "the kernel wakes me when something has." Those feel like the same architecture with a faster inner loop. They are not.

One pegs a core and scales by spending more CPU; the other sleeps and scales by spending less. Everything after phase 03, poll, kqueue, epoll, is the same idea getting cheaper. Phase 03 is where the idea arrives.

I had the syscalls filed under "non-blocking I/O" as one bucket for years. They are two different answers to two different questions, and I could not have told you that before I watched the CPU number drop from 99 to nothing by changing which call the loop sleeps in.

What's next

The framing version is roughly where this stops being a toy and starts being a thing with opinions: a wire format, a memory bound, a client it is willing to hang up on. It is also where C runs out of story to tell.

The next interesting question is not another syscall, it is the abstraction tower built on top of this one. I rebuilt the same ladder in Rust, raw unsafe libc calls up through mio up through tokio, specifically to see what each layer buys and what it hides. That is the part C structurally cannot show you, and it is what I am writing up next.

Adding io::ErrorKind::TooManyOpenFiles to Rust

Valentyn Kit — Sat, 27 Jun 2026 17:56:58 +0000

This is my second merged contribution to rust-lang, and like the first it lives in the low-level I/O layer.

The problem

When a process runs out of file descriptors, the OS returns EMFILE (the per-process limit) or ENFILE (the system-wide one). Until recently, Rust's standard library decoded both into io::ErrorKind::Uncategorized. If you wanted to react to "too many open files" specifically, you couldn't match on the kind. You had to drop to the raw errno:

if err.raw_os_error() == Some(libc::EMFILE) {
    // back off and retry
}

That works, but it leaks a platform detail into code that only wanted a category.

The fix

PR #158326 adds a dedicated variant to io::ErrorKind:

 pub enum ErrorKind {
     #[unstable(feature = "io_error_inprogress", issue = "130840")]
     InProgress,

+    /// The process or the whole system has reached its limit on
+    /// the number of open files or sockets.
+    #[unstable(feature = "io_error_too_many_open_files", issue = "158319")]
+    TooManyOpenFiles,
+
     // "Unusual" error kinds ...
     Uncategorized,
 }

and wires it into the per-platform decode tables, so both errnos map to it on unix and wasi:

libc::EMFILE | libc::ENFILE => TooManyOpenFiles,

with the matching ERROR_TOO_MANY_OPEN_FILES and WSAEMFILE on Windows. So now the category is first class:

match err.kind() {
    io::ErrorKind::TooManyOpenFiles => back_off_and_retry(),
    _ => return Err(err),
}

The variant is unstable for now (it implements ACP libs-team#818), behind #![feature(io_error_too_many_open_files)].

Why this corner

It is a small change, 13 lines. Reliability work usually is. This is the second PR I have landed in rust-lang's I/O code. The first documented the transient errors TcpListener::accept can hand you, the kind that are easy to treat as fatal by mistake. I am keeping at it on the low-level side, the layer the rest of the ecosystem stands on.

Framework knowledge evaporates. Low-level knowledge compounds.

Valentyn Kit — Thu, 25 Jun 2026 11:11:10 +0000

I learned async three times before it stuck. C# Tasks. Go goroutines. Rust tokio. Each time I was shipping within a week. Each time, a year later at a different job, I was asking the same questions again.

I hadn't forgotten the API. I'd learned the interface, not the mechanism.

What changed it: I built a TCP echo server in C from blocking sockets up through a kqueue/epoll event loop, one I/O model at a time, with no libraries beyond libc. Seven phases, seven runnable programs, each motivated by a concrete ceiling the previous one hit.

By the end, I understood what problem each design decision in tokio was solving. Go's netpoller and Node's libuv looked like two spellings of the same idea. Async had a shape I could hold. And I haven't needed to rebuild that mental model since.

That's the trade I kept not making: framework knowledge gets you productive fast and resets every few years; low-level knowledge costs more upfront and compounds.

How I got here

The fourth time I learned async I was three stacks in. C# Tasks, then Go goroutines, then Rust's tokio. Each time it took about a week to get up to speed, and each time I thought I understood it. Moving to the next job, the understanding had reset in ways I didn't expect: I could explain the primitives but only in the vocabulary of the most recent stack. Strip the framework's names away and I was less certain than six years should have made me.

Same thing with the other primitives I kept reaching for: idempotency, back-pressure, state machines. At each stop they showed up in different forms. I recognized them. I just didn't understand them in a way that would survive moving somewhere new.

At some point that stopped feeling like a normal learning curve.

The seven-phase TCP server

There's a document from 2003 by Dan Kegel called "The C10K Problem." The question it poses: how do you serve ten thousand clients with one server? The answer turns out to be very careful, and not what you'd first try.

I built through it phase by phase.

Phase 01 is the obvious approach: accept in a loop, read/write, one client at a time. It works, and it blocks on every syscall, so one slow client holds the whole server. That's the ceiling.

Phase 02: make the sockets non-blocking. Now read returns EAGAIN instead of sleeping, so one thread can check multiple clients. But you're busy-polling every file descriptor in a loop, burning 100% CPU even when nothing is happening. The wrong direction.

Phase 03: select. Tell the kernel "wake me when something on this list is ready," and it does. One wait, one wakeup. This is the idea: readiness notification. The problem is FD_SETSIZE, which caps you at 1024 file descriptors on most platforms, and the kernel still scans the entire bitmap O(n) on every call.

Phase 04: poll. No bitmap cap, same O(n) cost. You still rebuild the descriptor list on every call.

Phase 05 (macOS/BSD) and Phase 06 (Linux): kqueue and epoll. Register interest once; the kernel tells you only what's ready. O(ready), not O(registered).

Phase 07: framing and backpressure. Length-prefixed messages, per-client write buffers that stop reading when the output side is full. Now you have the protocol problem, not just the I/O problem.

Seven phases, each one runnable, each one the answer to the ceiling the previous one hit. No dependencies beyond libc. The whole thing is at github.com/valentynkit/building-tcp-servers-in-c.

What I was wrong about

I thought understanding non-blocking I/O was about knowing the right syscalls: select, then poll, then epoll_create1 + epoll_ctl + epoll_wait. Get the signatures right, ship the server.

What I actually needed was causation: why did each model replace the previous one, what ceiling did it hit, what does the next one do differently? That's what transfers. The syscall names are spellings; the causation is the thing.

I'd read "epoll is more efficient than select" dozens of times before building this. I understood the sentence. I didn't understand it in a way that let me derive it. After running my phase 03 server into the FD_SETSIZE limit during a test, I understood it in a way that doesn't need reading again.

What compounded

The model from those seven phases: a server that scales is a loop that asks the kernel "what's ready now," processes exactly those clients, and yields back. Everything else (Go goroutines backed by the netpoller, tokio tasks backed by mio backed by epoll, Node's event loop backed by libuv) is a scheduling layer on top of that kernel primitive.

When I read the mio source later, I recognized what it was doing. When I read about Go's netpoller, same. When I saw libuv's handle loop, same. Because I'd built the thing they're all built on, not because I studied each one until it made sense.

The same transfer shows up in production. We model every transaction as an explicit state machine: every state durable, every transition idempotent, so a crash or a chain reorg resumes from the last clean step instead of corrupting money-bearing state. I'd seen that same pattern in a message queue pipeline: the retry path where a downstream failure had to resume without reprocessing what already committed. Same primitive, different altitude. You recognize it from below, or you don't recognize it at all.

What this is

I'm rebuilding the foundations from scratch: TCP servers, allocators, storage engines, event loops. To understand what the systems I depend on are actually doing, written in public as I go.

I'm not a kernel hacker. I'm a backend engineer who got tired of relearning the same ideas at every new framework, went down to find where they live, and is climbing back up. The bugs stay in. The dead ends stay in. The parts I'm still working out stay in.

Current position: TCP servers and simple allocators done. Next I want to understand what a memory allocator is actually trading off, because I've been reaching for malloc for six years and that's started to bother me.

Code is at github.com/valentynkit. The TCP server is above; the allocator is next.