DEV Community: Chauncey Wang

In 2018 I hand-wrote a C++ deep learning framework so I'd never pad a batch. In 2023 LLM serving landed on the same structure.

Chauncey Wang — Thu, 24 Sep 2026 05:25:13 +0000

There's a sentence in the README of a library I wrote that I've been thinking about lately:

"To summarize, we believe that Padding-free Dynamic Batching is the feature that NLPers will dive into but is surprisingly not supported by today's deep learning libraries."

I wrote that around 2021, about InsNet, a C++14 deep learning library I'd been building since 2018. Then transformers ate the field, everyone padded their batches to rectangles like they always had, and I moved on.

Two years later, vLLM launched, "continuous batching" became the load-bearing idea of the entire LLM serving industry, and the input to a modern inference engine became — a flat, padding-free token stream with per-sequence offsets riding alongside as data.

I wasn't wrong. I was early, and I was aiming at the wrong layer of the stack.

This post is the story of that bet: how a C++ library made padding disappear, how the canonical prior art (DyNet) made a subtly different choice at the same fork, and how the modern serving stack (vLLM + FlashAttention) ended up rediscovering the same trick — with receipts from all three codebases, because last time I compared engines from memory people rightly asked for sources, and reading the actual code is where all the good surprises live anyway.

Why I hated padding

In 2018 I was a master's student doing NLP research — the last years before transformers swallowed the field. The workhorses were still RNNs and LSTMs, and the frontier I found interesting was the models whose computation graph changed shape with every single input: tree-LSTMs folded along a sentence's parse tree, transition-based parsers emitting a different sequence of stack operations for each sentence, hierarchical encoders running one sub-model per sentence and another over the document. Two examples in a batch almost never had the same shape — one had 7 tokens, its neighbor had 212, and their tree structures didn't line up at all.

Tree-LSTM, transition-based parser, hierarchical encoder — three inputs, three graph shapes, no shared rectangle.

The standard answer was padding: extend everything to the longest sequence in the batch, add a mask tensor, and burn FLOPs computing values you'd immediately multiply by zero. For flat same-ish-length batches this is a mild tax. For instance-dependent structures it's obscene — and worse than the wasted compute was the wasted thinking: every model became two models, the one you meant and the one that handles the mask.

I wanted to write the model for one instance and have the library figure out the batching. So I wrote a library: InsNet. It grew out of N3LDG — an earlier dynamic-computation-graph NLP library I helped build and first-authored the 2019 paper for — reworked into about 21,000 lines of first-party C++, of which 4,828 lines are one file of hand-written CUDA kernels.

Design decision #1: there is no such thing as a padded tensor

InsNet's core representation makes padding impossible rather than optional. A value's data lives in one flat buffer, and its shape is just two integers: the total element count and the width. In an NLP model every value is a 2-D matrix — d rows (the hidden size) by some number of columns, and only the column count ever changes. A transformer holds a whole sentence at once, so a value is a d×L matrix, one column per token; an RNN steps through the sentence one token at a time, so each value is a single column — a d×1 vector. Same d rows either way; the width is the only thing that moves, which is exactly why two integers pin the whole shape. There isn't even a stored length — divide the buffer's size by the hidden dimension and the token count falls out. A value is never a [batch, L_max, d] slice with a mask; it's a matrix as wide as the work in front of it. No batch dimension anywhere in the type system, no max length, no mask tensor. You cannot pad because there is nothing to pad to.

Padding stretches every sentence to the longest and burns compute on the gaps (29% here); InsNet keeps each value exactly as wide as its sentence.

Design decision #2: the batching key deliberately forgets sizes

The interesting part is how batching happens with no batch dimension. InsNet uses lazy execution: your model code builds a graph of small nodes, and nothing runs until you call forward(). At that point the executor repeatedly takes the current wave of ready nodes (Kahn's algorithm — every node whose inputs are all computed) and buckets the wave by a type signature:

// graph.h — the ready set, bucketed by signature
typedef std::unordered_map<std::string, std::vector<NodeAbs *>> NodeMap;
NodeMap free_nodes;

Each bucket becomes one batch, executed by one operator call.

The ready wave, grouped by signature: one signature per op call — the two Linear·W₁ nodes together, the two Linear·W₂ next, while the pending Adds wait.

And the whole design lives or dies on one question: what goes into the signature?

Here's InsNet's signature for a linear layer:

// operator/linear.cc
string typeSignature() const override {
    return Node::getNodeType() + "-" + addressToString(param_);
}

Operator type, plus the address of the weight matrix. That's it. Not the input width, not the number of columns. Two linear nodes batch together if they apply the same weights — even if one is transforming 3 columns and the other 300. The column count, the part that varies per instance, is deliberately left out of the key, so a single "batch" in InsNet genuinely contains matrices of different shapes.

Why width doesn't matter to a linear layer: W hits each column on its own, so a 3-column and a 300-column input concatenate into one GEMM.

This is the fork in the road, and to see why it matters you have to look at what the grown-ups did.

The road not taken: DyNet puts shapes IN the signature

Dynamic batching was not my idea. DyNet — the library from CMU that powered a lot of 2016–2018 NLP research — shipped on-the-fly autobatching in 2017 (Neubig et al., "On-the-fly Operation Batching in Dynamic Computation Graphs"), and its implementation is beautiful. Same skeleton as InsNet: build the graph lazily, compute a signature per node, batch same-signature ready nodes. But look at the signature for the workhorse op, affine transform (b + W*x):

// dynet/nodes-affinetransform.cc
Sig s(nt::affine);
s.add_node(args[0]);                     // identity of b
for(size_t i = 1; i < args.size(); i += 2) {
  s.add_node(args[i]);                   // identity of W
  s.add_dim(cg.nodes[args[i+1]]->dim);   // SHAPE of x  <-- the fork
}

The fork: DyNet's key includes shape → same-shape buckets; InsNet's key omits width → one batch of mixed shapes, and the raggedness moves into the kernel.

The shape of the data operand is part of the key. Only identically-shaped operations ever share a batch. That single decision shapes everything downstream:

DyNet's kernels stay simple. A batch is N same-shape tensors, so batched execution is just… a bigger tensor. Standard GEMM, standard everything.
But a batch must be physically assembled. A batched op needs one contiguous input, so DyNet checks at runtime whether the N operands happen to already sit contiguously in memory (in which case it aliases them, zero-copy) — and otherwise pays a gather:

// dynet/exec.cc — check contiguity, else memcpy into fresh memory
if (contig) {          // use current mem, zero copy
  my_xsi->v = min_node;
  my_batch.concat[i] = 2;
} else {               // gather into new mem
  combine_tensors(my_batch.ids, i, *my_xsi);
}

Variable-length sequences batch per time-step. Two LSTMs over a 7-token and a 212-token sentence share batches for steps 1–7; after that, the short sentence simply stops producing nodes. No padding, no masking — sequences just leave the batch when they end. (File that thought; it comes back at the end of this post.)

DyNet even ships a wonderful piece of engineering honesty: pass --dynet-autobatch 100 and it benchmarks its own three scheduling strategies on your graph and keeps the fastest — a built-in admission that the scheduler itself has a cost worth measuring.

So: same problem, same lazy-graph skeleton, and one bit of difference in the key. DyNet says a batch is same-shaped things, and keeps its kernels boring. InsNet says a batch is same-operation things, and pays for it one level down — because now the kernels have to handle a batch of differently-shaped matrices.

Why the two defaults differ comes down to where each era put the variable length. DyNet grew up in the RNN age, where a sentence's length becomes a variable number of steps — but each step's value is still a fixed d×1 vector. The length lives in the node count, not the node shape, so shape stays constant and keying on it costs nothing. Transformers move that length into the width of a value: one d×L matrix per sentence, L varying by input. Now the length lives in the shape, so keying on shape fragments the batch by length — two sentences of different length can't share a sequence-level op. InsNet drops the column count to undo exactly that: it keys each op on the fixed part — the weight matrix, or the attention head dimension — and lets the width vary, so a length-7 and a length-12 sentence batch together right through the attention matmuls, not just the linear layers. Same fork, opposite defaults — each matched to where its era hid the raggedness.

Design decision #3: kernels that take size arrays

Which brings us to the part of InsNet I'm simultaneously proudest and most embarrassed of: the CUDA. Here's the batched matmul kernel's signature and how it handles the ragged batch:

// cuda/impl.cu — one launch, a whole ragged batch
__global__ void KernelMatMul(dtype **a, bool transpose_a, dtype **b,
        bool transpose_b, int *a_rows, int *b_cols, int *ks,
        dtype **vals, bool acc, bool use_lower_triangle_mask = false) {
    int count_i = blockIdx.x;          // which instance am I?
    int b_col = b_cols[count_i];       // THIS instance's width
    int b_col_i = blockDim.x * blockIdx.z + threadIdx.x;
    if (b_col_i >= b_col) return;      // past my instance's edge: do nothing
    // ... then accumulate this cell's dot product over ks[count_i] ...
}

Look at the parameter list. Not tensors — arrays of per-instance pointers (dtype **a) and arrays of per-instance sizes (int *a_rows, int *b_cols, int *ks). The launch grid is rectangular, sized to the largest matrix in the batch; every thread first looks up which instance it belongs to and its instance's true dimensions, and threads that fall past their instance's real edge just return. One kernel launch, one batch, N different shapes — the raggedness rides in the size arrays, not in padded buffers, so the kernel masks threads instead of data. No thread ever computes a pad cell, and the real payoff isn't the memory saved so much as the compute never spent on values you'd only multiply by zero.

One rectangular launch over a ragged batch — the b_cols size array masks the threads past each instance's real width, no padded memory, only skipped threads.

(The causal attention mask is baked into the same kernel: use_lower_triangle_mask writes −1e30 — its finite stand-in for −∞ — above the diagonal. In 2018 that felt like a hack. In 2024 fusing the mask into the attention kernel is called "writing an attention kernel.")

The implementation is rough — a scalar accumulation loop, no tiling, no cuBLAS on this path (cuBLAS wants uniform shapes, and a ragged, masked batch isn't that).

One person, nights and weekends, hand-rolling ragged-batch GEMMs because the alternative was padding. You make trade-offs.

What the autograd got for free

One thing I want to defend properly: this wasn't an inference trick. InsNet trains. The executor records batches in the order it ran them, and backprop just replays the recording backwards:

// graph.cc — backward = the forward tape, reversed
for (int idx = count - 1; idx >= 0; --idx)
    execs.at(idx)->backwardFully();

Because forward executed in topological waves, the reversed tape is a valid reverse-topological schedule, and each backward step is batched exactly like its forward step was — the linear layer's backward is two batched GEMMs (weight grad and input grad) over the same concatenated columns. The padding-free property propagates through training for free: no masked loss terms, no gradient contributions from pad tokens, because pad tokens don't exist.

2023: the same bet, different battlefield

Now put InsNet down and look at what LLM serving converged on.

vLLM's scheduler — the "continuous batching" everyone talks about — recomposes its batch on every single step. The v1 scheduler's own docstring is admirably blunt:

# vllm/v1/core/sched/scheduler.py
# NOTE(woosuk) on the scheduling algorithm:
# There's no "decoding phase" nor "prefill phase" in the scheduler.
# Each request just has the num_computed_tokens and num_tokens_with_spec.
# At each step, the scheduler tries to assign tokens to the requests
# so that each request's num_computed_tokens can catch up ...

Requests join the running batch mid-stream, advance by whatever token budget allows, and leave the moment they finish. And the tensor that actually enters the model? A comment in the input-preparation code draws you the picture:

# vllm/v1/worker/gpu_model_runner.py
# E.g., [2, 5, 3] -> [0, 0, 1, 1, 1, 1, 1, 2, 2, 2]
req_indices = np.repeat(self.arange_np[:num_reqs], num_scheduled_tokens)
# cu_num_tokens: [2, 5, 3] -> [2, 7, 10]

Three requests with 2, 5, and 3 tokens this step become one flat array of 10 tokens. Not a [3, 5] matrix with padding — a length-10 stream, with the boundaries [2, 7, 10] carried alongside as data. Those cumulative offsets are handed to FlashAttention's variable-length entry point as cu_seqlens_q, and the attention kernel does per-sequence masking internally — one kernel launch spanning differently-sized sequences, boundaries as arrays, not as tensor shape.

Which is to say: the modern stack's answer to raggedness is put the sizes in a side-array and make the kernel consult it — the same structural move as int *b_cols in a 2018 impl.cu, executed by professionals with tiling, TMA, and a few billion dollars of demand behind them.

Flat memory, sizes carried as a side-array, a kernel that reads them — int *b_cols in 2018, cu_seqlens_q in 2023. An old friend in much better clothes.

Even the "sequences just leave the batch when they end" idea — DyNet's per-time-step LSTM batching from 2017 — is recognizable as the ancestor of continuous batching: batch membership defined per step by who still has work, rather than per batch by who arrived together.

The honest differences

I want to be precise here, because "I invented vLLM in 2018" is not the claim and would be false in at least three ways.

I didn't invent dynamic batching. DyNet's autobatching (2017) and TensorFlow Fold (2017) are prior art, published and cited; InsNet was a contemporary of that line making a different trade at the signature fork — raggedness in the kernels rather than homogeneity in the batches. I'd been working in that line since N3LDG (2019), which auto-batched dynamic graphs — tree-LSTMs included — faster than PyTorch; the padding-free reframing came later, with InsNet. The idea was in the air; my bet was on which layer should absorb the raggedness.

vLLM batches requests, not graphs. Its scheduler moves token-chunks of many requests through one static model, under @torch.inference_mode() — no autograd, no operator-level graph surgery. InsNet batched arbitrary training graphs, operators-first: it would happily batch two transformers from the same instance, something entirely outside vLLM's problem statement. Same enemy (padding), same weapon (flat memory + offset bookkeeping), different battlefield (serving scheduler vs. training executor).

And they solved the problem I never had. Serving needs the KV cache of every in-flight request to survive while the batch churns around it — that's PagedAttention, block tables decoupling logical sequence length from physical memory, and it has no analogue in InsNet because training graphs don't have persistent per-request state. The part with no analogue in InsNet is on the memory side, not the batching side.

What I got right, what I got wrong

Right: padding is not a law of nature. It's an artifact of insisting that a batch be a rectangle, and if you're willing to carry sizes as data and write kernels that read them, the rectangle dissolves — along with the mask logic that infects every model built on top of it. The 2021 README sentence holds up; "NLPers will dive into it" just turned out to mean the inference-serving industry rather than the tree-LSTM researchers I was writing for.

Wrong: the layer. I bet that padding-free batching mattered for training arbitrary structures, and built a general graph executor with autograd. The world's raggedness problem turned out to be concentrated in one place — transformer decoding, where sequences in a serving batch naturally diverge in length — and the winning implementations attached themselves to that single, ferociously-optimized case. Generality was the wrong axis. Specificity, plus paged memory, plus a scheduler that treats the batch as a per-step decision, is what shipped.

Also wrong, in a smaller way: nearly everything about how I wrote it. The erased early history. The scalar GEMM loop. A hyperparameter I'd tune differently today on basically every line. But the shape of the thing — flat buffers, size arrays, kernels that mask threads instead of data — is the shape the field landed on, and there's something quietly vindicating about opening flash_attn.py, seeing cu_seqlens_q, and recognizing an old friend wearing much better clothes.

The library is at github.com/chncwang/InsNet, erased early history and all.

Code excerpts are from the actual sources of InsNet, DyNet, and vLLM (v1 engine), read for this post. If you find a misreading, tell me — my previous source-diving post survived three rounds of adversarial review and I'd like this one to earn the same.

Nobody inside a Go game can say "game over" — so seven engines invented five referees

Chauncey Wang — Mon, 14 Sep 2026 03:19:39 +0000

Chess ends by rule: checkmate is checkmate, the referee is built into the move generator. Go doesn't work like that. A game of Go ends by agreement — both players pass, then they discuss which stones are dead, take them off the board, and count. The rules don't decide when the game is over; the players do.

Look at the two circled stones on the left board. They still have a liberty. No rule has captured them. They are also, to any human player, obviously dead — they can never make two eyes — which is why both players pass, lift them off, and count the points under them for Black. Their death is a fact about futures that were never played.

Which is a problem if your players are programs, because agreement is the one thing programs can't do — and that fact about unplayed futures is the fact every engine in this post has to manufacture for itself.

In 2012 I wrote FoolGo, a Monte-Carlo Go engine that plays about 40,000 random games per second on a 9×9 board. In my last post I compared its data structures with six other engines and found they had all converged on the same answer. This time I went looking at a question I never consciously answered when I wrote the thing: how does each engine know a game is over? A Monte-Carlo engine finishes tens of thousands of games per second, so whatever your answer is, it runs constantly, and it had better be cheap.

I had Claude Code read the sources of the same seven engines — FoolGo, GNU Go 3.8, Pachi, libego, Fuego 1.1, Leela Zero, and KataGo — and verified the load-bearing claims against the code myself. Last time everyone had converged. This time they didn't. Five different referees came back:

by exhaustion — play until the board runs out of answers (FoolGo, libego, Fuego)
by statistics — a stone is dead if it usually ends up dead (Pachi)
by reasoning — prove each group alive or dead with a specialized search (GNU Go)
by fiat — declare everything on the board alive and make it the players' problem (Leela Zero)
by mathematics — end the game only when life is provable (KataGo)

My engine cannot pass

Start with the confession. FoolGo cannot choose to pass — a pass happens only when a player has no legal move at all, a mechanical necessity so that one side running out of moves first doesn't deadlock the loop. It also has no move limit. Strictly speaking, it doesn't have a concept of the game ending at all — only a board with no moves left on it:

// src/board/full_board.h
bool FullBoard<BOARD_LEN>::IsEnd() const {
  return is_end_ || (PlayableIndexBitSet(Force::BLACK_FORCE).none()
      && PlayableIndexBitSet(Force::WHITE_FORCE).none());
}

A playout ends when neither color has a single playable point left. Each side keeps a bitset of playable points; stones fill the board until both bitsets are empty. That's the whole referee.

It's the same game as the opening figure — but where the humans stopped and agreed, FoolGo kept going. The invasion they lifted off the board by agreement, it captured and refilled en route. Same verdict on the dead stones, computed the slow way.

Why does that terminate? Because of the one thing a random player refuses to do: fill its own eyes. FoolGo marks a point as a real eye when all four orthogonal neighbors are friendly and enough diagonals are controlled — the threshold is a three-entry lookup table:

// src/board/full_board.h — diagonals required: center 3, edge 2, corner 1
static const PositionIndex TABLE[3] = { 3, 2, 1 };
auto state = calculator.CentralOrEdgeOrCorner(position);
if (TABLE[state] <= piece_or_eye_count) {
  SetRealEyeAsTrue(ForceAndPositionIndex(force, indx));
}

A real eye is removed from both colors' playable bitsets: the owner won't fill it, the opponent can't legally sit in it. Every other point eventually gets a stone. Groups with two real eyes survive; everything else gets captured and refilled; the board monotonically runs out of legal moves, and IsEnd() fires. Termination isn't a rule in FoolGo — it's a theorem, and the eye test is the proof.

The theorem has a crack in it, though, and the crack has a name: seki (双活, "mutual life"). Two groups can stand in a truce where neither has two eyes, but whoever fills a shared liberty first puts their own group in atari and dies. Correct play is to leave those points alone forever — which is to say, correct play is to pass.

FoolGo can't. Its bitsets exclude only real eyes, and a seki's shared liberties look like ordinary playable points. So once the rest of the board fills up, the engine — forbidden from passing while any playable point remains — is eventually forced to fill one, put its own group in atari, and hand the opponent a capture that correct play would never allow. The exhaustion referee doesn't merely fail to recognize mutual life; it is structurally incapable of letting a truce stand. Every seki in a FoolGo playout collapses, and which side it collapses against is decided by who runs out of safe moves first — so a position whose true value is "standoff" gets evaluated, over thousands of playouts, as a weighted coin flip.

To be honest about it: this was a choice, not an oversight. Seki is rare — the large majority of games never produce one — and ignoring it bought me a referee with no pass logic, no self-atari exceptions, no special cases at all. Accept a coin flip on a rare position, and the whole design stays one IsEnd() check. 2012 me judged that a good trade, and for a 9×9 hobby engine it probably was.

The grown-up engines all knew about this trap — and, tellingly, not one of them recognizes seki by looking for it. Each solves it in the style of its own referee, a preview of everything below.

Fuego solves it with an escape hatch. Pass is a legal move at every node of its search tree, and the values do the rest: filling a seki liberty leads to positions where your group is gone, passing leads to positions where the truce holds, and the search notices which door not to open. No detector — just an exit, and a search smart enough to take it. The documentation names the trap in so many words: "in the in-tree-phase of the game, pass moves are always allowed to avoid zugzwang situations, if there is a seki on the board." And note the "if": it's motivation, not a condition. Nothing in Fuego checks whether a seki exists — the pass move is appended to every node's move list unconditionally, one line, no questions asked. The tree never knows there's a truce; it discovers, a few thousand simulations later, that every move except pass loses a group. In other words: there is no seki algorithm anywhere in Fuego — MCTS itself discovers that pass is the only move that doesn't lose.

Pachi solves it with statistics. Its playout policy refuses bad self-atari (is_bad_selfatari) — putting your own group at one liberty for no gain; the check is careful enough to still permit sacrificial self-ataris like throw-ins and nakade (deliberate sacrifices that ruin the opponent's shape or eyespace) — so the truce survives most of its random futures — and then its ownership map notices what survived: a point that stays empty in 80% of playouts earns a named verdict, PJ_SEKI. That's how a truce looks when you count futures.

And GNU Go solves it by elimination, the prover's way. Its aftermath module tries to make every friendly group invincible and remove every enemy stone; whatever resists both — stones that "cannot be removed, nor turned invincible" — is declared alive in seki. Run that on our corner: every attempt to capture the white group means filling d1 and dying, so removal fails; every attempt to give it a second eye fails too, because there's no room for one. The stones survive both procedures while belonging to neither side's territory — and that leftover gets the label GNU Go reserved for it: ALIVE_IN_SEKI. Seki isn't detected; it's what's left when killing and securing have both been tried and both failed.

Three referees, three ways of letting a truce be a truce. Mine made everyone fight to the death.

The scoring, in turn, is exactly as simple as the no-pass design allows it to be. FoolGo counts black — and only black:

// src/board/full_board.h
PositionIndex BlackRegion() const {
  return black_pieces_count_ + eye_states_array_[BLACK_FORCE].RealCount();
}

Black's score is its stones plus its real eyes, as a fraction of the board; white's is defined as the complement, 1 − black_ratio. That sounds reckless until you remember what exhaustion guarantees: when IsEnd() fires, every point on the board is a stone or a real eye — nothing neutral, nothing contested. On such a board the complement of black is white, so one counter suffices. The scheme is internally consistent with the referee that feeds it — assuming, as ever, no seki.

The lemma everyone needs

Here's the thing though: strip away the naivety, and FoolGo's termination argument is everyone's termination argument. Every playout engine rests on the same lemma — a random player that never fills its own true eyes runs out of moves — and every one of them needs the same sub-routine: a cheap test for "is this point a real eye, or a false one?"

A false eye looks like an eye (four friendly orthogonal neighbors) but the diagonals betray it: with enough enemy stones on the diagonals, the connecting stones can be captured and the "eye" collapses. The classic heuristic is: in the center, an eye survives at most one bad diagonal; on the edge or corner, none. I found three independent implementations of that sentence — and a fourth engine that has one, but deliberately keeps it out of its playouts.

None of the four spends more than a few lines on it — and they split into exactly two schools:

Pachi, libego, and FoolGo count enemy diagonals — one rule, three spellings: Pachi's virtual enemy stone for the board edge, libego's single boolean, FoolGo's {3, 2, 1} table.
Fuego's playout never looks at the diagonals — the classic test exists in its codebase, but it's reserved for the search tree. The playout filter, inherited from Rémi Coulom's Crazy Stone paper, treats any point fully walled by friendly stones as an eye, false or not, with one release valve: the point becomes fillable the moment an adjacent wall block drops to its last liberty. Stricter before the fight — it refuses to fill even a blatant false eye, enemy diagonals and all, while no wall block is in danger — and exact during it.

Three codebases converge on the same diagonal count — the same forced-answer convergence I found in the chain data structures last time. Fuego's playout is the exception that sharpens the rule: the diagonal test was never the point. The point is a cheap answer to one question — will this eye still exist after the next move? — and you can buy it statically, by counting diagonals, or dynamically, by watching for atari. Either way, the answer is the termination proof of the whole Monte-Carlo method, implemented as a neighbor count.

Pachi's version carries my favorite comment in this entire investigation, admitting the heuristic's known defeat:

/* XXX: We attempt false eye detection but we will yield false
 * positives in case of http://senseis.xmp.net/?TwoHeadedDragon :-( */

Nobody trusts the natural ending

FoolGo lets every playout run to structural exhaustion, uncapped — the only engine of the seven that does. (It once had a cap; a rewrite dropped it, nobody ever noticed, and the orphaned constant still sits in a header.) Everyone else wears a belt with their suspenders:

libego ends a playout on two consecutive passes or a hard cap of 3 × board_area moves — and here's the detail I love: a playout that hits the cap is thrown away, not scored. DoOnePlayout just returns without updating the tree. An over-long game is treated as a measurement failure, not a data point.
Pachi caps at MAX_GAMELEN — 600 moves.
Fuego caps at 3 * size * size with a comment explaining exactly why the cap must exist: for speed, playouts only check simple ko. A superko cycle — a position repeating after four or six moves — would loop forever, and nobody's going to spend hash lookups on full cycle detection at playout speed. The cap is the cheap insurance against the rules corner they deliberately chose not to implement.

Here's what that fear looks like on a board — the classic triple ko: two eyeless groups locked together, three kos between them, both lives hanging on the fight:

How scared should anyone actually be? Not very. Of the 168,813 games played by Nihon Kiin professionals between 1924 and 2007, exactly 19 ended "no result" — the scoreboard's name for an unbreakable cycle. About one game in nine thousand. (The most famous one is also why triple ko is considered bad luck: in 1582 it suspended a game played in Oda Nobunaga's presence in Kyoto, and the next day Nobunaga was betrayed and killed.) Which explains FoolGo one more time: no cap, no superko check, nothing that could ever break this cycle — the same wager as the seki one. Accept a catastrophic failure mode on a one-in-nine-thousand position, and the referee stays one line long.

Fuego adds one more mechanism, my favorite name in the whole codebase — the mercy rule:

// gouct/GoUctGlobalSearch.h
m_mercyRuleThreshold = static_cast<int>(0.3 * size * size);

It keeps a running stone-difference counter, updated by captures after every playout move. The moment one side is ahead by 30% of the board — 102 stones on 19×19 — the playout stops and is scored as a certain win. No need to play the last 200 moves of a massacre. The game is over long before it's over, and Fuego is the only engine honest enough to write that down as a rule.

Scoring the corpse

Once a playout ends, someone has to count. The engines' scorers are a study in how much correctness you can trade for speed when you're only feeding a win-rate estimator.

libego's playout scorer doesn't flood-fill territory at all. It counts stones, then makes one assumption: any empty point left on a finished board must be an eye of whoever surrounds it — because playouts fill everything else:

// board.cpp — PlayoutScore(): stones + one pass over the empties
int RawBoard::EyeScore (Vertex v) const {
  return
    nbr_cnt[v].player_cnt_is_max (Player::Black ()) -
    nbr_cnt[v].player_cnt_is_max (Player::White ());
}

No dame arbitration, no seki handling, O(area), branch-light. The exact scorer exists in the codebase too: Tromp-Taylor, the computer-friendly statement of Go's rules — every stone counts as it stands, nothing is ever judged dead, and each empty region is flood-filled to see whose stones it touches: one color's only, it's their territory; both colors', it's nobody's. That scorer is reserved for the search tree, where positions end in passes rather than exhaustion and empty regions can still be large.

Here's the whole rule on one board:

Fuego makes that same split explicit with two scorers and a selector: if the two passes happened inside the playout (board full, position clean), use the fast ScoreSimpleEndPosition, whose per-point helper literally asserts there are no empty neighbors left to worry about. If the passes happened in the search tree (real-game-shaped position, open regions), use the flood-filling TrompTaylorScore. Where the passes happened tells you which scorer you're allowed to afford.

And FoolGo? Black's stones plus black's eyes, divided by 81, white gets the complement — which, on an exhaustion-finished board, is exact area counting.

Referee by statistics

Pachi's answer to the dead-stone problem is the most Monte-Carlo idea in this whole story: don't judge — count.

Every finished playout deposits its per-point winner into an ownermap — a per-intersection tally of who owned that point when the game ended. Play a few thousand random futures, and life and death becomes a frequency:

/* ownermap.c — a point's status, judged by its futures */
if      (n     >= total * thres)  return PJ_SEKI;   /* stays empty 80% of the time */
else if (n + b >= total * thres)  return PJ_BLACK;
else if (n + w >= total * thres)  return PJ_WHITE;
else                              return PJ_UNKNOWN;

A group is declared dead when the enemy ends up owning its points in at least 67% of playouts. Seki — the mutual-life stalemate that breaks naive scorers — falls out for free: it's a point that stays empty in 80% of futures, because neither side can afford to fill it — exactly what the frequency bars showed at d1, back in the seki section. Nobody wrote a seki detector. The statistics are the seki detector.

And here is the same tally meeting a harder version of the question this post opened with — the corner invasion unanswered, reinforced by a fresh white stone at h7, with black pressing from below at h5:

One measured footnote on that 73%: remove the black stone at h5 from that board and the tally slides to 65% — under the bar — and the verdict flips from dead to withheld. Pachi has furniture for exactly that case: ownermap_dead_groups files every group into one of two queues — dead or unclear — and board_position_final refuses to call the position finished while anything sits in the unclear queue: the game simply continues until the futures agree. The verdict isn't a boolean; it's a threshold crossing, and the referee knows which side it's standing on. (Both numbers come from my own uniform-random referee; Pachi's production playouts add patterns and capture heuristics — sharper futures, sharper verdicts.)

What sold me is how far Pachi trusts this. The GTP command final_status_list dead — the protocol question "which stones do we remove before counting?" — is answered by seeding the ownermap with at least 500 fresh playouts and reading death off the frequencies. Even Pachi's decision to pass in a real game is gated on the ownermap: it passes only when the opponent has passed, the ownership-based score estimate says it's winning, and the map says the position is final. The referee, the scorer, and the "is it safe to stop?" instinct are all the same object: a histogram.

Referee by reasoning — with a confession

GNU Go is the classical engine in the group — pre-Monte-Carlo, pure knowledge — and its referee is philosophically opposite. It doesn't even have a pass rule. It has an accountant:

/* engine/genmove.c */
move = PASS_MOVE;
*value = 0.0;

Every candidate move gets a value in estimated points. If the best value on the whole board is not strictly positive — after escalating through endgame patterns, re-examined capturing races, and finally dame worth a single point — the move stays PASS_MOVE. GNU Go passes when the ledger is empty. Passing isn't a decision; it's what's left when arithmetic finds nothing worth one point.

I watched it happen. GNU Go 3.8 still compiles, and fed the opening position over GTP, it prices the board out loud:

Deeper judgments get the full symbolic treatment — a dedicated life-and-death search (the owl code) stamps each group DEAD, CRITICAL, or ALIVE — but here is the confession: ask GNU Go to actually score a finished game, and it sets every symbolic verdict aside, plays the position out with one careful, deterministic playout (threats priced at zero), and reads life and death off the finished board. The reasoning engine's most trusted referee is a Monte-Carlo sample of size one — and the module that does it is named aftermath, which exists, per its own header comment, "to robustly determine life and death status": the proofs weren't robust enough to bet the score on. Run live on the opening position, the whole apparatus names exactly two dead stones — h9 and i9 — and returns B+12.0: the humans' verdict, reached by one careful playout instead of a handshake. (One point shy of the area count from the fiat figure only because GNU Go defaults to territory scoring, Japanese-style, while Tromp-Taylor counts area, Chinese-style — a parity quirk between the two systems, not a disagreement about life.)

Referee by fiat

Leela Zero — the AlphaZero-style engine — solved the dead-stone problem by deleting it:

// FastBoard.cpp — Needed for scoring passed out games not in MC playouts
float FastBoard::area_score(const float komi) const {
    auto white = calc_reach_color(WHITE);
    auto black = calc_reach_color(BLACK);
    return black - white - komi;
}

That's Tromp-Taylor area scoring where the flood-fill is seeded from every stone on the board, unconditionally. There is no dead-stone adjudication anywhere in the engine. If a stone is on the board when the game ends, it is alive, and it counts — even if it's a hopeless prisoner sitting in your territory.

Here's what that fiat does to the position this post opened with — the one where the humans agreed the invasion was dead:

The two "dead" stones aren't just alive — they poison everything that can reach them. Twenty-nine points of Black's territory become no-man's-land, and a game Black won by 13 becomes a game White wins by 20. Mercy, under this referee, costs more than the mercy was worth.

This isn't an oversight; it's the training contract. The scorer's rule is pushed onto the player: if you want your opponent's dead stones to not count, capture them before you pass. The network learns to physically clean the board because the referee will not do it any favors. The judgment that GNU Go performs with a specialized search module, and Pachi with a histogram, Leela Zero absorbs into the policy network's weights.

Referee by mathematics

KataGo looked at all of the above and did the most engineer thing possible: it treated "when is the game over, and what does it score" as a first-class rules-engineering problem. Its Rules object has knobs the other six engines don't even have vocabulary for: area vs territory scoring, a group tax (every living group forfeits its two eye points — the ancient Chinese 还棋头, surviving as an enum value), a button (half-point to whoever passes first, buying area rules the fine endgame incentives of territory rules), and spight-style single-pass endings under certain ko rules. And one more, wonderfully: friendlyPassOk. Under area scoring an uncaptured dead stone counts against you — the fiat figure showed the bill — so a strict engine must capture everything before daring to pass. This flag tells KataGo whether the opponent can be trusted to agree about dead stones after the passes; that is, whether the human ending is available. The problem this post opened with, shipped as a boolean.

But the crown jewel is the referee that can't be wrong. KataGo implements Benson's algorithm — the 1976 result that identifies groups that are unconditionally alive: alive even if their owner passes forever. (Stronger than what club players usually mean by the phrase — alive even if the opponent moves first — Benson's sense allows the owner no answering moves at all.) Pass-alive is not a heuristic, a frequency, or a search verdict; it's a fixpoint computation with a proof:

// game/board.cpp — https://senseis.xmp.net/?BensonsAlgorithm
// Walk all player heads and kill them if they haven't
// accumulated at least 2 vital liberties
while(true) {
  ...
  if(vitalCountByPlaHead[plaHead] < 2) {
    plaHasBeenKilled[i] = true;
  ...
}

Here's the distinction the theorem draws, on a board — two black groups any club player would call alive:

The bottom-left group's two eyes are each entirely walled by the chain — both vital, in Benson's vocabulary — and the fixpoint keeps it: no sequence of White moves, of any length, can ever capture it. The top-right group has more eyespace and less life: the corner point of its big eye touches only the eyespace, never the chain, and that single non-liberty is both why the region isn't vital and how the kill works if Black never responds: White fills the three real liberties while breathing on exactly that point, then takes the last eye as a capture. In a real game Black defends trivially, which is the whole distinction: one group's life is a theorem, the other's is a promise to keep answering. Benson's algorithm certifies only theorems.

And it's used exactly the way a theorem should be. During self-play, after every single move, KataGo checks: does every point on the board now resolve to a pass-alive owner? If yes, the game ends immediately — no passes needed, the outcome is provably settled. If even one point is unproven, the check bails and play continues:

// game/boardhistory.cpp — endGameIfAllPassAlive
if(area[loc] == C_WHITE)      boardScore += 1;
else if(area[loc] == C_BLACK) boardScore -= 1;
else return;   // one unproven point → keep playing

For everything Benson can't prove — the ordinary dead stones of an ordinary human game — KataGo falls back to a learned judge: the search's visit-averaged ownership map, thresholded (a chain is alive only if its average ownership is safely its own color, and no point of it is badly contested). Which is to say: KataGo's referee is a theorem where a theorem is possible, and Pachi's histogram — upgraded from playout frequencies to neural predictions — everywhere else.

Five referees

So: the same question, seven codebases, five answers. (Five philosophies, not five disjoint mechanisms — KataGo layers three of them, and GNU Go's confession shows reasoning leaning on exhaustion when the score is on the line.)

Referee	Engine(s)	"The game is over when..."	Dead stones are...
Exhaustion	FoolGo, libego, Fuego	the board runs out of legal non-eye moves	whatever got captured along the way
Statistics	Pachi	the ownership histogram says the position is final	stones the enemy owns in ≥67% of futures
Reasoning	GNU Go	no move on the board is worth a single point	what the owl search proves is dead
Fiat	Leela Zero	two passes — and the scorer believes the board as-is	nothing; kill them yourself before passing
Mathematics	KataGo	every point is provably pass-alive (or the rules say so)	Benson-unprovable chains the ownership net condemns

Last time, seven engines converged on one data structure, and the lesson was that the problem forces the answer. This time the same seven engines diverged into five referees, and I think the lesson is the mirror image: the board doesn't force an answer, because the board genuinely doesn't contain one. "Is this group dead?" is a question about futures that haven't been played, and each era answered it with whatever epistemology it had — exhaustive filling in 2012, statistics in the playout era, symbolic proof in the classical era, learned judgment after AlphaZero, and, in KataGo, an honest layering of all of them with a theorem at the bottom.

Grouped one level up, though, the five referees are really two. Pachi, GNU Go, and KataGo adjudicate death — by histogram, by owl proof, by theorem plus a trained judge. The exhaustion engines and Leela Zero refuse to adjudicate at all: they insist death be made physical — the corpse gets captured, by the playout or by you, or it isn't a corpse. Every referee makes death factual before it counts. The one thing no engine does is the human thing from the first figure: leave the dead stones on the board and agree. Agreement was never on the menu — so half of them replaced it with violence, and the other half with a courtroom.

FoolGo's referee asked no questions at all. It just waited for the board to run out of answers — no voluntary pass, no move cap, and a scoring function that counted one side and inferred the other. I built the degenerate case of the idea everyone else spent a decade refining. But it terminated, forty thousand times a second, for the same underlying reason every playout in this story terminates: a random player who refuses to fill his own eyes eventually has nothing left to say. Even the fool stops playing when the board is full.

Every code excerpt above is from the engines' actual sources: FoolGo (github), GNU Go 3.8, Pachi, libego, Fuego 1.1, Leela Zero, KataGo — read with Claude Code, with the quotable claims re-checked by hand. If you find a misreading, tell me — the previous post survived three rounds of adversarial review and I'd like this one to earn the same.

No neural nets, no tree: inside a Go engine that plays 40,000 games per second

Chauncey Wang — Fri, 21 Aug 2026 02:54:37 +0000

In December 2012 — three years before AlphaGo — I wrote a Go AI in C++ called FoolGo. It has no neural networks, no opening book, no hand-crafted evaluation function. Just Monte Carlo tree search with UCB1, and enough systems engineering that it simulates about 40,000 complete games of Go per second — benchmarked on my 2014 MacBook Air.

It never got past beginner strength on 9×9, and I've always been upfront about that. Most of the repo's stars actually arrived after AlphaGo beat Lee Sedol in 2016: once the whole world wanted to know how Go AI worked, people came looking for the pre-intuition machinery in readable form — what the field ran on before neural networks learned to feel a board. This post is a tour of the engineering that makes vanilla MCTS fast — because for vanilla MCTS, fast is the only thing there is.

Why throughput is the whole game

Pure MCTS has no way to evaluate a Go position directly. To judge a candidate move, it plays random games from that position all the way to the end, and counts wins. Strength is roughly a function of how many of these playouts you can afford per move. There's no clever prior to save you — the engine is its simulation throughput. So every design decision below is about the same thing: making "play a full random game of Go" as close to free as possible.

Board size fixed at compile time

Everything in FoolGo is templated on the board length:

template<BoardLen BOARD_LEN>
class FullBoard { /* ... */ };

A 9×9 engine and a 19×19 engine are different types, compiled separately. That sounds like C++ showing off, but the payoff is concrete: every array in the hot path — the board, the chain structures, the hash tables of the hasher — has a size known at compile time. No std::vector growth, no heap allocation during search, no pointer chasing where an array index will do.

Stones as disjoint sets, in flat arrays

In Go, stones of the same color that touch form a string, and the string — not the stone — is the unit of life and death: the moment a string's last liberty (adjacent empty point) is filled, the whole string is captured and removed from the board at once.

Watch what a single move can do:

Before the move, black has two strings — and they are not equal. The pair at c4–c5 shares five liberties as one unit; the lone stone at e5 has been squeezed down to exactly one. In Go terms it is in atari: one White move from being eaten. Connecting at d5 is the rescue — and notice that this single placement fuses two strings and the new stone into one four-stone string with five liberties. Count the rings on the right: the arithmetic isn't additive. Merging liberty sets means deduplicating them — a detail that will matter shortly — and enemy stones don't count: White's four stones block four of what would otherwise be nine. The rescue cost White too: its d4–e4 string just lost one of its own liberties. Push any string's count to zero and it gets eaten — captured and removed from the board at once. A stone has four neighbors, so a single placement can fuse up to four strings (plus the new stone) into one.

And that's the disjoint-set view of the same event — before, two sets; after, one:

before:   set A: head = c5,  stones: c4 → c5,  c5 → c5
                 liberties {b4, b5, c3, c6, d5}
          set B: head = e5,  stones: e5 → e5
                 liberties {d5}

play d5:  union(A, B, d5)

after:    set A: head = c5,  stones: c4 → c5,  c5 → c5,  d5 → c5,  e5 → c5
                 liberties {b4, b5, c3, c6, d6}

Each stone points straight at its set's head, so find is a single array read. The price shows up at merge time: the absorbed side's stones get repointed to the surviving head — and MergeLists chooses sides by size, relabeling the smaller list into the larger (the classic union-by-size trick that keeps total relabeling cheap). So the freshly played stone, a list of one, never wins the election: here d5 and then e5 are repointed into the c4–c5 pair's head. And the liberty sets OR together, minus the point just filled, plus the new stone's own empty neighbors. Hold that picture; the code below is nothing more than this, made fast.

Now count what the engine must answer on every move of every simulated game:

Which string does this neighbor belong to? — find
Did my move drop that enemy string's liberties to zero? If so, remove all of its stones, immediately — an aggregate query plus member enumeration
Is my move suicide? — the same query, pointed at my own string
Did my move connect friendly strings? Merge them — union

At 40,000 games per second, a couple hundred moves per game, that's millions of move-executions per second, each running several of these queries. They all have to be near-O(1).

This is the union-find problem wearing a Go costume — with two extra requirements the textbook version doesn't have: each set needs cheap member enumeration (to delete a captured string from the board) and a cheap aggregate statistic (the liberty count that decides capture and suicide). FoolGo's ChainSet answers all of it with two flat arrays — one node per board point, one list record per potential chain:

struct Node {
  PositionIndex next_, list_head_;
} nodes_[BoardLenSquare<BOARD_LEN>()];

struct List {
  PositionIndex tail_, len_;
  BitSet<BOARD_LEN> air_set_;
  AirCount air_count_;
} lists_[BoardLenSquare<BOARD_LEN>()];

list_head_ is the find pointer — every stone knows its string's representative. The next_ chain makes members enumerable without searching the board — that's the capture-removal path. And the per-list air_set_/air_count_ are the aggregates that answer capture and suicide checks in constant time. Merging two strings is a list splice plus head-pointer updates — index arithmetic on preallocated arrays, no allocation, no tree balancing.

Liberties as bitsets

The expensive bookkeeping is the liberties themselves ("airs" in FoolGo's vocabulary): every placed stone changes the liberties of all its neighbors. FoolGo stores each string's liberties as a bitset over board points. When strings merge, their liberty sets merge with a bitwise OR; counting is a popcount. Some of the hottest bookkeeping in the engine compiles down to word-sized bit operations.

Here is the connection at d5 again, seen through the bitsets:

                    b4 b5 c3 c6 d5 d6      (75 more bits, all 0)
pair {c4,c5}         1  1  1  1  1  0
lone {e5}            0  0  0  0  1  0
new  {d5}            0  0  0  0  0  1

OR                   1  1  1  1  1  1
clear d5 (filled)    1  1  1  1  0  1     popcount → 5

Three ORs, one bit-clear, one popcount. There is no deduplication logic anywhere: d5, a liberty of both black strings, is simply the same bit twice, and OR-ing makes the duplicate vanish by construction. On a 9×9 board the whole set fits in two machine words, so "merge three strings and recount the result's liberties" costs a handful of CPU instructions.

Anatomy of a move

All of that bookkeeping exists to make one operation cheap: actually playing a move. Here is FoolGo's real pipeline, from full_board.h.

First, is the move even legal? IsSuicide looks at the four neighbors of the empty point — and at nothing else:

for each of the 4 neighbors:
    empty?                                 → not suicide (instant liberty)
    friendly string with ≥ 2 liberties?    → not suicide (joins a string that still breathes)
    enemy string with exactly 1 liberty?   → not suicide (the move captures it, freeing space)
none of the above                          → suicide

Four array reads. No board scan, no trial placement — every clause is answered by a per-string air_count_ that the merge machinery has been keeping current all along. The friendly clause needs ≥ 2 because the new stone fills one of that string's liberties on arrival; the enemy clause is the elegant one — a move onto your last-looking point is perfectly legal if it kills the surrounder first.

Here are the clauses on the board:

In the left position, e5 fails every clause: no empty neighbor, no friendly string to join, no enemy string at one liberty — suicide, illegal. Add a single black stone at d3 (middle) and the three-stone white string d4–d5–e4 falls to its last liberty: the same point now passes the kill clause. Play it (right) and the pipeline below runs — the string is eaten first, RemoveChain walking its cyclic list, so by the time the black stone lands it has two liberties: points it just vacated.

Then, play it. PlayBasicMove runs the same four-neighbor scan once more, now with consequences, in a very deliberate order:

Captures first. Any enemy neighbor string down to its last liberty is eaten on the spot: RemoveChain walks its cyclic list and clears the stones — and the newly emptied point next door is immediately recorded as a liberty of the stone about to be placed. The new stone breathes into the space it just vacated.
Place the stone, and clear its point's bit from every adjacent string's liberty bitset — friend and enemy alike.
Merge. AddPiece creates the one-stone set and union-by-size folds it together with its friendly neighbors — the disjoint-set dance from earlier.
Suicide cleanup. If, after all that, the just-built string has zero liberties, it is removed by the very same RemoveChain — suicide isn't a special case, it's a capture whose victim is yourself.

Every step is either a four-neighbor loop or an operation on the structures above — nothing touches the other 76 points of the board. That locality, times forty thousand games a second, is the entire performance story.

There is no tree

The textbook picture of MCTS is a tree of nodes with parent/child pointers. FoolGo doesn't build one. Instead, every game state maps to a 64-bit Zobrist hash, and node statistics live in a flat hash table:

struct StaySelfHasher {
  std::size_t operator()(HashKey hash_key) const {
    return hash_key;   // the key is already a hash — don't hash a hash
  }
};

std::unordered_map<HashKey, NodeRecord, StaySelfHasher> node_record_map_;

(StaySelfHasher is my favorite two lines in the repo: the key is a Zobrist hash, which is already uniformly distributed, so the map's hasher is the identity function.)

This is a transposition table, and it quietly upgrades the search from a tree to a DAG: two different move orders reaching the same position share one node and one set of statistics, for free. Whenever different move orders transpose to the same hashed state, that's not a micro-optimization — it's extra effective simulations without simulating.

Zobrist hashing, incrementally

Recomputing a position's hash from scratch would cost O(board area) per move. Zobrist hashing makes it O(changed stones): XOR out what left, XOR in what arrived.

HashKey GetHash(const FullBoard<BOARD_LEN> &b) const;              // full
HashKey GetHash(HashKey hash, const BoardDifference &chng) const;  // incremental

The scheme is beautifully dumb. At startup, generate one random 64-bit number for every (point, state) pair — FoolGo's table is literally board_hash_[81][3] — plus numbers for the side to move and for each possible ko point. A position's hash is the XOR of the numbers matching its current contents. Everything rests on one property: XOR is its own inverse, so XOR-ing the same number twice removes it. "Update" and "undo" are the same operation, and a move's hash cost is proportional to what the move changed:

h' = h ^ Z[d5][EMPTY]      // d5 stops being empty...
       ^ Z[d5][BLACK]      // ...and becomes black
       ^ Z[BLACK] ^ Z[WHITE]   // turn marker: XOR out "Black to move", XOR in "White to move"
       // plus one pair of XORs per captured stone, if any

The turn-marker pair is there because whose move it is is part of the position. Look back at the diagram: with Black to move, e5 gets rescued; with White to move, e5 gets eaten — opposite fates, identical stones. If those two states hashed the same, the transposition table would merge them and their statistics would be garbage. So exactly one of two random "to move" numbers is always XOR-ed into the hash, and each move swaps them. Our connection at d5 thus touches four numbers; a capture would touch two more per removed stone. Nothing else on the board is looked at — which is what lets forty thousand games per second afford a fresh hash after every single move.

The hasher also carries a table for the ko point (ko_hash_, plus a no-ko number), and the reason is subtle. After a ko capture, one point is temporarily illegal to play. Two boards with identical stones but different ko status therefore have different legal moves — and if they hashed the same, the transposition table would happily merge them into one node, letting statistics gathered in one position answer questions about the other. Getting ko into the hash is the kind of detail that costs you an afternoon of debugging exactly once.

UCB1, verbatim

The selection policy is the classic formula, and the code is honest about it:

float Ucb(const NodeRecord &node_record, int visited_count_sum) {
  return node_record.GetAverageProfit()
      + sqrt(2 * log(visited_count_sum) / node_record.GetVisitedTime());
}

Average observed value plus an optimism bonus that shrinks as a node gets visited: exploitation plus exploration in one line. Here it is with numbers — three candidate moves, a thousand playouts spent so far:

parent visits N = 1000          2·ln N ≈ 13.8

move   visits n   avg profit   bonus √(2·ln N / n)    UCB
A         700        0.52             0.14            0.66
B         250        0.46             0.24            0.70
C          50        0.38             0.53            0.91   ← next playout goes here

The next simulation goes to C — the move with the worst observed average. That's not a bug; the bonus is uncertainty made numeric. After only 50 samples, the search cannot yet distinguish a bad move from an unlucky one, so C's claim on the budget is still large. If C keeps disappointing, its average stays low while its bonus shrinks like √(ln N / n), and the playouts drift back to A, whose higher mean now stands on 700 samples. Spend where confidence is thinnest, harvest where confidence is strongest — the formula does both without a single special case. Everything else in the search exists to make evaluating it cheap, millions of times.

Multithreading by mutual avoidance

FoolGo searches on multiple CPU threads sharing the one transposition table (a mutex guards it). The interesting choice is how threads avoid redundant work: rather than implementing virtual loss — the standard trick of temporarily penalizing a node while a thread explores it — FoolGo simply forbids a thread from descending into a node a peer is currently exploring. Here's what that means at the node we just scored:

thread 1 arrives:  UCB says C (0.91) → descends into C, marks it in-progress
thread 2 arrives:  UCB says C — but C is taken → settles for B (0.70)
thread 3 arrives:  C and B both taken → gets pushed to A (0.66)
threads return:    marks cleared, statistics updated, selection sees fresh numbers

Thread 2 wanted C — the policy's actual choice — and got the runner-up; thread 3 got third-best. That's the distortion: under contention, the search behaves as if the top candidates were briefly invisible. Virtual loss is the gentler version of the same idea — a temporary penalty instead of a wall, so a favorite whose lead is big enough can still absorb several threads at once. FoolGo's wall is cruder. But it's a few lines, it can't deadlock, and it was enough to scale a hobby engine across the cores of a laptop.

How the grown-ups solved the same problems

FoolGo's choices get more interesting next to the famous open-source engines that came after it — Leela Zero (2017, the community AlphaGo-Zero reproduction) and KataGo (one of today's strongest open-source Go engines) — and the ones that came before it: GNU Go (whose board code dates to the 1990s), Pachi, and libego (Łukasz Lew's minimalist MCTS library). I went digging through their codebases to compare notes on the same subproblems.

One skeleton, five engines

Five engines, written independently across more than two decades, landed on the same core data structure — flat arrays over board points, a circular linked list threading each string's stones, one stone as the representative. The names barely differ: GNU Go (1990s) has string_number[] and next_stone[], Pachi has group_at[] and groupnext_at[], FoolGo has list_head_ and next_, Leela Zero has m_parent[] and m_next[], KataGo has chain_head[] and next_in_chain[]. There is no lineage here — the problem shape forces the answer — and GNU Go's comment from the 1990s could caption them all: "the stones in a string are linked together in a cyclic list." Five engines, two decades, one skeleton.

Six ways to count a breath

The chain skeleton converged; liberty bookkeeping went six different ways:

GNU Go keeps an exact count plus the full list of liberty coordinates (string_libs[].list[]) — a classical engine wants to reason about specific liberties, not just count them.
Pachi caps the list at ten, refilled lazily — its own comment admits libs "is only LOWER BOUND for the number of real liberties!!!" Playouts rarely need more than "is this 0, 1, or 2."
libego never dedupes at all: pseudo-liberties plus algebra. Its Chain stores the count, sum, and sum of squares of its pseudo-liberty vertices — a chain is in atari exactly when count · Σx² = (Σx)², and the atari point is sum ÷ count.
FoolGo: the exact set as a bitset — merge is OR, dedup by construction.
Leela Zero: exact scalar counts, paid for with a dedup walk at merge time (plus a pseudo-liberty helper for rough checks).
KataGo: exact scalar counts, incremental, with bound-estimating shortcuts.

Same invariant, six answers — ranked, roughly, by how much each engine wants to know about its liberties versus merely count them.

Three clauses, rediscovered

Playing a move converged even harder than the data structures. FoolGo's three-clause suicide test — empty neighbor: no; friendly with two-plus liberties: no; enemy with exactly one: no — turns out to be structurally identical in Leela Zero's is_suicide and KataGo's isSuicide: the same three clauses, in the same order — independent rediscovery, because the test is simply the minimal correct answer the rules allow. There is no fourth clause to invent: a move survives by gaining a liberty directly, joining something that outlives it, or making room by killing. Leela Zero merely bolts a pseudo-liberty fast path onto the front (if (count_pliberties(i)) return false; — any adjacent empty point settles it immediately). The real differences live at the edges. Pachi caches, for every point, how many of its neighbors are black, white, or off-board — immediate_liberty_count is just 4 minus those counts — so the commonest clause is answered without visiting the neighbors at all. And GNU Go plays moves reversibly: every board mutation is pushed onto a change stack (its comments report 20–30 entries per typical move) so the classical engine can read out a line and take it all back on a single board — whereas the playout engines, FoolGo included, never undo anything: they copy the board and let the copy die with the game.

The wall, the tax, and no locks at all

Parallel search reads as generations of one lineage — though not a chronological one. Fuego, the strong open MCTS engine of FoolGo's own era, had already gone further than everyone: a fully lock-free multithreaded tree search (Enzenberger & Müller, 2009) — three years before FoolGo's global mutex. Its source rewards reading even now: each thread allocates nodes from its own pre-allocated array and links them to the parent only once fully initialized; if several threads expand the same node, the last writer wins and the others' work — including value updates already made — is knowingly thrown away. Correctness traded against ever waiting. Every node field is volatile, virtual loss is already there (m_virtualLossCount), and the in-code docs cite the exact chapters of the Intel manual whose memory-ordering guarantees the whole scheme leans on. The fool was not early; he was simple. Within the lineage of the simple: FoolGo's forbid-peers rule (2012) is the blunt ancestor, Leela Zero (2017) does it properly with textbook virtual_loss() / virtual_loss_undo() folded into node evaluations, and KataGo is the industrial endpoint — virtual loss as a tunable (numVirtualLossesPerThread, an atomic counter per node), over a sharded node table with a mutex pool and fully atomic stats structs.

From tree to graph

The part that surprised me most. Leela Zero, for all its strength, uses a literal pointer tree and does not merge transpositions; there's even a comment in uct_select_child about counting parent visits manually "to avoid issues with transpositions." But in v1.11.0 (March 2022), KataGo shipped what its release notes call "a new stronger MCTS implementation that operates on a graph rather than a tree" — transposed move orders recombined into shared nodes, keyed by hash in a sharded node table. Which is, architecturally, what FoolGo's unordered_map<HashKey, NodeRecord> was doing in 2012.

Before I take a bow: KataGo's author also wrote a first-principles document explaining that naively applying tree-MCTS statistics to a DAG — exactly what FoolGo does — is unsound: shared nodes break the running-statistics formulation in subtle ways, and doing it correctly (plus handling ko and superko) is the actual hard part. So no, my hobby engine did not do graph search before KataGo. It wandered into the right building ten years early, without knowing why the floor needed reinforcing. KataGo's contribution was the reinforcement.

What it adds up to — and where it stops

Flat arrays, bitwise liberty tracking, incremental hashing, a pointer-free "tree," and threads that stay out of each other's way: 40,000 games per second on a laptop from 2014.

And yet: beginner strength. That plateau is the honest lesson of the repo. Uniformly random playouts are a terrible evaluation function, and no amount of throughput fixes their bias — stronger engines of that era spent their effort on playout policy, and then 2016 arrived and neural networks replaced blind rollouts with intuition. AlphaGo kept the tree search; it swapped out exactly the part FoolGo had made fast.

The repo is github.com/chncwang/FoolGo — readable on purpose, PRs welcome, and still, I'd argue, one of the clearer ways to see what game-tree search looks like with the covers off.

These days I build ClinTrialFinder, an AI-powered clinical-trial matcher, and write about building it at chncwang.substack.com.

Claude Code can make videos: it records the app, narrates with ElevenLabs, and syncs audio to video automatically

Chauncey Wang — Sat, 15 Aug 2026 03:40:08 +0000

I'm a solo builder. I needed a 2-minute product demo for ClinTrialFinder — a free tool I built that matches cancer patients to clinical trials. I can fumble through OBS and iMovie, but I'm not proficient — and Claude Code does it faster.

So I asked Claude Code — an agentic coding tool — to make it. And it did: a narrated walkthrough where the voiceover lands exactly on the on-screen action. I never opened a screen recorder. I never opened a video editor. I never manually lined up a single caption to a single frame.

Here's the video it produced. This post is about the three things the agent did to make it — because I think that combination is new.

1. It recorded the app — no screen recording

Instead of me screen-capturing a session by hand, the agent wrote a Playwright script that drives the real, live web app: it opens the site, fills out the 10-step patient wizard with a synthetic case, submits, and records the finished results page — all headless, straight to video.

That means no manual take, no re-shooting when I fumble a click, no "oops the mouse jittered." The recording is code, so it's deterministic and repeatable. When the product changes, the agent re-runs the script and out comes a fresh clip. It even injected a fake cursor that glides between elements, because a headless recording has no real mouse pointer.

2. It generated the narration — no microphone

I didn't record a voiceover. The agent wrote the narration script, then called the ElevenLabs text-to-speech API to synthesize it in a clean, consistent voice. If I want to change a line, it edits the text and regenerates that clip in seconds — no re-recording, no "let me find a quiet room," no matching my tone across takes.

// the agent calls ElevenLabs per narration phrase
const res = await fetch(`https://api.elevenlabs.io/v1/text-to-speech/${VOICE}`, {
  method: 'POST',
  headers: { 'xi-api-key': KEY, 'Content-Type': 'application/json' },
  body: JSON.stringify({ text, model_id: 'eleven_multilingual_v2' }),
});

3. It aligned audio to video — no timeline editor

This is the part that normally needs a human in a video editor, dragging clips around a timeline until the words match the picture. The agent did it automatically, and this is the genuinely clever bit:

While recording, it logs the timestamp of every key on-screen moment — the submit click, the results appearing, a trial opening, the "copy to AI" dropdown.

const beat = name => console.log(`BEAT ${name} @ ${elapsed()}s`);
// BEAT submit-click   @ 46.83s
// BEAT results-shown  @ 68.73s
// BEAT trial-open     @ 90.17s

Then it cuts the narration into phrases, one per beat, and places each phrase at its beat's timestamp in the final mix (ffmpeg's adelay). The result: when the voice says "now it goes to work," the button is being clicked; when it says "open any trial," the trial is opening. The sync falls out of the recording itself — no dragging, no eyeballing, no manual alignment.

Why this matters

Making a product demo used to mean: screen-record a take, write a script, record a voiceover, then sit in an editor syncing them. Four manual steps, each needing a skill (or a person).

Here it was one conversation with an agent. The whole pipeline is code — record → narrate → align → assemble — so it's:

Repeatable: product changes? Re-run. Fresh, re-synced video.
Deterministic: same framing and pacing every time, no shaky live take.
Faster than me: I can muddle through OBS and iMovie, but slowly and not well — the agent does it faster, and I don't have to.

The agent also quietly handled the fiddly parts I'd never want to — a site that won't render headless (screenshot + Ken Burns instead), audio mixing that silently halves volume, a blank loading frame that throws off the timing. I didn't debug any of it; it did.

The shift, for me, is that making a demo video is now something you ask for, not something you produce. The example here is my own project — ClinTrialFinder, a free clinical-trial matching tool for cancer patients — but nothing about the approach is specific to it. If you have a web app and an agent, you can have a narrated, synced demo without touching a recorder or an editor.

Building a One-Person Software Shop with Claude Code

Chauncey Wang — Thu, 06 Aug 2026 11:30:07 +0000

I build a clinical-trial matching product alone — no cofounder, no team. Just one person, an AI coding agent, and a set of conventions that keep "vibes-based solo dev" from falling apart across weeks of parallel work.

The first version was a command-line tool, open-sourced. Turning it into a real web app — something a patient could open in a browser and trust with their situation — is the jump where a lot of solo projects quietly stall. I made it: today it's a live web app with ~30 drug pages, dozens of disease-specific trial landscapes, and a matcher real cancer patients use to find trials.

I'm also a cancer patient; I built this partly because I needed it to exist. So I care less about it being impressive than correct — a wrong trial match wastes a sick person's time. That constraint is the whole point: the interesting part isn't that an AI writes my code — it's the scaffolding that makes its work trustworthy when I'm not watching every keystroke.

A note on the tool first, then the transferable patterns — most of them scar tissue from something that broke once. Everything's sanitized — fake IPs, generic paths, invented tasks; the real secrets stay home. And the fake IPs aren't only for the article: the real task files never held a raw IP or email, and the server logs that hold IP addresses are purged within 14 days — matching the privacy policy.

Why Claude Code, specifically

People ask why Claude Code and not one of the other coding agents. Two reasons — one soft, one hard.

The soft one: it infers intent from less. I can hand it a terse, half-specified ask — "the ranking's off for first-line patients, dig into it" — and it usually fills the gaps the way I meant, not the way I literally typed. That matters when I'm running several sessions and can't write a full spec for each. Subjective, not a benchmark — but it's what keeps me reaching for it.

The hard one: my review workflow feeds real patient-submission data to the agent — I pull a patient's result set to check whether the matcher did right by them. ClinTrialFinder's privacy policy names Anthropic's Claude as a tool for exactly that — "quality-checking match results, diagnosing issues." Using Claude Code for the review keeps the implementation matched to what patients were told. Other disclosed vendors handle other steps; the point isn't that Claude is the only option — it's that the words I show patients and the tools I run stay in lockstep.

The core bet: your task tracker is a git repo

Most people reach for Jira, Linear, Notion. I keep every task as a plain file in a git repo of its own:

project-manager/            # a git repo
  tasks/
    412-fix-ranking-edge-case.html
    413-wizard-validation.html
    414-review-user-submission.html

One file per task, fixed shape — priority, status, the problem, the fix, verification plan, links to related tasks. No API, no board, no login. Just files in git — 500+ of them now, and the flat directory has never needed anything fancier.

Why this beats a tracker for an AI-assisted solo shop specifically:

Version-controlled. The backlog is plain files in git — every status flip and edit is a diff you can read, branch, or revert, exactly like source. It's a history you own and grep, not rows in someone else's database.
Queryable with the tools the agent already has. "Next free task number?" is ls | sort. "Which tasks touch the ranking bug?" is grep -rl. No integration — just grep.
Linkable and self-documenting. Tasks cross-link; six weeks later the why is one click away, written at the time, not reconstructed from memory.

The insight: when your teammate is an AI agent, your project management should be a git repo of plain files it can grep — not a SaaS it has to poke through an API. (I use HTML for the files — renders and links nicely — but the format is the least interesting part; Markdown would do.)

One file sits on top of that flat directory: a single overview — a hand-maintained index that splits the backlog into the full list and the sprint (what to actually work on now). The task files are the atomic units; the overview is the priority view.

And two kinds of judgment get their own role. Worker sessions append tasks freely as they surface — a bug found mid-fix, a spinoff from a review — but none of them promotes itself into the sprint. That call belongs to a dedicated "CEO" session: a Claude Code session that reads the backlog, decides what's worth doing now (via its own update-active-sprint skill), and proposes the forward-looking work the workers won't file on their own — new features, UI refinements, the product's next move. Reactive filing is mechanical and belongs to whoever hit the problem; setting direction — both prioritizing and proposing — is judgment, so I concentrate it in one named role instead of smearing it across four sessions that each think their task is the important one.

Parallelism — four sessions, each a standing role

Here's where it gets honest. I often run four Claude Code sessions at once, against a shared backlog and shared repos — but they're not four workers chewing through the same queue. Each holds a standing role: a CEO that prioritizes and proposes new work, an SEO/content session that also watches the traffic and files a review task whenever a patient submits, and two full-stack engineers working different tasks in parallel. And full-stack undersells it — one engineer session will write HTML and CSS, write the Python behind it, review a real patient submission and file the fix tasks its defects reveal, and rewrite a matching prompt, all in one afternoon; a single context spans what used to be four specialties.

That engineer session also writes as it works — logging its findings and progress back onto the task file, so the task becomes a running record of what was tried and learned, not a write-once spec. That record earns its keep: when a mid-work diagnostic disproves the task's own premise — the "bug" was correct behavior, the evidence was confounded — the CEO reads the update and demotes the task back to the backlog instead of shipping it. The sprint self-corrects.

And these sessions don't close when a task is done — each stays open and picks up the next thing in its lane, so it accumulates context instead of starting cold each time. A big multiplier for one person — the nearest thing to a team I've got.

They all run from the same home directory — one shared filesystem, not four sandboxes. That's mostly the point: the SEO session opens the analytics export I just downloaded to ~/Downloads, any session can reach any repo in the tree, and nobody has to shuttle files between isolated boxes.

The one place that sharing bites is the two engineers — point both at the same working tree and they'll clobber each other, one's half-finished edit sitting there when the other runs its tests. So each works in its own git worktree: a separate checkout on its own branch, sharing the repo's history but not its uncommitted state. Shared filesystem, isolated working copies — both build, commit, and test at once without ever seeing each other's in-progress files, and each branch merges to main only after I've reviewed it.

Those roles don't act in a vacuum — here's the full lifecycle they move a task through, for both kinds of task the shop runs.

Of those 500-plus task files, about 90 were spawned by reviewing real patient submissions — the rest are features and fixes the CEO session or I dreamed up. That ratio is the part I care about: every genuine submission gets audited, and the ones that expose a gap become fix tasks. Roughly 230 of the whole are done or shipped; the rest are a living backlog. The traffic isn't huge — a few hundred submissions over the tool's life, at least 100 real enough to enter an email to get their results back.

Every change gets its own URL before it's real

The counterpart to building in parallel: every change to the product needs somewhere to run that isn't production and isn't the other tasks in flight. So every task that touches the product gets its own isolated instance — a fresh clone of the repo, its own service on its own port, reachable at its own private review URL behind a reverse proxy.

Under the hood it's one small nginx config: a location block per instance, each routing a private sub-path to that instance's local port (/task-<name>/ → a service on 127.0.0.1:<port>). Spinning one up is a clone, a service on a fresh port, and a few lines of proxy. The honest ceiling is memory — each instance is a full running copy of the app, so the dev box has to hold several at once; that, more than anything, caps how many tasks I can keep live for review at the same time.

At any moment I might have several live, independently-viewable copies of the product up — one per task — each showing exactly that task's changes and nothing else. I open the URL, click through the actual rendered thing, and see the change in situ before it's anywhere near a user.

That's the deploy gate: nothing reaches production until I've looked at it running on its own instance. Build on an isolated branch → spin up an instance → review the real rendered page at its URL → approve → merge → deploy. The agent never pushes to prod on its own — the "approve" is mine, and it's a look at a running thing, not a diff.

The transferable pattern: give every unit of parallel work its own running, reviewable instance. "Does it actually work, rendered, in isolation?" is a question you can only answer if the work has somewhere to live that isn't production and isn't your other tasks. The isolation is what makes parallel and careful compatible.

Skills that encode judgment, not macros

Claude Code lets you define skills — named routines the agent runs on command. The naive use is automation: "deploy the site," "run the tests." Useful, but shallow.

The valuable skills encode how a domain expert thinks — the judgment, not just the steps.

My highest-value one reviews a user submission. It doesn't just dump data. It:

Pulls the result across three pools (shown to the user / computed-but-hidden / rejected).
Reconstructs what the user actually did from the logs — did they wait, did they click through, did they leave.
Checks whether a genuinely-good result got silently dropped before the scoring stage even saw it (a whole class of bug that's invisible if you only look at what was shown).
Cross-references anything it finds against a catalog of known past defects.

That's not a macro. That's a reviewer's worldview — what to suspect, what to verify, what caveat to attach to a claim — written down once and rerun consistently. When I invoke it, I'm not saving keystrokes; I'm borrowing a disciplined second brain that never gets lazy on step 3.

The transferable pattern: your best skills should capture the reasoning of your most careful self, especially the checks you'd skip when you're tired. Automation saves time. Encoded judgment saves you from your own shortcuts.

The operating manual: where corrections become rules

There's a file Claude Code reads at the start of every session — CLAUDE.md. Mine has grown into an operating manual: the project's standing rules, accumulated one mistake at a time.

Deployment discipline. Never deploy to production without explicit sign-off. Never edit files directly on the server — always local, commit, push, pull.
Expected behaviors. How to format a task, when to sync which repo, what "done" means.

Every one of those started as a bug. The first time the agent restarted prod and interrupted a live request, the fix wasn't "don't do that this time" — it was a line in the manual so it never happens again.

That's the load-bearing pattern: a correction that lives only in a chat window evaporates; a correction written to a file that loads next session is a permanent behavior change. The manual is the accumulated scar tissue of the project — the difference between an agent that repeats your mistakes and one that compounds your lessons. The test of a good rule is simple: can the mistake it came from happen again?

What actually transfers

Strip away my specifics and here's what I'd hand another solo builder working with an AI agent:

Make your project management out of primitives your agent is fluent in. Files, grep, git — not a SaaS behind an API. The backlog should be as greppable as the code. Let worker sessions append to it freely, but concentrate direction-setting — both prioritizing and proposing new work — into one role, a dedicated "CEO" session, instead of every session promoting its own.
Run parallel sessions as standing roles, not a shared queue. Give each a lane and let them share one filesystem so they see the same world — then isolate what would collide (a git worktree per engineer) so "more hands" never becomes clobbered work.
Give every change somewhere to run before production. Isolated per-task instances behind a reverse proxy let you review the real rendered product — so "approve" is a look at a running thing, not a diff, and prod stays a deliberate step.
Write skills that encode judgment, not just steps. Capture the careful reasoning you'd skip when tired. That's the compounding asset; automation is just the floor.
Keep an operating manual, and treat every correction as a candidate rule — written to a file that loads next session, not left in a chat window that evaporates. The manual is your project's scar tissue; the test of a rule is whether the mistake it came from can happen again.

None of this is about the AI being the smartest. It's about building the scaffolding that makes an AI agent's work trustworthy — which, when the output affects a sick person looking for a trial, is the only thing that matters.

I build ClinTrialFinder solo, with Claude Code, as a patient myself. If any of these patterns are useful in your own shop, I'd genuinely like to hear how they hold up — the failure modes especially.