Two days. Thirty-five thousand parameters. A capability the brain layer couldn't reach was sitting one circuit away the whole time.
Part 21 ended on three architectural walls and a choice. The brain layer I'd just spent two months restoring couldn't do property transfer. The dispatch needed an unbounded supply of guards. The two-stage reasoner had been actively poisoning predictions before I disabled it. Three options for what to do next. Change the existing architecture. Build something separate. Accept the ceiling.
I'd already seen what the separate-thing path looked like, because it had landed a week earlier and I hadn't realized at the time it was the answer.
The RelationalCircuit had been a small experiment to handle a specific dispatch failure. The frontier battery had a bucket called role_requirement that tested questions like "what does a teacher need?" and "what does a doctor need?" The brain layer scored zero out of five on the bucket. Three-quarters of the expected answer concepts (student, mammal, wheel, liquid, fruit) weren't even in Origin's vocabulary, because V2C was constrained to concepts the encoder had learned, and those concepts hadn't been learned. The standard fix would have been to expand V2C through a joint encoder retrain, which was the move that had been failing for vocabulary work generally. The May failures were still fresh.
A friend had suggested an alternative. Build a separate circuit with its own output concept space. The encoder stays frozen. The V2C list stays unchanged. The brain layer stays unchanged. Just a small dedicated circuit that produces a small dedicated vocabulary, wired into dispatch when a specific question shape arrives.
I built a small version of it in a few hours. Eighteen output concepts in a hand-curated vocabulary. A linear projection from the encoder's pooled output to a relation embedding. A small attention layer. A predict head. A self-gate that decided whether the circuit's output should be consulted at all. Two hundred and ninety training pairs in the shape "subject + relation → object." Forty epochs in three seconds of GPU. One hundred percent accuracy on a fifty-eight-pair held-out test set.
The frontier battery's role_requirement bucket went from zero to two out of five. The remaining three needed concepts the RelationalCircuit hadn't been trained on yet, but the pattern worked. Origin could now correctly produce "a teacher needs a student" without expanding V2C or retraining the encoder or touching the brain layer.
At the time I'd taken the win and moved on. The brain layer work seemed more important. The brain layer was the architecture from the v1 design. The brain layer was supposed to do the reasoning. The RelationalCircuit was a one-off patch for a narrow capability gap.
Then Stage 1 of the curriculum hit its ceiling. Property transfer at eight percent. The brain layer was structurally unable to do what I needed it to do, and a small dedicated circuit had been doing exactly that shape of thing for a week. I'd been calling it a patch. It might have been the pattern.
I built PropertyCircuit the next day.
The structure mirrored RelationalCircuit almost exactly. A subject input from the encoder's frozen embeddings. An attention layer for context. A small reasoning MLP. A predict head over a hand-curated property vocabulary. A self-gate that decided whether the circuit's prediction was confident enough to be useful. Thirty-five thousand three hundred and fifty-seven parameters. About a quarter of one percent of the brain layer's size.
The training data was the same fifty-three property pairs that had hit the curriculum's eight-percent transfer ceiling. Fire is hot. Ice is cold. Rock is heavy. Apple is red. Two hundred epochs took three seconds on the GPU.
The pair recovery was perfect. Fifty-three out of fifty-three. The circuit had memorized the training data, as expected. The interesting number was the held-out property transfer eval. The same twenty-five probes that had returned two correct on the brain layer.
The PropertyCircuit returned seven correct.
Three and a half times the brain layer's transfer rate. The specific case I'd been staring at all week, stone going to heavy, worked. The brain layer had been returning falls, sinks, and lift for stone. The PropertyCircuit returned heavy. Arctic went to cold. Oven went to hot. Milk went to white. Vinegar went to sour. The properties had transferred. Not perfectly. Not even at fifty percent. But at thirty-five thousand parameters, three seconds of training time, and zero impact on any other part of the system, the transfer rate was three and a half times higher than the architecture I'd just spent two months restoring.
I sat looking at the numbers and tried to make them mean less than they meant.
The brain layer had 4.5 million trainable parameters. The PropertyCircuit had thirty-five thousand. The brain layer took eleven seconds to train. The PropertyCircuit took three. The brain layer needed careful balance against the existing batteries to avoid regression. The PropertyCircuit was structurally separate from anything that affected the batteries. It couldn't regress them because it didn't touch them. The brain layer required gradient discipline, dead-slot masking, domain isolation, and frozen embeddings to even avoid making things worse. The PropertyCircuit had a single training script, a single small vocabulary, a single specific job, and it had outperformed the brain layer at that job by a factor of three and a half on the first try.
I'd been building monoliths.
The architecture I'd spent two months patching wasn't wrong because of any specific bug. It was wrong because of its shape. One model trying to learn everything. One predict head trying to output every concept. One brain layer trying to compose every property and every rule. Every fix had to be balanced against everything else, because everything was wired together. The gradient pollution problem. The domain head saturation. The two-stage reasoner collapsing into noise. The property-transfer ceiling. They all came from the same source. A monolith optimized for one thing tends to be worse at every other thing, and a monolith optimized for nothing tends to be mediocre at everything.
The RelationalCircuit and the PropertyCircuit weren't optimizing for nothing. They were each optimizing for exactly one thing. The relational one knew about subject-relation-object triples. The property one knew about concept-property assignments. The encoder didn't know they existed. The brain layer didn't know they existed. They sat off to the side of the main pipeline, each one a small expert with its own narrow vocabulary, and dispatch consulted them when their specific question shape arrived.
I'd seen this pattern before, in retrospect. The way human cognition seems to be structured. Specialized brain regions, each one carrying a slice of capability, coordinated by something faster and dumber than the regions themselves. The amygdala does threat detection because that's what the amygdala does. The hippocampus does episodic memory because that's what the hippocampus does. The thalamus routes between them. Nothing tries to do everything. The pattern is the architecture.
The v1 brain layer design had reached toward that pattern with the thalamus router and the routed micro-circuits. The structure was right in spirit. The implementation had become a monolith because every routed circuit shared the same predict head over the same 8,882 concept output space. The pattern was modular at the routing layer and monolithic everywhere else. The PropertyCircuit and the RelationalCircuit were what the v1 design would have looked like if the modularity had gone all the way down. Each circuit with its own vocabulary. Each circuit trained independently. Each circuit's gradients touching only the circuit's parameters.
The implications stacked up faster than I could write them down. The vocabulary growth problem dissolved, because individual circuits could have small vocabularies hand-curated to their capability. The gradient pollution problem dissolved, because each circuit's gradient flow stayed inside its own parameters. The regression risk dissolved, because adding a circuit didn't touch existing ones. The training cost dissolved, because 35 thousand parameters, yes 35 thousand trained in three seconds. The architecture had been the constraint all along, and the constraint had been the assumption that v2 needed one model doing everything.
I commit-and-merged the PropertyCircuit work. Wired it into dispatch behind a property_query intent. Started thinking about which capability gap to tackle next. There was a list of categories the frontier battery had surfaced that the brain layer couldn't handle. Each one looked like its own small circuit. The frontier was a roadmap.
The brain layer wasn't going away. It still did the compositional reasoning work that didn't fit into a single dedicated circuit. But it had stopped being the thing the architecture was organized around. It was becoming one of the experts in a small fleet, not the main pipeline that everything else had to negotiate with.
I'd been building v2 for months. The decision sitting in front of me now was whether v2 was the right name for what came next, or whether the modular fleet was different enough to be its own thing entirely. I knew which way I was going to lean. I wanted to see it run for another day or two first.
One guy. One GPU. One $1,800 computer in Arizona. Still building.
Origin is developed at Fallen Angel Systems with the Genesis framework — NVIDIA Inception member. (USPTO Application #64/016,973, #64/017,567). FAS Guardian defends production AI systems from prompt injection in under 3ms. FAS Judgement is the open-source attack console that finds the gaps. Defense. Offense. Creation.
fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub
Questions or consulting inquiries: josh@fallenangelsystems.com
Top comments (0)