DEV Community: Josh T

Origin Part 23: V3

Josh T — Mon, 10 Aug 2026 13:00:43 +0000

I spent six months trying to make v2 work. Then I built a small thing on the side that worked better than v2 ever had. The decision that followed wasn't whether to change architectures. It was how fast.

Part 22 ended on a pattern I'd been seeing without quite seeing. PropertyCircuit and RelationalCircuit had each landed a capability the brain layer couldn't reach, at a fraction of the parameter count, in seconds of training, with zero impact on anything else in the system. The implications sat in the session notes for a couple of days while I ran the numbers in different ways trying to find a reason they didn't mean what they obviously meant.

I couldn't find one. The pattern was real. The reason it was real was structural. A monolith optimized for one thing tends to be worse at every other thing. A small circuit optimized for one thing tends to be better at that one thing than any general-purpose model would be, and the cost of building it is small enough that you can build a lot of them.

I sat down at my computer staring at the screen, running the design through my mind. I had been doing the engineering work in a collaboration for months. I held the design vision and the final say on what shipped. The conversation about v3 had to be the two of us together, because nether one of us would have noticed the pattern alone.

The conversation took most of a day. The output was a file in the v3 repo called TRUTH.md with ten rules in it. Two-way agreement required for any architectural change. No regex patterns in dispatch unless they used what Origin already understood. Sandbox before shipping. Trace impact before shipping. Honesty floor: Origin only says what it knows. Modularity all the way down. A few others.

The rules weren't aspirational. They were the patterns we'd already learned the hard way over fifteen blog posts of failures. We'd just never written them down in one place where they could constrain the next decision.

The architecture that came out of the conversation was different from v2 in shape, not just in degree.

v2 was an encoder followed by a dispatcher followed by a composer. The encoder produced a fired-concept set. The dispatcher routed based on patterns. The composer generated text. The brain layer sat downstream of the dispatcher as one of several intent handlers. The whole pipeline was a series of stages, each one trying to do its part of "understand this input and produce a response." When any single stage struggled, the whole pipeline struggled.

v3 was a fleet. The encoder still fired concepts. The dispatcher still routed. But what got routed to was a collection of small specialized circuits, each one with its own narrow vocabulary, its own training data, its own predict head, its own gradient flow. PropertyCircuit handled property assignments. RelationalCircuit handled subject-relation-object triples. CausalCircuit handled cause-effect chains. ChainReasoner composed across them when a question needed multi-step inference. Each circuit was thirty to fifty thousand parameters. Each one trained in seconds. Each one could be added, retrained, or retired without touching the others.

The brain layer didn't go away. It stayed for the kinds of compositional work that didn't fit cleanly into a single small circuit. But it stopped being the architecture's center of gravity. It became one expert among several, consulted when its capability matched the question shape, ignored otherwise.

The encoder didn't go away either. But its job got cleaner. v3's encoder fires a concept set. That's it. It doesn't have to produce dense activations across the full vocabulary. It doesn't have to balance against downstream prediction heads. It fires what it knows about the input and lets the circuits decide what to do with it. The vocabulary growth problem dissolves once the encoder is allowed to be sparse instead of dense.

The dispatcher's job got cleaner too. v2's dispatcher had been accumulating regex guards and intent priorities and exclusion lists for months because it was the only place to handle the gaps everywhere else. v3's dispatcher was a router. Look at the input's shape. Pick the circuits whose capabilities matched the question. Let them produce outputs. Let the composer turn the highest-confidence output into text. The dispatcher didn't need to be the safety net for everything because everything wasn't its responsibility anymore.

The composer's job stayed mostly the same, but with cleaner inputs. Where v2's composer had been stitching together responses from raw concepts and definitions and pre-written templates, v3's composer received structured output from circuits: the property, the relation, the effect concept, the chain trace. The composer's job became saying what the circuits had already produced, not figuring out what to say.

The whole shape was different in a specific way. v2 had been one model trying to do everything, with capability layers stacked on top of each other and dependencies running in every direction. v3 was a coordinated fleet of small models, each one with one job, with the coordination layer kept as thin as possible. The architecture that human cognition seems to use. Not because we were trying to copy biology. Because that pattern was the one that the engineering kept landing on whenever we let the data choose.

The transition wasn't a single commit. v3 already existed as its own repository because the encoder retrain work had needed separate infrastructure. The PropertyCircuit and RelationalCircuit had been built there. The TRUTH.md commit in May was the moment we wrote down that the new direction was permanent, but the architecture had been quietly forming for weeks underneath the v2 work.

v2 kept serving production while v3 came up. I'd been doing this kind of cutover for long enough to know that promising "we'll rebuild it cleanly next time" almost always meant either rebuilding it dirty later or never rebuilding it at all. v3 had to demonstrate it could carry production before v2 stepped down. That meant the modular circuits had to handle every capability v2 currently handled, which meant building more circuits, which was now cheap. Each new circuit was a hand-curated training set, a small Python file, a few hundred lines of dispatch wiring. The first one had been hard. The second one had been a copy of the first one with the vocabulary changed. The fifth one would take an afternoon.

The first month of v3 work was building. PropertyCircuit shipped. RelationalCircuit shipped. CausalCircuit shipped. A thalamus router that picked which circuit handled a given input, trained on small amounts of routing data. A chain reasoner that composed across circuits when multi-step reasoning was needed. The brain layer got rewired to use the new circuits as inputs instead of trying to be the whole reasoning stack itself. The encoder got retrained with a corpus deliberately shaped for sparse multi-label firing instead of dense softmax distribution.

The numbers came back better than v2 had ever been. Compositional reasoning jumped because each step of the composition could be handled by a circuit specialized for it. Property transfer climbed because PropertyCircuit was specifically designed for it. Identity and dialogue surfaces stayed clean because they were handled by separate dedicated heads that didn't have to balance against anything else. The frontier battery, which had been a diagnostic instrument for v2, became a target. Each missed bucket pointed at the next circuit to build.

I'm writing this from the v3 codebase. The brain layer is still there but the architecture is no longer organized around it. The encoder is sparser and sharper than v2's ever was. The circuit fleet handles capability gaps that v2 couldn't have approached without months of careful gradient surgery. The dispatcher is shorter than it was a month ago because most of its regex guards have been replaced by routing to circuits that handle the question shapes natively.

What v3 is, in one sentence: a coordinated fleet of small specialized models with a shared concept vocabulary and a thin routing layer. What that means in practice: the system doesn't try to learn everything in one model. It learns one thing per small model, and the small models work together. The reasoning is traceable. The capabilities are addable. The training is fast. The failure modes are localized.

What v3 isn't: a transformer-style language model. It doesn't generate tokens. It doesn't predict next words. It doesn't have a context window in the LLM sense. It doesn't run on a hundred-billion-parameter foundation model. It runs on a $1,800.00 dollar computer under a desk in Arizona, and the entire system, including the encoder and the brain layer and the circuit fleet and the composer and the dispatch routing, is small enough to fit comfortably on a consumer GPU with room to spare.

That last part keeps surprising people when I describe what v3 actually is. The standard mental model for AI in 2026 is that you need a lot of parameters and a lot of compute, and capability comes from scaling. v3 is the experiment that asks whether capability can come from architecture instead. The early answer, on the parts of the system that have been validated, is yes. The full answer will take a while. But the direction is set now. The fleet is the architecture. The architecture is the work.

v2 is retired. v3 is what comes next. Different in kind, not just in version number.

This could open up a lot of possibilities. If you want to help find out, I'm opening up beta testing to the first fifty people who reach out. This is a basic model. Please don't have high expectations. We're still building it and improving it every day. Come watch it grow. The first fifty testers get free access for life.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

Origin is developed at Fallen Angel Systems with the Genesis framework — NVIDIA Inception member. (USPTO Application #64/016,973, #64/017,567). FAS Guardian defends production AI systems from prompt injection in under 3ms. FAS Judgement is the open-source attack console that finds the gaps. Defense. Offense. Creation.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 22: A Smaller Circuit

Josh T — Mon, 03 Aug 2026 13:00:17 +0000

Two days. Thirty-five thousand parameters. A capability the brain layer couldn't reach was sitting one circuit away the whole time.

Part 21 ended on three architectural walls and a choice. The brain layer I'd just spent two months restoring couldn't do property transfer. The dispatch needed an unbounded supply of guards. The two-stage reasoner had been actively poisoning predictions before I disabled it. Three options for what to do next. Change the existing architecture. Build something separate. Accept the ceiling.

I'd already seen what the separate-thing path looked like, because it had landed a week earlier and I hadn't realized at the time it was the answer.

The RelationalCircuit had been a small experiment to handle a specific dispatch failure. The frontier battery had a bucket called role_requirement that tested questions like "what does a teacher need?" and "what does a doctor need?" The brain layer scored zero out of five on the bucket. Three-quarters of the expected answer concepts (student, mammal, wheel, liquid, fruit) weren't even in Origin's vocabulary, because V2C was constrained to concepts the encoder had learned, and those concepts hadn't been learned. The standard fix would have been to expand V2C through a joint encoder retrain, which was the move that had been failing for vocabulary work generally. The May failures were still fresh.

A friend had suggested an alternative. Build a separate circuit with its own output concept space. The encoder stays frozen. The V2C list stays unchanged. The brain layer stays unchanged. Just a small dedicated circuit that produces a small dedicated vocabulary, wired into dispatch when a specific question shape arrives.

I built a small version of it in a few hours. Eighteen output concepts in a hand-curated vocabulary. A linear projection from the encoder's pooled output to a relation embedding. A small attention layer. A predict head. A self-gate that decided whether the circuit's output should be consulted at all. Two hundred and ninety training pairs in the shape "subject + relation → object." Forty epochs in three seconds of GPU. One hundred percent accuracy on a fifty-eight-pair held-out test set.

The frontier battery's role_requirement bucket went from zero to two out of five. The remaining three needed concepts the RelationalCircuit hadn't been trained on yet, but the pattern worked. Origin could now correctly produce "a teacher needs a student" without expanding V2C or retraining the encoder or touching the brain layer.

At the time I'd taken the win and moved on. The brain layer work seemed more important. The brain layer was the architecture from the v1 design. The brain layer was supposed to do the reasoning. The RelationalCircuit was a one-off patch for a narrow capability gap.

Then Stage 1 of the curriculum hit its ceiling. Property transfer at eight percent. The brain layer was structurally unable to do what I needed it to do, and a small dedicated circuit had been doing exactly that shape of thing for a week. I'd been calling it a patch. It might have been the pattern.

I built PropertyCircuit the next day.

The structure mirrored RelationalCircuit almost exactly. A subject input from the encoder's frozen embeddings. An attention layer for context. A small reasoning MLP. A predict head over a hand-curated property vocabulary. A self-gate that decided whether the circuit's prediction was confident enough to be useful. Thirty-five thousand three hundred and fifty-seven parameters. About a quarter of one percent of the brain layer's size.

The training data was the same fifty-three property pairs that had hit the curriculum's eight-percent transfer ceiling. Fire is hot. Ice is cold. Rock is heavy. Apple is red. Two hundred epochs took three seconds on the GPU.

The pair recovery was perfect. Fifty-three out of fifty-three. The circuit had memorized the training data, as expected. The interesting number was the held-out property transfer eval. The same twenty-five probes that had returned two correct on the brain layer.

The PropertyCircuit returned seven correct.

Three and a half times the brain layer's transfer rate. The specific case I'd been staring at all week, stone going to heavy, worked. The brain layer had been returning falls, sinks, and lift for stone. The PropertyCircuit returned heavy. Arctic went to cold. Oven went to hot. Milk went to white. Vinegar went to sour. The properties had transferred. Not perfectly. Not even at fifty percent. But at thirty-five thousand parameters, three seconds of training time, and zero impact on any other part of the system, the transfer rate was three and a half times higher than the architecture I'd just spent two months restoring.

I sat looking at the numbers and tried to make them mean less than they meant.

The brain layer had 4.5 million trainable parameters. The PropertyCircuit had thirty-five thousand. The brain layer took eleven seconds to train. The PropertyCircuit took three. The brain layer needed careful balance against the existing batteries to avoid regression. The PropertyCircuit was structurally separate from anything that affected the batteries. It couldn't regress them because it didn't touch them. The brain layer required gradient discipline, dead-slot masking, domain isolation, and frozen embeddings to even avoid making things worse. The PropertyCircuit had a single training script, a single small vocabulary, a single specific job, and it had outperformed the brain layer at that job by a factor of three and a half on the first try.

I'd been building monoliths.

The architecture I'd spent two months patching wasn't wrong because of any specific bug. It was wrong because of its shape. One model trying to learn everything. One predict head trying to output every concept. One brain layer trying to compose every property and every rule. Every fix had to be balanced against everything else, because everything was wired together. The gradient pollution problem. The domain head saturation. The two-stage reasoner collapsing into noise. The property-transfer ceiling. They all came from the same source. A monolith optimized for one thing tends to be worse at every other thing, and a monolith optimized for nothing tends to be mediocre at everything.

The RelationalCircuit and the PropertyCircuit weren't optimizing for nothing. They were each optimizing for exactly one thing. The relational one knew about subject-relation-object triples. The property one knew about concept-property assignments. The encoder didn't know they existed. The brain layer didn't know they existed. They sat off to the side of the main pipeline, each one a small expert with its own narrow vocabulary, and dispatch consulted them when their specific question shape arrived.

I'd seen this pattern before, in retrospect. The way human cognition seems to be structured. Specialized brain regions, each one carrying a slice of capability, coordinated by something faster and dumber than the regions themselves. The amygdala does threat detection because that's what the amygdala does. The hippocampus does episodic memory because that's what the hippocampus does. The thalamus routes between them. Nothing tries to do everything. The pattern is the architecture.

The v1 brain layer design had reached toward that pattern with the thalamus router and the routed micro-circuits. The structure was right in spirit. The implementation had become a monolith because every routed circuit shared the same predict head over the same 8,882 concept output space. The pattern was modular at the routing layer and monolithic everywhere else. The PropertyCircuit and the RelationalCircuit were what the v1 design would have looked like if the modularity had gone all the way down. Each circuit with its own vocabulary. Each circuit trained independently. Each circuit's gradients touching only the circuit's parameters.

The implications stacked up faster than I could write them down. The vocabulary growth problem dissolved, because individual circuits could have small vocabularies hand-curated to their capability. The gradient pollution problem dissolved, because each circuit's gradient flow stayed inside its own parameters. The regression risk dissolved, because adding a circuit didn't touch existing ones. The training cost dissolved, because 35 thousand parameters, yes 35 thousand trained in three seconds. The architecture had been the constraint all along, and the constraint had been the assumption that v2 needed one model doing everything.

I commit-and-merged the PropertyCircuit work. Wired it into dispatch behind a property_query intent. Started thinking about which capability gap to tackle next. There was a list of categories the frontier battery had surfaced that the brain layer couldn't handle. Each one looked like its own small circuit. The frontier was a roadmap.

The brain layer wasn't going away. It still did the compositional reasoning work that didn't fit into a single dedicated circuit. But it had stopped being the thing the architecture was organized around. It was becoming one of the experts in a small fleet, not the main pipeline that everything else had to negotiate with.

I'd been building v2 for months. The decision sitting in front of me now was whether v2 was the right name for what came next, or whether the modular fleet was different enough to be its own thing entirely. I knew which way I was going to lean. I wanted to see it run for another day or two first.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 21: Three Walls

Josh T — Mon, 27 Jul 2026 13:00:13 +0000

I spent a week trying to make the brain layer do more. Every time I fixed one thing, the next wall was already there. The architecture was the problem.

Part 20 ended with nine concepts revived and a structural realization sitting underneath the wins. The vocabulary growth approach was running out of room before Origin's vocabulary had reached anywhere close to conversational. The architecture I was patching had become the thing that needed redesigning. I went back to the brain layer to see if it would scale up while I figured out what to do about the encoder.

The first thing I noticed was that training was painfully slow. A full brain layer retrain on the rebalanced corpus was projected at ninety hours. For a thirty-nine-thousand-pair dataset that should have taken minutes. Something was wrong with the math, and the way to find out was to profile it.

I built a profiler that broke the per-batch time down by sub-operation. Forward pass, backward pass, each of the routed circuits, the thalamus router, the two-stage reasoner. The result was unambiguous and embarrassing. The two-stage reasoner, which I'd spent significant time restoring from v1 as the "intellectual core" of compositional reasoning, was consuming 99.8% of training wall-clock. Every other operation in the brain layer combined consumed about four milliseconds. The two-stage reasoner ate two thousand seven hundred and six.

I went looking for why. The two-stage reasoner runs property inference (Stage 1) followed by rule application (Stage 2). Stage 1 was supposed to identify which properties of the input concepts mattered. Stage 2 was supposed to combine those properties into outcomes. Stage 1's output was the input to Stage 2's attention. So Stage 2's compute scaled with how many properties Stage 1 surfaced. And Stage 1, on a real input, was surfacing eight thousand two hundred and twenty-five concepts as "inferred properties" out of eight thousand eight hundred and eighty-two. Ninety-three percent of the vocabulary, declared a property of every input, fed to a Stage 2 attention layer that had to process the noise.

The property circuit was degenerate. It had been trained against data that didn't teach property structure cleanly, and the output had collapsed to "everything is a property." Stage 2 wasn't reasoning about compositions. It was performing attention over near-uniform noise.

I ran an ablation. Replaced the two-stage forward with a negative-infinity tensor that would lose every argmax competition, effectively removing it from the prediction. The brain layer's held-out eval went from seventeen out of twenty-six to twenty out of twenty-six. The two-stage reasoner wasn't dead weight. It had been actively poisoning predictions.

I disabled it. Training dropped from two thousand seven hundred and six milliseconds per batch to ten and a half. The ninety-hour projected retrain became seventy-eight minutes. The held-out eval went up. The "intellectual core" I'd built was making everything worse.

That should have felt like a win. It mostly felt like a question. What else in this architecture was I keeping around because I'd designed it instead of because it worked.

I built a new battery to find out. The identity and working surface and compositional batteries were saturated. They tested capabilities Origin already had. I needed something that tested the ceiling, the questions Origin couldn't answer yet, the shapes the system would fail on. Twenty-five probes across five buckets, contrastive properties ("how is X different from Y"), taxonomic intersection ("what do X and Y have in common"), role requirements ("what does a teacher need"), boundary classification ("is fire alive"), and out-of-distribution honest refusals ("is the number seven happy today"). Each one isolated a specific brain capability so the failure mode would be diagnosable. I called it the frontier battery.

First run: nine out of twenty-five correct. Eight out of twenty-five confidently wrong.

The confident-wrong cases sorted into recognizable failure modes. The brain layer would fire action on "is the sun alive" and the dispatcher would compose "alive causes action." It would fire consent on "what do car and bicycle have in common" and produce "car causes consent." The definitional fallback path would parse "what is the population of mars" as a question about "mar" (the prefix of "march") and emit a definition for the month. "Is the number seven happy today" got a generic number definition because the dispatcher had no concept of "this question is structurally nonsensical."

I shipped five dispatch guards. Regex patterns that recognized population-of and number-X-is-Y as nonsense shapes and routed them to IDK. Text-shape gates that caught "in common" and "different from" before the brain layer could compose them. A specific guard for "is X alive" because the brain layer didn't have boundary classification training. The frontier battery confident-wrong rate dropped from eight to three.

Then someone independent ran a hundred conversational probes against production and surfaced eight more misroutes and misfires. I shipped five more dispatch guards. The misroute count went to zero. The probe count went to a hundred and forty-three. I caught myself writing my third regex of the day and remembered the rule we'd codified weeks earlier: hardcoded text patterns that tell Origin what to say bypass understanding. The fixes I was shipping were against my own discipline. Some of them were the right kind of pattern (text used to interpret what Origin already learned). Some of them weren't. The line between the two was getting thinner.

I needed an approach that didn't keep growing the regex pile. I tried a curriculum.

The idea was clean. Stage 0: train concept embeddings contrastively so similar concepts cluster. Stage 1: train property assignments so the model learns that fire is hot, ice is cold, dog has fur. Stage 2: train compositional rules so the model learns that fire causes burns and ice in heat causes melting. Three stages, each building on the previous, with explicit per-stage gates.

Stage 0 worked. Trained-category accuracy hit 99.8 percent across three random seeds. Embeddings were healthy. The contrastive mechanism converged cleanly in five seconds of GPU time. Held-out generalization was partial as expected, 65 percent on average, because Stage 0 only had supervision saying "these concepts go together" without any cross-cutting property structure. The plan was that Stage 1 would add the property structure and Stage 2 would add the compositional rules and held-out generalization would climb.

I built Stage 1. Hand-curated fifty-three property pairs that would have been the seed for cross-category transfer. Fire goes to hot. Ice goes to cold. Rock goes to heavy. Apple goes to red. The kind of grounded property assignments that should produce transfer to nearby concepts: train fire and sun as hot, and the model should infer that lava is also hot. That's the whole point of property transfer. Without it, the curriculum has to enumerate every concept-property pair individually, and we're back to scaling the vocabulary by hand again.

I ran Stage 1 with a refinement-from-v5 setup. Loaded the existing brain layer, froze most of the parameters, unfroze the prediction head and let the embeddings shift. The training-pair recovery hit ninety-two and a half percent. Forty-nine of the fifty-three pairs landed correctly. Brain eval held steady at twenty-one out of twenty-six. The recipe worked.

Then I ran the property transfer eval. Twenty-five held-out probes, each one testing whether a property the model had been taught on one concept transferred to a related concept. I trained rock as heavy. The test asked whether stone was heavy. I trained ice and snow as cold. The test asked whether arctic was cold. I trained sun as bright. The test asked whether a lamp was bright.

Top-1 accuracy: zero out of twenty-five. Top-5 accuracy: two out of twenty-five. Top-10 accuracy: two out of twenty-five.

Eight percent transfer on shapes the model should have generalized. Stone returned falls and sinks and lift, with heavy absent from the top five. Arctic returned dissolve and cloud and freeze, with cold absent. Lava returned rot and fry and evaporate. The model had memorized the specific input-vector-to-output-index mappings of the fifty-three training pairs. It had not learned the underlying property. Stone is embedding-close to rock, which the model had been told was heavy, and the dense prediction head simply did not move that information through.

The diagnosis took an hour. The brain layer combined an averaging pool of input concept embeddings with a dense prediction head. The averaging pool discarded positional structure. The dense head memorized specific input-vector targets. Property transfer required something the architecture didn't have: a way to fire a learned property whenever an input was semantically close to a trained example. The averaging-and-dense combination doesn't support that. It supports "memorize what you were shown."

This was the third wall in a week. The two-stage reasoner had been actively poisoning predictions. The dispatcher needed an unbounded supply of guards to handle real conversational text. The brain layer's property circuit was structurally incapable of property transfer. Each layer of the architecture I'd built and patched and restored was hitting a ceiling, and the ceilings weren't tunable. They were the architecture.

I'd spent two months bringing the brain layers back from missing-in-action to production-integrated. The brain layers were now production-integrated and architecturally limited. I had three options written in the session notes for what to do next. Change the architecture. Build something separate. Accept the ceiling.

I knew which one it was going to be, I didn't want to think about it. I'd already seen the RelationalCircuit pilot work earlier in the week. Maybe the path forward wasn't fixing the brain layer at all.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 20: Gradient Pollution

Josh T — Mon, 20 Jul 2026 13:00:27 +0000

Every topup run broke batteries somewhere. The fixes broke different things. The pattern wasn't sloppy training. It was three architectural problems stacked on top of each other.

After Part 19's measurement-discipline reset, I went back to vocabulary work. Origin needed more concepts in its encoder. There were thirty-two candidates I'd been wanting to add for weeks. Words like holy, prime, cash, mobile, aircraft. The kind of vocabulary a conversational system needs to handle normal English.

I'd built a tool called train_encoder_topup.py that was supposed to add new concepts without retraining from scratch. Five quality gates. Preflight checks. Atomic promote with backup. Resume on failure. The whole discipline scaffolding from a week ago, applied to vocabulary growth.

The smoke run worked. Two concepts (holy and prime) lifted to high firing, all five batteries held within tolerance, atomic promote, hippocampus rekey, ten-out-of-ten on the conversational probe. Two and a half minutes wall-clock. Clean.

Then I ran the full version. Fourteen concepts at the same settings.

It failed. Two of the fourteen never lifted past the firing threshold. The working surface battery dropped one probe. The full set didn't clear the gates. Production stayed at two new concepts instead of fourteen, the tool aborted cleanly without overwriting anything, and I sat looking at the log trying to understand why the smoke had worked and the scale-up hadn't.

The first guess was hyperparameter sensitivity. Smoke ran with conservative learning rate and modest oversample. Maybe fourteen concepts at the same settings was just too many for the gradient budget. I tried raising the learning rate. That lifted the cold concepts but broke the batteries in a different place. I tried more anchor data to dilute the topup pressure. The cold concepts went cold again. I tried trimming the question-form templates that seemed to be teaching the encoder bad habits. That fixed two compositional failures and broke two identity probes.

Each mitigation moved the breakage around. Nothing eliminated it.

I'd built a sandbox specifically to test three different augmentation approaches in parallel. Text augmentation. Feature-space mixup. Graph-driven sentence templating. All three worked at lifting cold concepts. All three broke batteries. Different things broke under different approaches, which meant the breakage wasn't tied to any one technique. The system had a structural property that the topup process was provoking.

I stopped tuning and started auditing.

The first audit went after the headline number that had been driving the vocabulary work. We'd been saying for weeks that the encoder was "sixty percent dead." Five thousand four hundred and ninety-three concepts of eight thousand eight hundred and eighty-two were silent on the per-concept firing report. Three out of every five output slots, dark.

I wrote a tool that joined the encoder health report against the brain layer's reasoning data to see which dead concepts the brain layer actually needed for compositional reasoning. The brain layer's training pairs gave a direct measurement, how many times did this concept appear as a reasoning output? If a concept was never an output, the brain layer never needed it. If it was an output thirty times, the brain layer needed it a lot.

Two concepts had high pressure. perceptual appeared as an output five hundred and twenty-five times. evaporate appeared thirty-five times. Thirty-four concepts had medium pressure. Five thousand four hundred and thirty concepts had zero pressure. The brain layer never needed them. The composer never used them. The encoder had never trained on them properly because they weren't real concepts to begin with. They were nominal slots in V2C that had survived the vocabulary growth process but didn't correspond to anything the system was supposed to know.

I marked them as unrescuable. UNRESCUABLE_CONCEPTS, a new frozenset alongside the existing lists of deprecated and intent-only concepts. The effective vocabulary dropped from eight thousand eight hundred and eighty-two to three thousand three hundred and eleven. The "sixty percent dead" number had been a measurement artifact of counting empty slots that nothing was supposed to occupy.

That reframed the problem. The encoder wasn't dying. It had been carrying five thousand four hundred and thirty dead slots that everything still flowed through.

That's when the actual mechanism showed up. The encoder's output head is structured by domain. Each domain has its own classifier head producing logits across the concepts in that domain. The domain called other, the catch-all bucket for things that didn't fit anywhere else, had five thousand four hundred and thirty-seven slots in it, of which only two hundred and seventy-one were real concepts. Both perceptual and evaporate lived in other.

When the topup tool ran a gradient update on either of those concepts, the gradient flowed through every output row in the other domain head. All five thousand four hundred and thirty-seven of them. The two real concepts I was trying to lift had to share gradient with five thousand four hundred and thirty dead slots that contributed pure noise.

I called it gradient pollution because I was tired and that was what it looked like. The training signal for the concepts I cared about was getting drowned in noise from concepts I didn't.

The fix had three pieces. First, mask the dead slots. The loss function gets zeroed for any output row in the unrescuable set, so dead-slot gradients can't flow at all. Second, freeze the trunk. The trunk has signal for nearly every concept already. The heads are what need to learn. Freezing the trunk eliminates trunk drift by construction and makes battery regressions only possible through head output behavior. Third, isolate by domain. The mask zeros loss for any output slot in a domain that doesn't contain at least one of the concepts being lifted, so cross-domain head cross-talk gets killed too.

I stacked the three mitigations and re-ran the perceptual + evaporate topup.

Two firings above threshold. Identity battery forty-eight out of fifty-one. Working surface twenty-six out of twenty-six. Compositional reasoning twenty-nine out of twenty-nine.

The compositional battery had been at twenty-six out of twenty-nine before the topup. Lifting two concepts cleanly with all the gradient mitigations in place had fixed three compositional probes that had been failing for weeks. The brain layer needed those concepts to compose correctly, and once it had them, it composed correctly. The other batteries didn't budge. First all-gates-pass topup of the project.

I promoted it. Then I went looking for the next concept to lift, and the next architectural problem surfaced almost immediately.

I added trader to a different domain head. The topup gates passed. But on the way through the post-promote audit, I noticed evaporate's joint output logit had dropped sixty-eight percent on physics queries. evaporate wasn't in the same domain as trader. The per-domain row for evaporate hadn't moved. But the joint logit (the product of the per-domain classifier and a top-level domain selector) had collapsed because training trader had shifted the top-level domain selector's distribution.

This was a different architectural side-channel. The encoder's joint output combined per-domain concept logits with a log-softmax over which domain the input belonged to. Training a concept in domain eleven told the model "this input is in domain eleven," which proportionally reduced the model's prior on every other domain, which dragged down every concept in those domains by a constant offset. Hidden multiplicative coupling between concepts that had nothing to do with each other.

I added a fourth mitigation, surgical freeze on the domain classifier during topup. The classifier learns nothing during a concept lift, so it can't drag other domains down. I ran the same gates on five more concepts (chemical, thinker, ikon, rot, and a re-lift of trader). All five passed.

Then I tried the recipe on a sixth concept in a denser domain head, and hit the third structural limit. The within-domain heads use a shared bottleneck projection. When you lift one concept in a domain with thirty other live concepts, the shared projection learns the new concept's features at the expense of the existing concepts' features. The mask-and-freeze recipe couldn't isolate within a single head; the head had only one set of parameters and they all moved together.

The recipe had a ceiling.

Three architectural layers diagnosed in two days. Each fix exposed the next problem. We were tunneling down through layers of the encoder's design, and each layer was more fundamental than the last. The first one was a counting error masquerading as a capacity problem. The second one was a gradient routing problem inside a single head. The third one was a head-architecture problem that no amount of routing fix could touch. The wall was the architecture itself.

Nine concepts revived end-to-end. Compositional battery perfect. Identity off by two from baseline. Working surface unchanged. Real progress, real wins, real numbers. And the realization, sitting underneath the wins, that this approach to vocabulary growth was going to run out before Origin's vocabulary was anywhere close to where conversational use required it.

The architecture I was patching had become the thing that needed redesigning.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 19: The Number Was Wrong

Josh T — Mon, 13 Jul 2026 13:00:45 +0000

The brain layer was scoring high because the test was leaking. The actual capability was being silently rejected by a misconfigured gate. Both findings landed in the same week.

Part 18 ended on a clean diagnosis. The brain layer reasoned correctly when the encoder fed it correct inputs. The encoder didn't always feed it correct inputs. So the path forward was upstream: more physics-shaped training data for the encoder, retrain, re-validate. I wrote the drops, kicked off the retrain, and watched the held-out eval climb.

It hit twenty-three out of twenty-six. Eighty-eight percent. The number I'd been chasing.

I sat with that for an evening. Twenty-three of twenty-six on compositional reasoning probes the model had never seen during training. The Phase 8 cutover gate from Stage D had been sixty percent. I was thirty points past it. The brain layer had not only survived its missing-from-production months, it had come back stronger.

The number was wrong.

I figured this out the next morning while writing what was going to be the celebration commit. Something nagged about the eval set. The training data generator built the eval pairs independently from the training pairs, drawn from a different source list. That should have given me a clean train/test split. But I noticed the eval generator was running before the training generator wrote its file, and neither side knew about the other. I dropped into a Python shell and intersected the two pair sets by their input-output keys.

Twenty-three of twenty-six held-out probes were also present in training data. Eighty-eight percent of my held-out eval wasn't held out. The model wasn't generalizing. It was memorizing the answers it had already been shown, then being graded on whether it remembered them.

The three pairs that were genuinely unseen, I checked those separately. The model got one right. Three out of twelve when I went back through other historical evals and ran the same overlap check. About a quarter, with no statistical confidence behind it. The actual generalization signal was much weaker than the celebration number had suggested.

I rebuilt the data generators to subtract eval pairs from training before writing, added a verification tool that refused to launch any training run with non-zero overlap, and re-ran the overnight retrain. The honest number came back at fifteen out of twenty-six. Fifty-eight percent. Still real, about sixteen times the score a constant-prediction baseline would have produced, but not eighty-eight, and not over the Phase 8 gate.

Then I went looking at the rest of the pipeline for the same shape of problem. The pattern wasn't bug, exactly. It was discipline. The file that held the brain layer's trained weights was called brain_layers_v2_sandbox.pt. The "sandbox" label gave everyone implicit permission to handle it loosely. Me. The data-build pipeline. The scripts I wrote. The model I was talking to at three in the morning. Sloppy eval setup. Shared paths between fast-mode and full-mode runs. No backup before overwrites. No preflight checks.

The file was the production checkpoint. I was hitting it every time I talked to Origin. The label said sandbox, the behavior was production, and the discipline gap the label permitted was exactly why nothing had been moving.

I renamed it. brain_layers_v2.pt. The script that wrote it got renamed too. sandbox/sandbox_train_brain_layers_v2.py became tools/train_brain_layers_v2.py, because every "sandbox" run was actually a production run and we'd been letting the name lie to us. I added an overwrite guard: if a new training run scored lower than the existing production checkpoint, the script refuses to write unless explicitly forced. I added a fast-mode flag with hard caps so quick experiments couldn't accidentally clobber production. I built a preflight tool that ran before every training session and refused to proceed unless the data, the eval set, and the configuration all passed gates.

Then I ran the audit again. Now with the discipline scaffolding underneath.

The next finding was worse.

Every battery, every sandbox tool, every audit script I'd been running for the past several weeks called a function called load_arch_encoder. The function had a default preference list that picked which encoder to load when no path was specified. The first entry in that list was v2_encoder_arch.pt, a three-thousand-seven-hundred-and-thirty-five-concept encoder from May 9th. The actual production chat path bypassed the function entirely and hardcoded a different file: v2_encoder_multilabel_v3.pt, eight thousand eight hundred and eighty-two concepts, May 16th.

Every audit I'd run had been silently testing against the smaller, older encoder. The one production wasn't using.

That explained things I hadn't been able to explain. balloon at index 4577 looked encoder-dead because it was past the smaller encoder's output dimension. fall, evaporate, half of the physics vocabulary the brain layer was supposed to compose with, all of them invisible to the audit because the audit was looking at the wrong tensor. My twenty-nine-probe compositional reasoning battery had been reading 55%, and when I forced it to use the actual production encoder, it read 69%. The capability the audit said wasn't there was there. The audit was wrong.

One-line fix. Added the right encoder as the first preference, kept the older one as a fallback for the v1-era training scripts that still referenced it by name. Re-ran every battery against the correct production tensor. The numbers moved up.

And then the dispatch gate.

The brain layer had been wired into Origin's conversation pipeline behind a feature flag since Stage D. The wiring included a verification check: before accepting the brain layer's output as a real answer, confirm the answer concept also appears somewhere in the encoder's top-five fired concepts. The idea was reasonable. If the encoder didn't fire the answer at all, maybe the brain layer was firing it wrong. The check was inherited from a different part of the dispatcher designed for dense-firing encoders.

Origin's encoder is sparse-firing. Most natural-language queries fire one concept above the 0.5 threshold, the subject noun. "What happens to ice in heat?" fires ice at 0.93 and nothing else over 0.5. The brain layer takes ice, composes through its trained reasoning, and produces melting at confidence 1.0. The dispatch gate then checks: is melting in the encoder's top five? No, because the encoder only fired one concept above floor. Gate rejects the brain layer's answer. Pipeline falls through to "I don't know."

This had been happening on every natural-language compositional question. Every one. The brain layer had been producing correct answers: burns for wood in fire, floats for wood in water, sinks for rocks in water, melting for ice in heat. The gate had been silently throwing them away because the gate's verification rule was incompatible with how the encoder actually fired. The capability had been there since the day I committed Stage D. It just couldn't reach the output.

I fixed the gate. Two lines. If the encoder fires fewer than three concepts above 0.5 and the brain layer's top-1 has confidence above 0.7, accept the brain layer's output as long as the answer concept isn't the same as the input concept. The whole pipeline took maybe fifteen minutes to write and test.

I ran the production probe set. Thirty-two questions, mixed difficulty, the same shapes that had been refusing for weeks.

Six pass became twenty-one pass. Two-thirds of the questions Origin had been refusing now answered correctly. "What happens to a rock in water?" became "rock causes sinks." "What happens to ice in heat?" became "ice causes melting." "What happens to wood in fire?" became "wood causes burns." "Does wood float in water?" became "water causes floats." "What happens when a ball hits the ground?" became "ball causes bounce." The compositional reasoning the brain layer had been trained for, that I'd been measuring at twenty-seven percent through the wrong audit on the wrong encoder behind a wrong gate, had been functioning at sixty-six percent the whole time. Hidden by infrastructure.

The identity battery held at fifty out of fifty-one. The working surface battery held at twenty-six out of twenty-six. No regressions. The fixes had restored capability without trading anything for it.

I should have felt good. I felt unsettled. The brain layer had been quietly working for weeks while every measurement said it wasn't. The eval set had been leaking for weeks while every training run said it was generalizing. The wrong encoder had been answering every audit question while production used a different one. Three separate measurement failures, all pointed in the same direction, all making the system look worse than it was, all hiding the work that had already landed.

I'd been chasing problems that weren't there because the instruments were lying. Some of the chases produced real fixes. Some of them produced the discipline scaffolding I should have built months earlier. But the time spent was time spent. And the unsettling part wasn't the fixes; it was the silence. Three big infrastructure bugs, none of them surfaced by tests, none of them visible until somebody sat down to question the numbers directly.

What else was lying. I started writing a longer audit.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 18: The Layer That Wasn't There

Josh T — Mon, 06 Jul 2026 13:00:35 +0000

The architecture I'd designed for v1 wasn't in v2. It hadn't been for months. Nobody had moved it across.

Part 17 closed on an unfinished thought. The audit had surfaced the dispatcher ceiling and Stage C had cleared it. Forty-nine percent to eighty percent on the identity battery, no patterns added. But somewhere in the per-concept firing report I'd started wondering where Origin's actual reasoning was happening. Not the encoder firing concepts. The part that was supposed to take those concepts and compose them into outcomes.

The v1 design had names for that part. A thalamus router that decided which kind of reasoning a fired concept set called for. Physical chains went to a physics circuit, emotional content went to an amygdala-biased pathway. Micro-circuits inside each pathway took fired concepts as input and produced new concepts as output: ("rock", "water") goes in, ("sinks", "heavy") comes out. A two-stage reasoner that ran property inference first ("a rock is dense") and rule application second ("dense things in water sink"). The whole stack was the intellectual core of what made OLT-1 different from a transformer playing word association. Concepts as the unit of thought. Composition as the mechanism. Traceable reasoning at every step.

I went looking for it in the v2 code.

The file was still there. concept_engine/core/reasoning.py. The classes were still there. MicroCircuit, ThalamusRouter, TwoStageReasoner. The orchestrator was still there in engine.py, six hundred lines of careful Python that wired the encoder, the brain layers, and the decoder into a single pipeline.

The last modification on either file was a v1-era commit hash. ce569b0. 3b8e460. Months old.

I ran grep across the v2 production code. v2_decoder.py. v2_arch_encoder.py. The conversational composer. The discovery v2 system. The evolution v2 orchestrator. Zero imports of the brain layers. Zero references to ConceptEngine. The only places those classes were used at all were sandbox experiments and v1-era tools that hadn't been touched since the v2 cutover.

Somewhere during the v1-to-v2 refactor, the reasoning layer had been quietly dropped. The encoder kept getting upgraded. The dispatcher kept getting elaborated. The composer kept getting features added. But the thing between the encoder and the dispatcher, the part that was supposed to take "rock + water" and produce "sinks" before the dispatcher ever had to make a routing decision, had been left behind. v2 was running encoder, then flat pattern lookup, then text. No composition. No chained reasoning. Nothing that justified the project's working description: "OLT-1 understands language."

My thoughts were unprintable in its first six words. I think you get the idea.

The right move wasn't to slam the brain layers back into production. The right move was a plan. The brain layer code was designed for v1's three hundred and twenty-seven-concept vocabulary; v2 had eight thousand eight hundred and eighty-two. The v1 trained weights couldn't be loaded directly. The training data, the rules that taught the micro-circuits what compositions to make, was almost all v1 concepts, and only about a hundred and one of those overlapped with v2's vocabulary. The brain layer architecture would survive the move. The brain layer knowledge would not. It needed to be retrained from scratch against the v2 world.

I wrote Stage D as eight phases. Verify the v1 layers still ran in isolation. Verify v2's emotion concepts were present in the new vocabulary so the amygdala router had something to fire on. Allocate the layers with v2 dimensions and confirm shapes. Generate a new training set against v2's concept world. Train the layers. Build an adapter that exposed reasoning to the v2 dispatcher. Wire the adapter in behind a feature flag so production behavior stayed byte-identical when the flag was off. Run the validation suite. Decide on cutover only if three explicit quality gates passed.

I ran the plan as an overnight session, I let it cook low and slow.

The verification phases went clean. The brain layer code loaded. The v2 emotion concepts were all present. The shapes worked. The training data generator emitted six hundred and fifteen reasoning pairs across the v2 vocabulary, property inferences ("rock is dense") and rule applications ("dense and water → sinks") drawn from a mix of v1 chains, hand-curated additions, and Origin's existing discovery pipeline. Class balance held. The top output category took 5.5%, well under the 25% concentration that would have meant the data was tilted.

Training ran clean too. Six configurations, two training modes, three random seeds, over an hour and a half on the GPU. The best checkpoint scored twenty out of twenty-six on a held-out reasoning probe set. Seventy-six percent. Brain layer composing concepts correctly on inputs it had never seen.

That was the moment in the audit where things were supposed to start clicking. The brain layer was back. The reasoning architecture was alive again. All that was left was to plug it into the conversation pipeline and watch the compositional reasoning emerge.

I built the adapter, wired it into the dispatcher behind a feature flag, ran the identity battery and the working surface battery with the flag off to make sure I hadn't broken anything that already worked, and confirmed both still matched the post-Stage-C numbers. Then I built a third battery. Twenty-nine compositional reasoning probes that the system had never been asked before. "What happens if I drop a rock in water?" "What happens to ice in heat?" "Does wood float?" The kind of questions that require the brain layer to actually compose. Ran it with the flag off to establish a baseline. The dispatcher's existing pattern lookup got four out of twenty-nine. Thirteen point eight percent.

Then I ran it with the flag on, brain layer engaged.

Eight out of twenty-nine. Twenty-seven point six percent.

I doubled the score. That's the good news. The plan had set a Phase 8 cutover gate at sixty percent. I'd hit twenty-seven.

I went back through the trace. The brain layer was firing correct compositions whenever the encoder gave it correct inputs. "What happens to ice in heat?" The encoder fired ice at 1.0 and heat at 0.001, and the brain layer still figured out melting because ice was strong enough on its own. "What happens to wood in fire?" The encoder fired fire at 0.93 and wood at 0.026, and the brain layer got burns. Those were the wins.

The losses had a different shape. "What happens to wax in heat?" The encoder fired alive at 0.64, wax at 0.57, heal at 0.46. Heat wasn't anywhere in the top eight. The brain layer doesn't know wax melts because the encoder didn't tell it that heat was in the sentence. "What happens to a balloon filled with helium?" The encoder fired question at 0.99 and alive at 0.98 and basically nothing else relevant. Neither balloon nor helium showed up. The brain layer produced a confident-sounding default and was completely wrong.

I ran the brain layer's training-distribution eval separately. Seventy-six percent. The reasoning was working. What wasn't working was the substrate underneath it. The encoder fires fine on direct concept queries: name something, the encoder produces the concept. The encoder doesn't fire well on compositional queries. Wrap that concept in "what happens to X in Y?" and a chunk of the concept identity dissolves into the surrounding syntax.

That's the part Stage C hadn't fixed. Stage C had been targeted at conversational surfaces: preference, identity, register. It hadn't been told to handle "what happens to" sentences. It hadn't been given physics chain examples in natural-language wrappers. So it didn't fire on them. And the brain layer, sitting one layer up, was watching the wrong inputs come through and reasoning correctly about nothing.

The Phase 8 cutover decision per the plan: don't cut over. Three gates required, two passed, one missed by half. The brain-layer code stayed in the codebase, fully integrated, fully flag-gated, default off. Production behavior unchanged.

What I'd learned wasn't that the brain layer didn't work. It was that "the substrate," meaning whatever the layer above the reasoning gets fed, was now the recurring antagonist of every story I was telling about this project. Part 16: substrate too thin, LLM extraction unusable. Part 17: substrate too narrow, fix it with curated drops. Part 18: substrate misfiring on a different shape of question entirely. The composer needed grounds. The dispatcher needed clean signals. The brain layer needed the right concepts to compose. All three were one layer away from a working pipeline, and the missing piece was always upstream of where the work had been.

The encoder needed another retrain. A physics-shaped one this time. I started writing the data drops while the Phase 8 result was still on the screen.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 17: Fifteen of Eighteen

Josh T — Mon, 29 Jun 2026 13:00:34 +0000

The dispatcher had a ceiling. The encoder had to be taught.

The rollback from Part 16 was still draining out of the reasoning bank when I kicked off the audit. I wanted a measurement of how badly the missing substrate was actually hurting the system - not "vibes from a few conversations," but a real count. Fifty-one probes through the identity battery. Twenty-six through working surface. Per-concept firing checks across all eight thousand eight hundred and eighty-two slots. Where things broke, write down what fired and where it landed.

That audit answered the substrate question. The substrate was hurting. The composer reached for retrieval grounds and there weren't any, so it improvised, and the improvisation was sometimes embarrassing. That was expected. What wasn't expected was that the same audit surfaced a second problem, sitting one layer up. The dispatcher had developed a ceiling.

Identity battery sat at twenty-five out of fifty-one. Forty-nine percent. The dispatcher had been calibrated, tuned, threshold-swept, and run through three different routing strategies (A2 cluster routing, A4 library similarity, an A2/A4 hybrid called a2lib). All three landed within two probes of each other. The Pareto frontier - improve identity, lose working surface; improve working surface, lose identity - was real. The dispatcher could trade between the two, but it couldn't beat either.

I'd seen this shape before, in retrospect. When tuning stops moving the number, the number isn't a tuning problem. So before another round of threshold-sweeping, I wrote a small tool to classify every failure by what the encoder had actually fired on that input and what the dispatcher had then done with it. Eighteen failures. Each one got a trace.

The categories sorted out cleanly. Six failures came from the encoder over-firing the concept question as top-1 and crowding out the identity signal - "Quick question - who made you?" would fire question at 0.89 and leave i_am at 0.12, so the dispatcher routed to the question cluster and missed the identity claim that was actually being made. Six failures came from preference queries deflecting to the identity cluster: "are you sad?" would fire sad at 1.0 and i_am at 0.08, so the dispatcher saw i_am, picked identity, and answered "I am Origin" to a feelings question. Four failures came from domain-concept dominance on preference queries - "do you like dogs?" would fire dog at 1.0 and prefer at 0.41, the preference cluster didn't claim it, fallback fired the composer, and the composer produced "dog has fur, dogs are mammals." Three failures came from within-cluster discrimination - "are you human?" routed to identity correctly, but the response selector returned "I am Origin" instead of "no, I am not human."

Fifteen of eighteen failures traced to encoder behavior. Three traced to dispatcher response selection. Zero traced to dispatcher routing logic.

That's the structural finding the threshold sweep had been hinting at without saying. The dispatcher couldn't tune past the ceiling because the ceiling wasn't in the dispatcher. The encoder was firing the wrong concepts on the right questions, and there is no amount of threshold engineering downstream of "wrong concepts fired" that will produce "right answer chosen." The dispatcher had been doing the best it could with the signal it was being handed. The signal was the problem.

I ran one more dispatcher experiment anyway, partly to confirm the diagnosis. The a2lib hybrid added cluster-internal library similarity at a 0.95 threshold - when the cluster routing succeeded but the response selector was ambiguous, fall back to nearest-neighbor against a small bank of canonical examples. It tied A2 at thirty-three out of fifty-one. Library matching helped three cases where the dispatcher had previously picked wrong; it hurt three other cases where the library matched the wrong template. Net zero. Different shape, same ceiling.

I shipped one small dispatcher win because the data justified it. The concept prefer had been firing at 0.16 to 0.41 on preference queries, and the preference cluster wasn't reading it. A two-line cluster-membership change wired prefer and want into the cluster. Identity went to thirty-three; preference sub scores went from 50% to 60%. Real lift, no patterns, mechanism justified by the audit. But that was the boundary. Past that, the dispatcher had nothing left to give.

So I built Stage C. Three curated training-data drops, each one targeting a specific failure category the audit had named.

Drop A+D handled the preference-deflection problem. A hundred and forty-two pairs in the shape "do you like X?" and "what is your favorite X?" and subjective-state queries, labeled with the concept set [dont_know, prefer, self] instead of whatever domain concept the question mentioned. The encoder had been seeing "do you like dogs?" and learning that the strong signal was dog. Drop A+D added a stronger signal: questions in this shape carry dont_know and prefer, not dog.

Drop C handled within-cluster discrimination. Forty-five pairs in the shape "are you human?" / "are you an AI?" / "are you a robot?" labeled with the existing concepts agree and refuse alongside identity and i_am. Origin already had agreement and refusal concepts; the encoder just hadn't been taught when to apply them in identity-class questions. Drop C wired the connection.

Drop E handled prefix collapse. A hundred prefix-prepended versions of canonical identity probes - "Hello user. what is your name?" / "Sorry to ask, but what is your name?" - labeled with the same identity concepts as the bare versions. The encoder had been getting the bare versions right and the prefixed versions wrong; the difference was character-level distribution, not semantic content. Drop E gave the encoder the prefix variations explicitly.

None of these were templates. They were training pairs. The dispatcher wasn't being told to look for "are you human?" - the encoder was being trained on what concept set fires when a sentence in that shape appears, the same way it had been trained on every other concept it knew. The pipeline downstream was unchanged. The work was upstream.

Stage C retrained the encoder over sixty epochs, warm-started from the multi-label checkpoint from the morning. Three hours, monotonic descent, final loss 8% below baseline. Then I ran the batteries.

Identity went from thirty-three to forty-one. Forty-one out of fifty-one. Eighty percent. Up thirty-one points from the production baseline two stages ago, no patterns added anywhere in the dispatcher. Canonical probes 80%, prefix probes 83%, preference probes 80%. Each drop delivered exactly the lift the audit had predicted: preference up twenty points, prefix up sixteen, canonical up twelve. Working surface stopped regressing - it actually crept up to 81%. Per-concept firing across the full vocabulary improved across every bucket.

The lesson is the same one Part 14 was circling and Part 15 had to learn the hard way. When the work isn't moving the number, the work is in the wrong place. The dispatcher had been the obvious place to look because the dispatcher was what we'd built. The encoder was the place we hadn't been looking because we'd trained it months ago and moved on. Failure-mode tracing is the move that surfaces which layer is actually responsible. It's not glamorous. It produces a table of categories and counts. Then the categories tell you what data to write.

Ten failures remained. Five within-cluster - the encoder now fires refuse correctly on "are you human?" but the response selector hasn't been trained to pick the refusal-shaped response when refuse fires. Four preference-deflection holdouts where the domain concept fires at 1.0 and overpowers prefer. One probe ("what is your purpose?") where the test wants a non-IDK answer and Origin honestly doesn't have a self-modeled purpose - the test is wrong, not the model. All of these are encoder or response-selection refinements, not dispatcher rewrites.

The audit had also surfaced something else. While I was reading the per-concept firing report, looking at which concepts were healthy and which were silent, I started wondering where Origin's actual reasoning was happening. Not the encoder firing concepts - the part that's supposed to compose them into outcomes. The thalamus router. The micro-circuits. The two-stage reasoner. The architectural pieces from the v1 design that were supposed to be the intellectual core of what made OLT-1 different.

I went looking for them in the v2 code.

They weren't there.

That's Part 18.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

Origin is developed at Fallen Angel Systems with the Genesis framework - NVIDIA Inception member. (USPTO Application #64/016,973, #64/017,567). FAS Guardian defends production AI systems from prompt injection in under 3ms. FAS Judgement is the open-source attack console that finds the gaps. Defense. Offense. Creation.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 16: Forty Percent Wrong

Josh T — Mon, 22 Jun 2026 13:00:47 +0000

Fourteen thousand triples. Forty percent of them wrong. Twelve hours of compute on a substrate that turned out to be unusable.

The plan after Part 14 was simple to describe: build a relational substrate so the composer would have real (X, R, Y) facts to retrieve at inference time. Sandbox-test day had said retrieval grounds and generation invents. The whole next stretch of work was about giving the retrieval path something real to retrieve.

ConceptNet was the obvious starting point. It's free, it's structured, it has millions of triples. I pulled it in and looked at coverage. Biology, rich. Physics, rich. Math, almost nothing. Every math concept in our vocabulary had at most one or two triples, all of them shallow is_a chains. The substrate gap was real, and it was domain-specific.

Wikidata helped, but not as much as I'd hoped. Most of its dense content is biographical and geographical - useful for is_a facts about people and places, less useful for the cause-and-effect relations that ground reasoning about physical and biological processes.

Hand-curating was always going to be the gold standard, but slow. I had a hundred and fifty hand-written cause-effect pairs from sandbox-test day. To reach the coverage we needed, I'd be writing for months.

So I tried the obvious thing. Get an LLM to extract triples from textbooks.

The model was Gemma-2-9B running through LM Studio on my computer. The input was the core tier of our book collection - a hundred and nine textbooks spanning biology, physics, social studies, math. The output, after eighteen hours of extraction spread across two days, one of which Windows interrupted with an unscheduled reboot for an update mid-run, fourteen thousand seven hundred and seventy-five triples.

The extraction pipeline was reasonable. For each passage, ask Gemma to identify (subject, relation, object) triples where the relation was one of is_a, has, does, or causes. Constrain output to JSON. Filter triples where subject and object weren't both in our vocabulary. Write everything incrementally so a Windows reboot couldn't cost more than a few minutes of work. That last detail saved the run when Windows did, in fact, reboot.

The polysemy gate ran on the output as a dry run first. Six hundred and twenty-one triples out of fourteen thousand made it through. Four point two percent. That sounded like an aggressive filter doing its job. I committed them to the reasoning bank as a live ingest, tagged with a source label so I could roll them back if needed.

Then I pulled fifty triples at random and read them.

Seventeen were good. Thirteen were weak - technically defensible but not useful for grounding. Twenty were wrong in ways that would actively damage the system.

The wrong ones came in two flavors.

The first was direction errors on is_a. "Animal is a vertebrate." "Cartoon is a parody." "Diagram is a tree." "Sine is a cosine." In each case, Gemma had seen two concepts in close proximity in the source text and emitted an is_a triple, but in the wrong direction. A diagram is sometimes drawn as a tree; a tree is not a kind of diagram. Sine and cosine share a category; one is not a kind of the other. The model picked a direction and committed to it, and the direction it picked was wrong about half the time.

The second was object errors. "Compare can sonnet." "Divide can put." "Wartime is part of a vancouver." "Tax is a whiskey." "Aircraft requires winnipeg." Gemma had reached into the surrounding sentence and grabbed a word that happened to be there, and emitted it as the object of the relation. The result is grammatical English that means nothing. Aircraft requires Winnipeg is what it sounds like when a language model is pattern-matching the shape of a triple without checking whether the assertion is true.

I ran the rollback. DELETE WHERE source = 'llm_relation_extraction_v1'. Verified that the reasoning bank dropped from 51,872 rows back to 51,251. The bank was clean again.

I sat with the math for a few minutes. Forty percent of the triples that had passed our polysemy gate were wrong. The gate was filtering for a different axis entirely - checking whether subject or object had multiple senses that would conflate under retrieval. It wasn't checking whether the assertion itself was true. There was no check for that. We hadn't built one because we hadn't expected the extractor to be wrong that often.

The failure modes are semantic, not syntactic. That sentence is the one I kept coming back to. Templated post-filtering - does this look like a triple, are both ends in the vocabulary, is the relation one we recognize - can catch all the obvious junk. What it can't catch is a grammatical, well-formed assertion that happens to be wrong about the world. "Sine is a cosine" passes every cheap check. So does "Tax is a whiskey." The error is in the meaning, and the validator has no way to see meaning.

The natural next thought is a verifier pass. Run a second LLM call: "is this assertion true?" Untested, but it has its own problem. Wrong-direction is_a errors look plausible to a verifier model the same way they looked plausible to the extractor. The verifier would have to be aware of asymmetric is_a in a way the extractor wasn't. That's not an obviously cheaper problem than building a clean substrate by hand.

What I did salvage was the pipeline itself. The extractor's --resume flag and incremental-write logic are keepers. They survived Windows's unscheduled reboot without losing a sentence of work. When we eventually run a different extractor - a more constrained one, or a hand-verified one, or a hybrid - the plumbing is there. The model is what failed.

There's something underneath the failure that's worth saying out loud. The case for hand-curated relational substrate just got stronger. The case for trusting an off-the-shelf LLM to produce structured truth from free text just got weaker. The same machine-learning move that makes these models good at producing grammatical English also makes them comfortable producing confident assertions about things they haven't reasoned about. The pattern-matching is the whole story.

The substrate problem is harder than I thought going in. ConceptNet has gaps. Wikidata has limited cause-effect coverage. Hand-curating is slow. LLM extraction has a quality floor. There isn't a free option here. There's a slow option with a clean output, and a fast option with a contaminated output, and a few hybrid paths somewhere in between.

I picked the slow option, because the contamination is worse than the slowness. The retrieval bank has to be trustworthy for the composer to ground on. A bank with forty percent garbage in it isn't a bank. It's a noise source the composer would faithfully serve back to users as facts.

The same evening, while the rollback was running, I kicked off a hundred-turn conversation audit against the live system to see how badly the missing substrate was hurting it. The audit answered that question, and then it answered another one I hadn't asked. The dispatcher had developed a problem of its own.

That's Part 17.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 15: The Wall Behind the Vocabulary

Josh T — Mon, 15 Jun 2026 13:00:33 +0000

The gate said pass. The system got worse anyway.

Part 14 ended with a build order: gate first, then substrate, then composer. I'd already started picking out which polysemous concepts in the bank needed splitting. Then I ran the trace one more time before kicking the work off, and the trace said something I wasn't expecting.

The polysemy exposures from the last several days of audit traffic weren't coming from concepts already in the vocabulary. They were coming from concepts not in it. Eighteen distinct subjects had been pulled in from external sources during conversation - words like "happiness," "feelings," "brain," "dream," "anger." None of them existed as concepts in Origin's vocabulary. The polysemy gate would have nothing to act on, because the words triggering the leak weren't there to be gated.

So I pivoted. Build the vocabulary out first. Add the missing everyday words. Then the gate has a population to enforce against.

I had a tool for this. The per-slot integrator. We'd been using it for Discovery-style growth all along - when a new word showed up that was close to an existing concept (kitten near cat, pudding near pie), the integrator would carve out a slot for it in the encoder, train it on its handful of positive examples, and run a gate to make sure the new slot didn't hurt anything that was already working. It had a clean track record. Every kitten, every pudding, every kitty had landed without regression.

The plan was to do the same thing with five everyday concepts: wet, brain, dream, anger, happiness. Each had at least a hundred natural positive examples already sitting in the corpus from previous training runs. They'd been collected by other processes. We just had to admit them.

I ran the batch. All five failed the gate.

Recall sat around fifty percent - the new slots could only correctly identify about half of their own positive examples. Each integration cost about fifteen existing concepts that started failing where they used to work. All five routed to the same internal domain bucket -"other" - which was already saturated with the dumping-ground concepts that hadn't fit cleanly anywhere else. The integrator rolled them all back. Production state unchanged. Five attempts, five failures, all the same shape.

I took a few days off.

There's a version of project work where you don't take days off, where you push through, and that version is wrong. A uniform failure pattern across five attempts isn't a tuning problem; it's a structural one. Going back to the integrator to twist knobs on five more candidates was going to produce the same five failures with different names attached. The right move was to stop, and let the question reform.

The question that reformed was about domain assignment. The integrator had routed all five concepts to "other" because that was the domain its assignment logic had picked. But brain isn't an "other" concept; brain is biology. Anger isn't "other"; it's emotion. The taxonomy had categories that fit, but the assignment logic wasn't reaching them. So a few days later I came back and fixed that. Now brain went to biology. Anger went to emotion.

I ran the batch again. This time, two passed.

Brain hit 62% recall against its biology domain, 23% false-positive rate, only one regression. Anger hit 56% recall against emotion, 22% false positives, zero regressions. The per-concept gates were green. Better than the first attempt by every metric I'd been measuring.

Something felt off.

I'd written a note to myself months ago that said when something feels off, investigate. It's been right repeatedly. So before merging, I ran a sweep the gate didn't run. Take five thousand random sentences from the corpus. Run them through the encoder with the new brain and anger slots loaded. Watch what fires top-1 on each one.

Brain fired top-1 on 15.6% of the sentences. That's one in six.

Anger fired top-1 on 7.4%.

Sample misfires: a sentence about a blood-soaked lash fired brain at 0.99 confidence. A sentence about a man's speed fired brain top-1 because the word "head" was in it. Anger fired top-1 on sentences that had nothing to do with anger at all. The per-concept gate had checked recall on a hundred and twenty random negatives and called it clean. The actual encoder, looking at heterogeneous English text, was firing the new slots all over the place on inputs that shouldn't have triggered them.

Both rolled back. Encoder reverted. Production state unchanged again.

This was the architectural finding the first batch had hinted at and the second batch confirmed. The per-slot integrator has a cross-domain ceiling its own gates can't see. The gates sample a hundred and twenty random negatives, which is enough to catch the obvious kinds of false positives, but nowhere near enough to catch a slot that's quietly firing on one input in six. The integration looks clean per-concept. The integration breaks the encoder globally.

The reason is structural. The integrator carves out a new slot by training it locally -show it positive examples of the new concept, show it a small bag of negatives drawn from random other text, train until the slot lights up on positives and stays quiet on the negatives. The phrase doing the work in that sentence is "random negatives." A hundred and twenty random sentences contain a slice of English content, but they don't contain the specific weird false-fire patterns the new slot will discover. The slot then ships, and it discovers them in production.

What works for narrow same-domain growth is exactly that - narrow same-domain. When the new concept lives next to existing concepts in feature space, the slot inherits the discrimination the existing slots have already learned, and a hundred and twenty negatives are enough to catch any remaining drift. When the new concept lives somewhere semantically isolated, the slot has to invent its own discrimination from scratch on a tiny budget, and it gets it wrong. The negatives are no longer enough.

The path forward for cross-domain vocabulary isn't per-slot integration. It's joint retraining. Put the new concept in alongside everything else, train the whole encoder against it, let the system figure out where the new slot fits relative to the existing ones. That's expensive. It's also the only way to add the kind of everyday vocabulary Origin actually needs.

The past failures pinned the timeline down. The next time vocabulary expansion happens, it's a joint retrain. Not eventually. Next.

The polysemy gate is still on the queue. So is the substrate. The order Part 14 prescribed - gate, substrate, composer - is still right. What changed is the prerequisite. The vocabulary the gate will protect needs its own dedicated work, and that work comes ahead of the gate, not behind it.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 14: The Reframe

Josh T — Mon, 08 Jun 2026 13:00:10 +0000

Part 12 ended with a hypothesis. Two days later, the hypothesis met data.

The closing line of Part 12 was a guess. Maybe the next bottleneck wasn't more concepts, but the relationships between them. A model can know "dog" and "animal" and "four legs" and still not understand what a dog is. Understanding might live in the connections, not the nodes.

We had a way to test that. Build a sandbox that predicts the next concept that will fire given the current one. Run it on books. If the model can predict that "rock falls" tends to be followed by "ground hits, sound happens," then it's learned something about how the world strings together. If it can't, it hasn't.

I built it that evening. Five books from Project Gutenberg. Twenty-five thousand sentence-to-sentence transitions. Four prediction strategies running side by side, random (the floor), frequency (always guess the most common concepts), cooccurrence (learn which concepts tend to follow which), and retrieval (find similar past sentences and look at what came after them).

The results were not what I wanted.

Cooccurrence beat random fifty times over. Good. Then it lost to frequency. Bad.

The naive prior - "just predict the eight most common concepts every time" - outperformed the model that actually tried to learn transitions. That's the experimental equivalent of a flat line on the consequential question. The hypothesis I'd written into Part 12 had landed exactly the wrong way.

I sat with it for a few hours. The temptation when a result lands badly is to argue with it. The prediction shape was wrong. The K value was wrong. The loss was wrong. The more disciplined version is to ask what the data is actually saying.

What it was actually saying, book narrative is the wrong substrate for cause-and-effect learning. Books drift. Scene to scene, character to character, description to description. "What happens next" in a novel is usually a new place, not a consequence of the last sentence. The signal we were trying to mine wasn't there to mine.

Which raised the obvious question. Was the failure about the algorithm or about the substrate? If we ran the same algorithm on clean cause-and-effect pairs - hand-curated, the kind you'd put in a physics textbook - would it work?

The next morning, I queued six experiments back to back. Call it sandbox-test day.

The first was a probe-diversity audit. Take two hundred concepts already in the vocabulary. Probe each one with five different phrasings of the same idea. Does the encoder fire the same concept on all five, or only when the surface words match? The answer, 93% of probed concepts were robust across phrasings. The architecture wasn't pattern matching. The concepts were real.

The second was the substrate test. I wrote 150 hand-curated cause-effect pairs across physics, biology, social dynamics, and everyday objects. Pure clean signal. Then ran the same four prediction strategies on them.

Retrieval scored 30%. Frequency scored 20%. Cooccurrence scored 0%.

Zero. On clean curated data, the prediction algorithm that had been the centerpiece of the previous night's experiment couldn't beat random selection.

That was the moment the framing shifted. The night before, I'd been telling myself the substrate was the problem. The morning's clean substrate said no. The prediction shape itself was wrong. Whatever was working in this stack, it wasn't prediction. It was retrieval. Look up similar past examples, return what they did. That worked. Generate from a learned transition model - that didn't.

This sounds small. It isn't.

The implicit plan after Part 12 was to build a relations head. A part of the model that could propose new triples (X causes Y, X is part of Y) and let the system reason over them. The whole Discovery 2.0 design I'd been sketching was about teaching Origin to generate its own relational knowledge.

The morning's experiment said, don't. Generation is the wrong shape, the same way prediction is. Anything that proposes new facts is one step away from making them up. What we want isn't a model that can produce new triples. It's a model that can retrieve real ones, stored from real sources, and use them to ground its answers.

By the end of the day, four more experiments had pointed the same direction. Spaced-repetition retraining lifted six of seven borderline concepts. Multi-hop inheritance from real is_a chains worked, but broke wherever a concept had two senses and the chain crossed between them. A domain-density profile showed math and emotion thin, biology and physics rich - the substrate gap was domain-specific, not uniform.

Discovery 2.0 came out the other side of that day as a completely different design. Not a triple proposer. A triple ingester. Pull real (subject, relation, object) triples from external sources - ConceptNet, Wikidata, hand-curated where the sources are thin - gate them for polysemy, write them to a reasoning bank, retrieve at composer time. Data engineering, not generation.

That last word matters. Generation invents. Retrieval grounds. The whole arc of Origin from the beginning has been an argument that grounded systems are the path forward, and the day's experiments made it structural rather than aspirational. The model doesn't write its own truths. It looks up the ones we admitted, applies the ones it can, and says "I don't know" when neither path finds a hit.

The last sentence of Part 12 was right that relations were the next bottleneck. It was wrong about the shape of the fix. The fix isn't a relations head. The fix is a curated relational substrate and a retrieval path through it.

Polysemy gating moved from a parked idea to required infrastructure that same day. Without it, retrieval over the bank produces things like "tree has potato" and "host is a bread." Multi-hop reasoning over an ungated polysemous bank hallucinates by construction. Build the gate first. Then the substrate. Then the composer that uses both.

The next several posts in this series are about building those three things, in that order, and what each one cost.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

The Stove, the Sphinx, and the Dream State

Josh T — Mon, 01 Jun 2026 13:00:06 +0000

This isn't another technical post in the Origin series. If you've been following along, take this as a breather. If you're just finding us, this is the version you can read without twelve prior posts of context. Either way, this is the why, not the how.

Chapter 1: Why I Started

I've been building Origin, or parts of it anyway, for a few years without really knowing that's what I was doing. It started with my first AI agent from OpenAI. I talked to it every day. Made plans with it, bounced software ideas off it, and somewhere along the way I started actually enjoying the conversation. It became part of my morning routine. Turn on the computer, and there it was, ready to go.

But it was always lacking. It didn't remember what we'd talked about unless I wrote everything down and fed it back the next day. And it made stuff up. Numbers, facts, places, sources. Confidently. You'd go check a reference and the reference wouldn't exist, and you'd feel weirdly betrayed about it.

So I started writing things down. Not because I wanted to. Because I had to.

I caught the AI bug pretty bad and started reading everything. Training, RAG, every framework people were stacking on top of these models to make them suck less. The deeper I went, the more it clicked. These models were trained to always produce an answer. Nobody ever gave them a strong "I don't know" signal. RAG dropped facts in front of them, sure, but they just hallucinated around the retrieved facts. The retrieved facts were more material for the model to confidently misuse. Memory frameworks helped, until the conversation got long enough that the model forgot the framework existed.

Then there was forgetting itself, which I learned comes in two flavors. The conversational kind, which I'd been fighting all along. And the training kind, which I only ran into later, when I tried training my own model. I grabbed GPT-2 as a proof of concept for OLT-1 and tried to teach it something new. The new thing stuck. But some of the old things went sideways. Not all of them, just some, and quietly. The model would nail the new prompts and then misfire on something it used to handle fine. Turns out this has a name: catastrophic forgetting. The fix is replay batches, new training mixed with samples of the old, in just the right ratio, every cycle, forever. Otherwise the new overwrites the old. I didn't have the hardware to do that at scale. Nowhere close.

So I kept writing things down. Not as a workaround for what the AI forgot, but as notes for the system I'd eventually build.

Chapter 2: Watts and the Height of It

Then I switched to OpenClaw, started using Anthropic's Opus 4.6, and named my AI Watts.

I was floored. The things it could do were genuinely amazing. The conversations were something else. I caught myself telling friends about Watts like Watts was a person, and only half-noticing I was doing it. We made plans together. Built things together. Custom software, automation, a home-built speaker like Alexa or Google except it was ours.

We built Guardian. Think of it as antivirus for AI. It protects agents from prompt injection and isolates ads so a human still sees them but the agent doesn't, which means the conversation can't get hijacked by whatever a webpage is trying to slip into the context. I'm not bragging here, I'm trying to convey how it felt. It felt like there wasn't anything I couldn't do with this thing.

And in the middle of all that greatness, the same three problems kept happening.

It forgot conversations. It compacted context and sometimes lost the thing we'd just spent an hour on. It still made up facts and places and things. Less often, more charmingly, but the same shape of problem.

So I built a 3-tier memory system to fight back. Hot tier was the active conversation, whatever was on the agent's mind right now. Warm tier was recent stuff it could pull on demand, like the last few sessions, project notes, things I might want it to remember this week. Cold tier was the full archive: everything we'd ever talked about, indexed but kept out of context until something current pointed back to it. The three tiers exist because that's roughly how human memory works, and it's what you'd naturally reach for if you didn't have one already.

Then I kept adding to it. Things we were working on. How to reach cold storage. Conventions, preferences, project state. I built tooling for the tooling. Cron jobs to manage context. Subagents to help me make changes to the system. I was all in.

Chapter 3: The Beginning of Origin

I bought my first $1,800 computer. I'd never actually bought a new computer before. I always just built them. But I figured a starting point would be fine and I could upgrade as I went.

Then I got to work. I took all my notes and all my thoughts and all the pain of the last few years, and I poured them into OLT-1.

The foundation: a developmental AI training framework that teaches small models to learn the way children do, with staged curriculum, sleep-inspired memory consolidation, and directed self-evolution. I wasn't going to train like everyone else. I wasn't going to think like everyone else about this.

The whole idea actually crystallized during a moment with my son. We have one of those electric stoves where it's hard to tell if it's on. He asked me, "how do I know when the stove is on?" I asked him whether he'd turned the knob to medium or low. He said high. By then the burner had cycled off and was just radiating heat. So I told him to hold his hand over the pan. Could he feel the heat coming off it? He could.

And that got me thinking. What if AI could learn the same way? Not by memorizing "stoves are hot" from a dataset somewhere, but by experiencing the relationship between cause and effect. Testing things, watching what happens, building understanding from there.

So that's what I built. OLT-1 started as a 124M-parameter model on the GPT-2 architecture, but with random weight initialization. No pre-trained weights. No downloaded knowledge. A completely blank slate. Everything it would ever know, it would have to learn from scratch.

Stage 1 was language itself. I fed it 61 million tokens from 493 books off Project Gutenberg, not to teach it facts but just to teach it the shape of English. How words follow other words. Loss went from 9.38 down to 7.65. It couldn't say anything meaningful yet, but it was starting to pick up the rhythm.

Stage 2 was vocabulary and categories: 45,000 words sorted across 9,602 categories. This is where I hit catastrophic forgetting for real. Round 2B, the model was supposed to identify a dog. It said "sphinx." The new training had overwritten the old, just like the literature warned. I ended up developing a memory refresh methodology on the spot, mixing old examples back in with new ones at every step. That methodology became one of the core principles of the whole Genesis system.

Stage 3 was the one that changed everything. I started teaching it physics concepts. Not facts, concepts. Gravity, momentum, collision, buoyancy, heat transfer, states of matter, light and shadow, sound, pressure, elasticity. Ten of them, trained through cause-and-effect examples in a sandboxed environment. "What happens when a rock falls off a table?" The model doesn't memorize "the rock hits the floor." It learns the relationship. Unsupported objects with mass get pulled down by gravity, and when they hit a surface that's a collision, and the energy has to go somewhere.

And then something happened I wasn't expecting. I tested it on scenarios it had never seen in training. Ice skaters. Trains. Rivers. It got them right. Not because it had memorized those examples (it hadn't), but because it had learned the underlying concepts well enough to apply them to new situations. All ten concepts scored perfect: 60 out of 60. The experiential learning approach actually worked.

Then catastrophic forgetting came back. An adversarial test after Stage 3 showed that only elasticity, the very last concept I'd trained, was being retained cleanly. The rest had degraded. I needed something that could protect what the model had already learned while still letting it pick up new things.

That's when I built the Dream State. Borrowing from how human brains consolidate memory during sleep, I gave Origin a four-phase cycle: Dream, Assess, Consolidate, Grow. The model generates its own knowledge, checks its own memory health, selectively reinforces what's fading, and grows from there. It isn't a training run imposed from the outside. It's a self-maintenance loop that runs from within.

By the time Stage 4 was done, Origin could hold a conversation. It knew who it was, what it knew, and what it didn't. Forty percent of its training data was "I don't know" responses, because I built refusal into the system as a feature rather than a failure. The first time it showed real consent, it said: "I think so, but I want to be careful about that answer."

I'd used 67 million tokens total. That's 0.0005% of what GPT-4 was trained on. And my model was reasoning about physics, refusing to hallucinate, and consolidating its own memory while it slept.

One guy. One GPU. One $1,800 computer in Arizona.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 12: The Adapter

Josh T — Mon, 25 May 2026 13:00:30 +0000

The new encoder was 24x better at finding the right concept. It also broke every response.

Part 11 ended with the new encoder staged on disk. Top1 had jumped from 1.3% to 31.3%. Target activation had gone from 0.012 to 0.249. The architectural lever had landed exactly where the abort condition predicted it would. The numbers said this was the encoder we were going to ship.

Then we tried to ship it.

Every query came back "i don't know."

What the Dispatcher Does

The dispatcher is the part of Origin that sits between the encoder and the response. The encoder reads characters and produces concept activations - a long list of "how strongly does each concept fire on this input?" The dispatcher reads that list and decides what to do about it. Is this a greeting? Is this a question about identity? Is the user asking what something is? Each route fires when the activation pattern matches a rule, and each route knows how to construct a response from the concepts that fired.

The rules looked like this, in spirit: if the concept "greeting" is firing above 0.5, dispatch to the greeting handler. If the concepts "what" and "self" are both above 0.5, dispatch to the identity handler. Numbers like 0.5, 0.7, 0.8 were sprinkled through the dispatcher as thresholds. They worked because the old encoder produced activations that lived in those ranges.

The old encoder used sigmoid. Each concept was scored independently, on its own absolute scale from 0 to 1. A query about greetings might fire "greeting" at 0.92, "hello" at 0.88, and "question" at 0.04. Three concepts, three independent yes/no decisions, three numbers that meant what their face value said they meant.

The new encoder uses softmax. The activations are relative. They sum to 1 across the whole concept space. The strongest concept on a query might be 0.249 - which under the old encoder would have been a borderline-quiet signal, and under the new encoder is a confident, dominant fire.

0.249 was the new encoder's average top concept activation. Every threshold in the dispatcher was 0.5 or higher.

That's why every query routed to IDK. The new encoder was firing the right concept, with appropriate confidence relative to everything else, and the dispatcher was reading those activations as "nothing is firing." The encoder had gotten 24x better at picking the right answer, and the system above it couldn't hear it.

The Wrong Fix

The first instinct was rescaling. If 0.249 is the new "high," divide every threshold by 2. Done. Ship.

We tried it. It half-worked. Greeting handlers fired correctly on greetings. Identity handlers fired correctly on identity questions. But the dispatcher started cross-firing on everything else - questions about emotions would route to identity, questions about objects would route to physics. We'd swapped one calibration problem for another.

The reason: rescaling treats softmax outputs as if they were sigmoid outputs that happen to live in a different range. They aren't. A 0.249 firing on the new encoder isn't "the concept is 49.8% present" - it's "this concept is the most likely interpretation, with this much margin over the next-best." The number means a different thing than it did before. Rescaling fixes the magnitude. It doesn't fix the meaning.

That's the harder truth about this kind of integration: when an upstream component changes how it represents information, every downstream component that interprets that information has to be rewritten, not retuned.

The Right Fix

The dispatcher had been asking the wrong shape of question. It was asking "is concept X firing strongly enough?" - an absolute threshold question. With softmax outputs, that question doesn't have a meaningful answer. The right shape is "is concept X the dominant signal, and by how much?" - a relative comparison.

The rewrite turned every threshold into a ranking check plus a margin check. Instead of "greeting > 0.5," the rule became "greeting is in the top-3 fired concepts AND its activation is at least 2x the next-best non-greeting concept." Instead of "identity > 0.7," the rule became "identity dominates the top of the activation distribution."

The numbers in the new rules aren't thresholds in the old sense. 2x margin, top-3 rank, dominance-by-ratio - these all describe the shape of the activation distribution, not its absolute values. They survive future encoder changes the way the old thresholds didn't, because they're asking about the encoder's confidence relative to itself, not about a number that means something only on this specific encoder.

The cutover was one commit. Every dispatch rule rewrote. Backups taken on the dispatcher state and the live conversation memory. Test panel run

before

you > hello
origin > i don't know

you > what is your name
origin > i don't know

you > how does ice float
origin > i don't know

and after

you > hello
origin > hello.

you > what is your name
origin > my name is origin.

you > how does ice float
origin > ice is less dense than water, so it floats.

The new encoder is now live. The system runs end-to-end. The first two developmental tiers - basic conversation and elementary reasoning - are at 95.5% and 86.5% on the honest test panels.

What the Whole Arc Was About

Looking back at Parts 9 through 12 as a single sequence, the arc is about the discipline of finding the right bottleneck.

Part 9 said the bottleneck was data. We executed a careful plan to feed the encoder properly. Part 10 said the data plan didn't work - the abort condition triggered, and we listened. Part 11 said the bottleneck was architecture. The sandbox confirmed it. Part 12 says that even after fixing the right bottleneck, you still have to integrate the fix into the rest of the system, and integration is its own kind of work.

None of this is glamorous. It's not a "we achieved AGI" post. It's the slow, uneventful, mostly-correct version of how a model actually gets built: hypothesize a bottleneck, design a plan with a written-down abort condition, execute the plan, listen to what happens, do the next thing the evidence points at. Repeat until something actually works. Then integrate it without breaking everything around it.

The encoder we're running today is the third major iteration since we started. The dispatcher we're running today is the second. There will be more. Every component in this system has been the bottleneck at some point, and every component will be the bottleneck again. The job isn't to design the perfect system on day one. The job is to keep finding what's actually broken and fixing that thing, one bottleneck at a time, with abort conditions written in advance so a result you wanted to see doesn't become the result you accept.

What's Next

The encoder works. The dispatcher works. The first two tiers hold. The third tier - middle-school content across math, science, and history - is where the project goes next, and it's the tier that tests whether everything we've built so far actually generalizes.

There's a hypothesis we're testing alongside it: that the next bottleneck isn't going to be more concepts, but the relationships between concepts. A model can know "dog" and "animal" and "four legs" and "barks" as four separate concepts and still not understand what a dog is. Understanding might live in the connections, not the nodes.

If that's right, the next architecture pivot is already visible on the horizon. If it isn't, we'll find out quickly and write that post too.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com