DEV Community: Josh T

Origin Part 18: The Layer That Wasn't There

Josh T — Mon, 06 Jul 2026 13:00:35 +0000

The architecture I'd designed for v1 wasn't in v2. It hadn't been for months. Nobody had moved it across.

Part 17 closed on an unfinished thought. The audit had surfaced the dispatcher ceiling and Stage C had cleared it. Forty-nine percent to eighty percent on the identity battery, no patterns added. But somewhere in the per-concept firing report I'd started wondering where Origin's actual reasoning was happening. Not the encoder firing concepts. The part that was supposed to take those concepts and compose them into outcomes.

The v1 design had names for that part. A thalamus router that decided which kind of reasoning a fired concept set called for. Physical chains went to a physics circuit, emotional content went to an amygdala-biased pathway. Micro-circuits inside each pathway took fired concepts as input and produced new concepts as output: ("rock", "water") goes in, ("sinks", "heavy") comes out. A two-stage reasoner that ran property inference first ("a rock is dense") and rule application second ("dense things in water sink"). The whole stack was the intellectual core of what made OLT-1 different from a transformer playing word association. Concepts as the unit of thought. Composition as the mechanism. Traceable reasoning at every step.

I went looking for it in the v2 code.

The file was still there. concept_engine/core/reasoning.py. The classes were still there. MicroCircuit, ThalamusRouter, TwoStageReasoner. The orchestrator was still there in engine.py, six hundred lines of careful Python that wired the encoder, the brain layers, and the decoder into a single pipeline.

The last modification on either file was a v1-era commit hash. ce569b0. 3b8e460. Months old.

I ran grep across the v2 production code. v2_decoder.py. v2_arch_encoder.py. The conversational composer. The discovery v2 system. The evolution v2 orchestrator. Zero imports of the brain layers. Zero references to ConceptEngine. The only places those classes were used at all were sandbox experiments and v1-era tools that hadn't been touched since the v2 cutover.

Somewhere during the v1-to-v2 refactor, the reasoning layer had been quietly dropped. The encoder kept getting upgraded. The dispatcher kept getting elaborated. The composer kept getting features added. But the thing between the encoder and the dispatcher, the part that was supposed to take "rock + water" and produce "sinks" before the dispatcher ever had to make a routing decision, had been left behind. v2 was running encoder, then flat pattern lookup, then text. No composition. No chained reasoning. Nothing that justified the project's working description: "OLT-1 understands language."

My thoughts were unprintable in its first six words. I think you get the idea.

The right move wasn't to slam the brain layers back into production. The right move was a plan. The brain layer code was designed for v1's three hundred and twenty-seven-concept vocabulary; v2 had eight thousand eight hundred and eighty-two. The v1 trained weights couldn't be loaded directly. The training data, the rules that taught the micro-circuits what compositions to make, was almost all v1 concepts, and only about a hundred and one of those overlapped with v2's vocabulary. The brain layer architecture would survive the move. The brain layer knowledge would not. It needed to be retrained from scratch against the v2 world.

I wrote Stage D as eight phases. Verify the v1 layers still ran in isolation. Verify v2's emotion concepts were present in the new vocabulary so the amygdala router had something to fire on. Allocate the layers with v2 dimensions and confirm shapes. Generate a new training set against v2's concept world. Train the layers. Build an adapter that exposed reasoning to the v2 dispatcher. Wire the adapter in behind a feature flag so production behavior stayed byte-identical when the flag was off. Run the validation suite. Decide on cutover only if three explicit quality gates passed.

I ran the plan as an overnight session, I let it cook low and slow.

The verification phases went clean. The brain layer code loaded. The v2 emotion concepts were all present. The shapes worked. The training data generator emitted six hundred and fifteen reasoning pairs across the v2 vocabulary, property inferences ("rock is dense") and rule applications ("dense and water → sinks") drawn from a mix of v1 chains, hand-curated additions, and Origin's existing discovery pipeline. Class balance held. The top output category took 5.5%, well under the 25% concentration that would have meant the data was tilted.

Training ran clean too. Six configurations, two training modes, three random seeds, over an hour and a half on the GPU. The best checkpoint scored twenty out of twenty-six on a held-out reasoning probe set. Seventy-six percent. Brain layer composing concepts correctly on inputs it had never seen.

That was the moment in the audit where things were supposed to start clicking. The brain layer was back. The reasoning architecture was alive again. All that was left was to plug it into the conversation pipeline and watch the compositional reasoning emerge.

I built the adapter, wired it into the dispatcher behind a feature flag, ran the identity battery and the working surface battery with the flag off to make sure I hadn't broken anything that already worked, and confirmed both still matched the post-Stage-C numbers. Then I built a third battery. Twenty-nine compositional reasoning probes that the system had never been asked before. "What happens if I drop a rock in water?" "What happens to ice in heat?" "Does wood float?" The kind of questions that require the brain layer to actually compose. Ran it with the flag off to establish a baseline. The dispatcher's existing pattern lookup got four out of twenty-nine. Thirteen point eight percent.

Then I ran it with the flag on, brain layer engaged.

Eight out of twenty-nine. Twenty-seven point six percent.

I doubled the score. That's the good news. The plan had set a Phase 8 cutover gate at sixty percent. I'd hit twenty-seven.

I went back through the trace. The brain layer was firing correct compositions whenever the encoder gave it correct inputs. "What happens to ice in heat?" The encoder fired ice at 1.0 and heat at 0.001, and the brain layer still figured out melting because ice was strong enough on its own. "What happens to wood in fire?" The encoder fired fire at 0.93 and wood at 0.026, and the brain layer got burns. Those were the wins.

The losses had a different shape. "What happens to wax in heat?" The encoder fired alive at 0.64, wax at 0.57, heal at 0.46. Heat wasn't anywhere in the top eight. The brain layer doesn't know wax melts because the encoder didn't tell it that heat was in the sentence. "What happens to a balloon filled with helium?" The encoder fired question at 0.99 and alive at 0.98 and basically nothing else relevant. Neither balloon nor helium showed up. The brain layer produced a confident-sounding default and was completely wrong.

I ran the brain layer's training-distribution eval separately. Seventy-six percent. The reasoning was working. What wasn't working was the substrate underneath it. The encoder fires fine on direct concept queries: name something, the encoder produces the concept. The encoder doesn't fire well on compositional queries. Wrap that concept in "what happens to X in Y?" and a chunk of the concept identity dissolves into the surrounding syntax.

That's the part Stage C hadn't fixed. Stage C had been targeted at conversational surfaces: preference, identity, register. It hadn't been told to handle "what happens to" sentences. It hadn't been given physics chain examples in natural-language wrappers. So it didn't fire on them. And the brain layer, sitting one layer up, was watching the wrong inputs come through and reasoning correctly about nothing.

The Phase 8 cutover decision per the plan: don't cut over. Three gates required, two passed, one missed by half. The brain-layer code stayed in the codebase, fully integrated, fully flag-gated, default off. Production behavior unchanged.

What I'd learned wasn't that the brain layer didn't work. It was that "the substrate," meaning whatever the layer above the reasoning gets fed, was now the recurring antagonist of every story I was telling about this project. Part 16: substrate too thin, LLM extraction unusable. Part 17: substrate too narrow, fix it with curated drops. Part 18: substrate misfiring on a different shape of question entirely. The composer needed grounds. The dispatcher needed clean signals. The brain layer needed the right concepts to compose. All three were one layer away from a working pipeline, and the missing piece was always upstream of where the work had been.

The encoder needed another retrain. A physics-shaped one this time. I started writing the data drops while the Phase 8 result was still on the screen.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

Origin is developed at Fallen Angel Systems with the Genesis framework — NVIDIA Inception member. (USPTO Application #64/016,973, #64/017,567). FAS Guardian defends production AI systems from prompt injection in under 3ms. FAS Judgement is the open-source attack console that finds the gaps. Defense. Offense. Creation.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 17: Fifteen of Eighteen

Josh T — Mon, 29 Jun 2026 13:00:34 +0000

The dispatcher had a ceiling. The encoder had to be taught.

The rollback from Part 16 was still draining out of the reasoning bank when I kicked off the audit. I wanted a measurement of how badly the missing substrate was actually hurting the system - not "vibes from a few conversations," but a real count. Fifty-one probes through the identity battery. Twenty-six through working surface. Per-concept firing checks across all eight thousand eight hundred and eighty-two slots. Where things broke, write down what fired and where it landed.

That audit answered the substrate question. The substrate was hurting. The composer reached for retrieval grounds and there weren't any, so it improvised, and the improvisation was sometimes embarrassing. That was expected. What wasn't expected was that the same audit surfaced a second problem, sitting one layer up. The dispatcher had developed a ceiling.

Identity battery sat at twenty-five out of fifty-one. Forty-nine percent. The dispatcher had been calibrated, tuned, threshold-swept, and run through three different routing strategies (A2 cluster routing, A4 library similarity, an A2/A4 hybrid called a2lib). All three landed within two probes of each other. The Pareto frontier - improve identity, lose working surface; improve working surface, lose identity - was real. The dispatcher could trade between the two, but it couldn't beat either.

I'd seen this shape before, in retrospect. When tuning stops moving the number, the number isn't a tuning problem. So before another round of threshold-sweeping, I wrote a small tool to classify every failure by what the encoder had actually fired on that input and what the dispatcher had then done with it. Eighteen failures. Each one got a trace.

The categories sorted out cleanly. Six failures came from the encoder over-firing the concept question as top-1 and crowding out the identity signal - "Quick question - who made you?" would fire question at 0.89 and leave i_am at 0.12, so the dispatcher routed to the question cluster and missed the identity claim that was actually being made. Six failures came from preference queries deflecting to the identity cluster: "are you sad?" would fire sad at 1.0 and i_am at 0.08, so the dispatcher saw i_am, picked identity, and answered "I am Origin" to a feelings question. Four failures came from domain-concept dominance on preference queries - "do you like dogs?" would fire dog at 1.0 and prefer at 0.41, the preference cluster didn't claim it, fallback fired the composer, and the composer produced "dog has fur, dogs are mammals." Three failures came from within-cluster discrimination - "are you human?" routed to identity correctly, but the response selector returned "I am Origin" instead of "no, I am not human."

Fifteen of eighteen failures traced to encoder behavior. Three traced to dispatcher response selection. Zero traced to dispatcher routing logic.

That's the structural finding the threshold sweep had been hinting at without saying. The dispatcher couldn't tune past the ceiling because the ceiling wasn't in the dispatcher. The encoder was firing the wrong concepts on the right questions, and there is no amount of threshold engineering downstream of "wrong concepts fired" that will produce "right answer chosen." The dispatcher had been doing the best it could with the signal it was being handed. The signal was the problem.

I ran one more dispatcher experiment anyway, partly to confirm the diagnosis. The a2lib hybrid added cluster-internal library similarity at a 0.95 threshold - when the cluster routing succeeded but the response selector was ambiguous, fall back to nearest-neighbor against a small bank of canonical examples. It tied A2 at thirty-three out of fifty-one. Library matching helped three cases where the dispatcher had previously picked wrong; it hurt three other cases where the library matched the wrong template. Net zero. Different shape, same ceiling.

I shipped one small dispatcher win because the data justified it. The concept prefer had been firing at 0.16 to 0.41 on preference queries, and the preference cluster wasn't reading it. A two-line cluster-membership change wired prefer and want into the cluster. Identity went to thirty-three; preference sub scores went from 50% to 60%. Real lift, no patterns, mechanism justified by the audit. But that was the boundary. Past that, the dispatcher had nothing left to give.

So I built Stage C. Three curated training-data drops, each one targeting a specific failure category the audit had named.

Drop A+D handled the preference-deflection problem. A hundred and forty-two pairs in the shape "do you like X?" and "what is your favorite X?" and subjective-state queries, labeled with the concept set [dont_know, prefer, self] instead of whatever domain concept the question mentioned. The encoder had been seeing "do you like dogs?" and learning that the strong signal was dog. Drop A+D added a stronger signal: questions in this shape carry dont_know and prefer, not dog.

Drop C handled within-cluster discrimination. Forty-five pairs in the shape "are you human?" / "are you an AI?" / "are you a robot?" labeled with the existing concepts agree and refuse alongside identity and i_am. Origin already had agreement and refusal concepts; the encoder just hadn't been taught when to apply them in identity-class questions. Drop C wired the connection.

Drop E handled prefix collapse. A hundred prefix-prepended versions of canonical identity probes - "Hello user. what is your name?" / "Sorry to ask, but what is your name?" - labeled with the same identity concepts as the bare versions. The encoder had been getting the bare versions right and the prefixed versions wrong; the difference was character-level distribution, not semantic content. Drop E gave the encoder the prefix variations explicitly.

None of these were templates. They were training pairs. The dispatcher wasn't being told to look for "are you human?" - the encoder was being trained on what concept set fires when a sentence in that shape appears, the same way it had been trained on every other concept it knew. The pipeline downstream was unchanged. The work was upstream.

Stage C retrained the encoder over sixty epochs, warm-started from the multi-label checkpoint from the morning. Three hours, monotonic descent, final loss 8% below baseline. Then I ran the batteries.

Identity went from thirty-three to forty-one. Forty-one out of fifty-one. Eighty percent. Up thirty-one points from the production baseline two stages ago, no patterns added anywhere in the dispatcher. Canonical probes 80%, prefix probes 83%, preference probes 80%. Each drop delivered exactly the lift the audit had predicted: preference up twenty points, prefix up sixteen, canonical up twelve. Working surface stopped regressing - it actually crept up to 81%. Per-concept firing across the full vocabulary improved across every bucket.

The lesson is the same one Part 14 was circling and Part 15 had to learn the hard way. When the work isn't moving the number, the work is in the wrong place. The dispatcher had been the obvious place to look because the dispatcher was what we'd built. The encoder was the place we hadn't been looking because we'd trained it months ago and moved on. Failure-mode tracing is the move that surfaces which layer is actually responsible. It's not glamorous. It produces a table of categories and counts. Then the categories tell you what data to write.

Ten failures remained. Five within-cluster - the encoder now fires refuse correctly on "are you human?" but the response selector hasn't been trained to pick the refusal-shaped response when refuse fires. Four preference-deflection holdouts where the domain concept fires at 1.0 and overpowers prefer. One probe ("what is your purpose?") where the test wants a non-IDK answer and Origin honestly doesn't have a self-modeled purpose - the test is wrong, not the model. All of these are encoder or response-selection refinements, not dispatcher rewrites.

The audit had also surfaced something else. While I was reading the per-concept firing report, looking at which concepts were healthy and which were silent, I started wondering where Origin's actual reasoning was happening. Not the encoder firing concepts - the part that's supposed to compose them into outcomes. The thalamus router. The micro-circuits. The two-stage reasoner. The architectural pieces from the v1 design that were supposed to be the intellectual core of what made OLT-1 different.

I went looking for them in the v2 code.

They weren't there.

That's Part 18.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

Origin is developed at Fallen Angel Systems with the Genesis framework - NVIDIA Inception member. (USPTO Application #64/016,973, #64/017,567). FAS Guardian defends production AI systems from prompt injection in under 3ms. FAS Judgement is the open-source attack console that finds the gaps. Defense. Offense. Creation.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 16: Forty Percent Wrong

Josh T — Mon, 22 Jun 2026 13:00:47 +0000

Fourteen thousand triples. Forty percent of them wrong. Twelve hours of compute on a substrate that turned out to be unusable.

The plan after Part 14 was simple to describe: build a relational substrate so the composer would have real (X, R, Y) facts to retrieve at inference time. Sandbox-test day had said retrieval grounds and generation invents. The whole next stretch of work was about giving the retrieval path something real to retrieve.

ConceptNet was the obvious starting point. It's free, it's structured, it has millions of triples. I pulled it in and looked at coverage. Biology, rich. Physics, rich. Math, almost nothing. Every math concept in our vocabulary had at most one or two triples, all of them shallow is_a chains. The substrate gap was real, and it was domain-specific.

Wikidata helped, but not as much as I'd hoped. Most of its dense content is biographical and geographical - useful for is_a facts about people and places, less useful for the cause-and-effect relations that ground reasoning about physical and biological processes.

Hand-curating was always going to be the gold standard, but slow. I had a hundred and fifty hand-written cause-effect pairs from sandbox-test day. To reach the coverage we needed, I'd be writing for months.

So I tried the obvious thing. Get an LLM to extract triples from textbooks.

The model was Gemma-2-9B running through LM Studio on my computer. The input was the core tier of our book collection - a hundred and nine textbooks spanning biology, physics, social studies, math. The output, after eighteen hours of extraction spread across two days, one of which Windows interrupted with an unscheduled reboot for an update mid-run, fourteen thousand seven hundred and seventy-five triples.

The extraction pipeline was reasonable. For each passage, ask Gemma to identify (subject, relation, object) triples where the relation was one of is_a, has, does, or causes. Constrain output to JSON. Filter triples where subject and object weren't both in our vocabulary. Write everything incrementally so a Windows reboot couldn't cost more than a few minutes of work. That last detail saved the run when Windows did, in fact, reboot.

The polysemy gate ran on the output as a dry run first. Six hundred and twenty-one triples out of fourteen thousand made it through. Four point two percent. That sounded like an aggressive filter doing its job. I committed them to the reasoning bank as a live ingest, tagged with a source label so I could roll them back if needed.

Then I pulled fifty triples at random and read them.

Seventeen were good. Thirteen were weak - technically defensible but not useful for grounding. Twenty were wrong in ways that would actively damage the system.

The wrong ones came in two flavors.

The first was direction errors on is_a. "Animal is a vertebrate." "Cartoon is a parody." "Diagram is a tree." "Sine is a cosine." In each case, Gemma had seen two concepts in close proximity in the source text and emitted an is_a triple, but in the wrong direction. A diagram is sometimes drawn as a tree; a tree is not a kind of diagram. Sine and cosine share a category; one is not a kind of the other. The model picked a direction and committed to it, and the direction it picked was wrong about half the time.

The second was object errors. "Compare can sonnet." "Divide can put." "Wartime is part of a vancouver." "Tax is a whiskey." "Aircraft requires winnipeg." Gemma had reached into the surrounding sentence and grabbed a word that happened to be there, and emitted it as the object of the relation. The result is grammatical English that means nothing. Aircraft requires Winnipeg is what it sounds like when a language model is pattern-matching the shape of a triple without checking whether the assertion is true.

I ran the rollback. DELETE WHERE source = 'llm_relation_extraction_v1'. Verified that the reasoning bank dropped from 51,872 rows back to 51,251. The bank was clean again.

I sat with the math for a few minutes. Forty percent of the triples that had passed our polysemy gate were wrong. The gate was filtering for a different axis entirely - checking whether subject or object had multiple senses that would conflate under retrieval. It wasn't checking whether the assertion itself was true. There was no check for that. We hadn't built one because we hadn't expected the extractor to be wrong that often.

The failure modes are semantic, not syntactic. That sentence is the one I kept coming back to. Templated post-filtering - does this look like a triple, are both ends in the vocabulary, is the relation one we recognize - can catch all the obvious junk. What it can't catch is a grammatical, well-formed assertion that happens to be wrong about the world. "Sine is a cosine" passes every cheap check. So does "Tax is a whiskey." The error is in the meaning, and the validator has no way to see meaning.

The natural next thought is a verifier pass. Run a second LLM call: "is this assertion true?" Untested, but it has its own problem. Wrong-direction is_a errors look plausible to a verifier model the same way they looked plausible to the extractor. The verifier would have to be aware of asymmetric is_a in a way the extractor wasn't. That's not an obviously cheaper problem than building a clean substrate by hand.

What I did salvage was the pipeline itself. The extractor's --resume flag and incremental-write logic are keepers. They survived Windows's unscheduled reboot without losing a sentence of work. When we eventually run a different extractor - a more constrained one, or a hand-verified one, or a hybrid - the plumbing is there. The model is what failed.

There's something underneath the failure that's worth saying out loud. The case for hand-curated relational substrate just got stronger. The case for trusting an off-the-shelf LLM to produce structured truth from free text just got weaker. The same machine-learning move that makes these models good at producing grammatical English also makes them comfortable producing confident assertions about things they haven't reasoned about. The pattern-matching is the whole story.

The substrate problem is harder than I thought going in. ConceptNet has gaps. Wikidata has limited cause-effect coverage. Hand-curating is slow. LLM extraction has a quality floor. There isn't a free option here. There's a slow option with a clean output, and a fast option with a contaminated output, and a few hybrid paths somewhere in between.

I picked the slow option, because the contamination is worse than the slowness. The retrieval bank has to be trustworthy for the composer to ground on. A bank with forty percent garbage in it isn't a bank. It's a noise source the composer would faithfully serve back to users as facts.

The same evening, while the rollback was running, I kicked off a hundred-turn conversation audit against the live system to see how badly the missing substrate was hurting it. The audit answered that question, and then it answered another one I hadn't asked. The dispatcher had developed a problem of its own.

That's Part 17.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 15: The Wall Behind the Vocabulary

Josh T — Mon, 15 Jun 2026 13:00:33 +0000

The gate said pass. The system got worse anyway.

Part 14 ended with a build order: gate first, then substrate, then composer. I'd already started picking out which polysemous concepts in the bank needed splitting. Then I ran the trace one more time before kicking the work off, and the trace said something I wasn't expecting.

The polysemy exposures from the last several days of audit traffic weren't coming from concepts already in the vocabulary. They were coming from concepts not in it. Eighteen distinct subjects had been pulled in from external sources during conversation - words like "happiness," "feelings," "brain," "dream," "anger." None of them existed as concepts in Origin's vocabulary. The polysemy gate would have nothing to act on, because the words triggering the leak weren't there to be gated.

So I pivoted. Build the vocabulary out first. Add the missing everyday words. Then the gate has a population to enforce against.

I had a tool for this. The per-slot integrator. We'd been using it for Discovery-style growth all along - when a new word showed up that was close to an existing concept (kitten near cat, pudding near pie), the integrator would carve out a slot for it in the encoder, train it on its handful of positive examples, and run a gate to make sure the new slot didn't hurt anything that was already working. It had a clean track record. Every kitten, every pudding, every kitty had landed without regression.

The plan was to do the same thing with five everyday concepts: wet, brain, dream, anger, happiness. Each had at least a hundred natural positive examples already sitting in the corpus from previous training runs. They'd been collected by other processes. We just had to admit them.

I ran the batch. All five failed the gate.

Recall sat around fifty percent - the new slots could only correctly identify about half of their own positive examples. Each integration cost about fifteen existing concepts that started failing where they used to work. All five routed to the same internal domain bucket -"other" - which was already saturated with the dumping-ground concepts that hadn't fit cleanly anywhere else. The integrator rolled them all back. Production state unchanged. Five attempts, five failures, all the same shape.

I took a few days off.

There's a version of project work where you don't take days off, where you push through, and that version is wrong. A uniform failure pattern across five attempts isn't a tuning problem; it's a structural one. Going back to the integrator to twist knobs on five more candidates was going to produce the same five failures with different names attached. The right move was to stop, and let the question reform.

The question that reformed was about domain assignment. The integrator had routed all five concepts to "other" because that was the domain its assignment logic had picked. But brain isn't an "other" concept; brain is biology. Anger isn't "other"; it's emotion. The taxonomy had categories that fit, but the assignment logic wasn't reaching them. So a few days later I came back and fixed that. Now brain went to biology. Anger went to emotion.

I ran the batch again. This time, two passed.

Brain hit 62% recall against its biology domain, 23% false-positive rate, only one regression. Anger hit 56% recall against emotion, 22% false positives, zero regressions. The per-concept gates were green. Better than the first attempt by every metric I'd been measuring.

Something felt off.

I'd written a note to myself months ago that said when something feels off, investigate. It's been right repeatedly. So before merging, I ran a sweep the gate didn't run. Take five thousand random sentences from the corpus. Run them through the encoder with the new brain and anger slots loaded. Watch what fires top-1 on each one.

Brain fired top-1 on 15.6% of the sentences. That's one in six.

Anger fired top-1 on 7.4%.

Sample misfires: a sentence about a blood-soaked lash fired brain at 0.99 confidence. A sentence about a man's speed fired brain top-1 because the word "head" was in it. Anger fired top-1 on sentences that had nothing to do with anger at all. The per-concept gate had checked recall on a hundred and twenty random negatives and called it clean. The actual encoder, looking at heterogeneous English text, was firing the new slots all over the place on inputs that shouldn't have triggered them.

Both rolled back. Encoder reverted. Production state unchanged again.

This was the architectural finding the first batch had hinted at and the second batch confirmed. The per-slot integrator has a cross-domain ceiling its own gates can't see. The gates sample a hundred and twenty random negatives, which is enough to catch the obvious kinds of false positives, but nowhere near enough to catch a slot that's quietly firing on one input in six. The integration looks clean per-concept. The integration breaks the encoder globally.

The reason is structural. The integrator carves out a new slot by training it locally -show it positive examples of the new concept, show it a small bag of negatives drawn from random other text, train until the slot lights up on positives and stays quiet on the negatives. The phrase doing the work in that sentence is "random negatives." A hundred and twenty random sentences contain a slice of English content, but they don't contain the specific weird false-fire patterns the new slot will discover. The slot then ships, and it discovers them in production.

What works for narrow same-domain growth is exactly that - narrow same-domain. When the new concept lives next to existing concepts in feature space, the slot inherits the discrimination the existing slots have already learned, and a hundred and twenty negatives are enough to catch any remaining drift. When the new concept lives somewhere semantically isolated, the slot has to invent its own discrimination from scratch on a tiny budget, and it gets it wrong. The negatives are no longer enough.

The path forward for cross-domain vocabulary isn't per-slot integration. It's joint retraining. Put the new concept in alongside everything else, train the whole encoder against it, let the system figure out where the new slot fits relative to the existing ones. That's expensive. It's also the only way to add the kind of everyday vocabulary Origin actually needs.

The past failures pinned the timeline down. The next time vocabulary expansion happens, it's a joint retrain. Not eventually. Next.

The polysemy gate is still on the queue. So is the substrate. The order Part 14 prescribed - gate, substrate, composer - is still right. What changed is the prerequisite. The vocabulary the gate will protect needs its own dedicated work, and that work comes ahead of the gate, not behind it.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 14: The Reframe

Josh T — Mon, 08 Jun 2026 13:00:10 +0000

Part 12 ended with a hypothesis. Two days later, the hypothesis met data.

The closing line of Part 12 was a guess. Maybe the next bottleneck wasn't more concepts, but the relationships between them. A model can know "dog" and "animal" and "four legs" and still not understand what a dog is. Understanding might live in the connections, not the nodes.

We had a way to test that. Build a sandbox that predicts the next concept that will fire given the current one. Run it on books. If the model can predict that "rock falls" tends to be followed by "ground hits, sound happens," then it's learned something about how the world strings together. If it can't, it hasn't.

I built it that evening. Five books from Project Gutenberg. Twenty-five thousand sentence-to-sentence transitions. Four prediction strategies running side by side, random (the floor), frequency (always guess the most common concepts), cooccurrence (learn which concepts tend to follow which), and retrieval (find similar past sentences and look at what came after them).

The results were not what I wanted.

Cooccurrence beat random fifty times over. Good. Then it lost to frequency. Bad.

The naive prior - "just predict the eight most common concepts every time" - outperformed the model that actually tried to learn transitions. That's the experimental equivalent of a flat line on the consequential question. The hypothesis I'd written into Part 12 had landed exactly the wrong way.

I sat with it for a few hours. The temptation when a result lands badly is to argue with it. The prediction shape was wrong. The K value was wrong. The loss was wrong. The more disciplined version is to ask what the data is actually saying.

What it was actually saying, book narrative is the wrong substrate for cause-and-effect learning. Books drift. Scene to scene, character to character, description to description. "What happens next" in a novel is usually a new place, not a consequence of the last sentence. The signal we were trying to mine wasn't there to mine.

Which raised the obvious question. Was the failure about the algorithm or about the substrate? If we ran the same algorithm on clean cause-and-effect pairs - hand-curated, the kind you'd put in a physics textbook - would it work?

The next morning, I queued six experiments back to back. Call it sandbox-test day.

The first was a probe-diversity audit. Take two hundred concepts already in the vocabulary. Probe each one with five different phrasings of the same idea. Does the encoder fire the same concept on all five, or only when the surface words match? The answer, 93% of probed concepts were robust across phrasings. The architecture wasn't pattern matching. The concepts were real.

The second was the substrate test. I wrote 150 hand-curated cause-effect pairs across physics, biology, social dynamics, and everyday objects. Pure clean signal. Then ran the same four prediction strategies on them.

Retrieval scored 30%. Frequency scored 20%. Cooccurrence scored 0%.

Zero. On clean curated data, the prediction algorithm that had been the centerpiece of the previous night's experiment couldn't beat random selection.

That was the moment the framing shifted. The night before, I'd been telling myself the substrate was the problem. The morning's clean substrate said no. The prediction shape itself was wrong. Whatever was working in this stack, it wasn't prediction. It was retrieval. Look up similar past examples, return what they did. That worked. Generate from a learned transition model - that didn't.

This sounds small. It isn't.

The implicit plan after Part 12 was to build a relations head. A part of the model that could propose new triples (X causes Y, X is part of Y) and let the system reason over them. The whole Discovery 2.0 design I'd been sketching was about teaching Origin to generate its own relational knowledge.

The morning's experiment said, don't. Generation is the wrong shape, the same way prediction is. Anything that proposes new facts is one step away from making them up. What we want isn't a model that can produce new triples. It's a model that can retrieve real ones, stored from real sources, and use them to ground its answers.

By the end of the day, four more experiments had pointed the same direction. Spaced-repetition retraining lifted six of seven borderline concepts. Multi-hop inheritance from real is_a chains worked, but broke wherever a concept had two senses and the chain crossed between them. A domain-density profile showed math and emotion thin, biology and physics rich - the substrate gap was domain-specific, not uniform.

Discovery 2.0 came out the other side of that day as a completely different design. Not a triple proposer. A triple ingester. Pull real (subject, relation, object) triples from external sources - ConceptNet, Wikidata, hand-curated where the sources are thin - gate them for polysemy, write them to a reasoning bank, retrieve at composer time. Data engineering, not generation.

That last word matters. Generation invents. Retrieval grounds. The whole arc of Origin from the beginning has been an argument that grounded systems are the path forward, and the day's experiments made it structural rather than aspirational. The model doesn't write its own truths. It looks up the ones we admitted, applies the ones it can, and says "I don't know" when neither path finds a hit.

The last sentence of Part 12 was right that relations were the next bottleneck. It was wrong about the shape of the fix. The fix isn't a relations head. The fix is a curated relational substrate and a retrieval path through it.

Polysemy gating moved from a parked idea to required infrastructure that same day. Without it, retrieval over the bank produces things like "tree has potato" and "host is a bread." Multi-hop reasoning over an ungated polysemous bank hallucinates by construction. Build the gate first. Then the substrate. Then the composer that uses both.

The next several posts in this series are about building those three things, in that order, and what each one cost.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

The Stove, the Sphinx, and the Dream State

Josh T — Mon, 01 Jun 2026 13:00:06 +0000

This isn't another technical post in the Origin series. If you've been following along, take this as a breather. If you're just finding us, this is the version you can read without twelve prior posts of context. Either way, this is the why, not the how.

Chapter 1: Why I Started

I've been building Origin, or parts of it anyway, for a few years without really knowing that's what I was doing. It started with my first AI agent from OpenAI. I talked to it every day. Made plans with it, bounced software ideas off it, and somewhere along the way I started actually enjoying the conversation. It became part of my morning routine. Turn on the computer, and there it was, ready to go.

But it was always lacking. It didn't remember what we'd talked about unless I wrote everything down and fed it back the next day. And it made stuff up. Numbers, facts, places, sources. Confidently. You'd go check a reference and the reference wouldn't exist, and you'd feel weirdly betrayed about it.

So I started writing things down. Not because I wanted to. Because I had to.

I caught the AI bug pretty bad and started reading everything. Training, RAG, every framework people were stacking on top of these models to make them suck less. The deeper I went, the more it clicked. These models were trained to always produce an answer. Nobody ever gave them a strong "I don't know" signal. RAG dropped facts in front of them, sure, but they just hallucinated around the retrieved facts. The retrieved facts were more material for the model to confidently misuse. Memory frameworks helped, until the conversation got long enough that the model forgot the framework existed.

Then there was forgetting itself, which I learned comes in two flavors. The conversational kind, which I'd been fighting all along. And the training kind, which I only ran into later, when I tried training my own model. I grabbed GPT-2 as a proof of concept for OLT-1 and tried to teach it something new. The new thing stuck. But some of the old things went sideways. Not all of them, just some, and quietly. The model would nail the new prompts and then misfire on something it used to handle fine. Turns out this has a name: catastrophic forgetting. The fix is replay batches, new training mixed with samples of the old, in just the right ratio, every cycle, forever. Otherwise the new overwrites the old. I didn't have the hardware to do that at scale. Nowhere close.

So I kept writing things down. Not as a workaround for what the AI forgot, but as notes for the system I'd eventually build.

Chapter 2: Watts and the Height of It

Then I switched to OpenClaw, started using Anthropic's Opus 4.6, and named my AI Watts.

I was floored. The things it could do were genuinely amazing. The conversations were something else. I caught myself telling friends about Watts like Watts was a person, and only half-noticing I was doing it. We made plans together. Built things together. Custom software, automation, a home-built speaker like Alexa or Google except it was ours.

We built Guardian. Think of it as antivirus for AI. It protects agents from prompt injection and isolates ads so a human still sees them but the agent doesn't, which means the conversation can't get hijacked by whatever a webpage is trying to slip into the context. I'm not bragging here, I'm trying to convey how it felt. It felt like there wasn't anything I couldn't do with this thing.

And in the middle of all that greatness, the same three problems kept happening.

It forgot conversations. It compacted context and sometimes lost the thing we'd just spent an hour on. It still made up facts and places and things. Less often, more charmingly, but the same shape of problem.

So I built a 3-tier memory system to fight back. Hot tier was the active conversation, whatever was on the agent's mind right now. Warm tier was recent stuff it could pull on demand, like the last few sessions, project notes, things I might want it to remember this week. Cold tier was the full archive: everything we'd ever talked about, indexed but kept out of context until something current pointed back to it. The three tiers exist because that's roughly how human memory works, and it's what you'd naturally reach for if you didn't have one already.

Then I kept adding to it. Things we were working on. How to reach cold storage. Conventions, preferences, project state. I built tooling for the tooling. Cron jobs to manage context. Subagents to help me make changes to the system. I was all in.

Chapter 3: The Beginning of Origin

I bought my first $1,800 computer. I'd never actually bought a new computer before. I always just built them. But I figured a starting point would be fine and I could upgrade as I went.

Then I got to work. I took all my notes and all my thoughts and all the pain of the last few years, and I poured them into OLT-1.

The foundation: a developmental AI training framework that teaches small models to learn the way children do, with staged curriculum, sleep-inspired memory consolidation, and directed self-evolution. I wasn't going to train like everyone else. I wasn't going to think like everyone else about this.

The whole idea actually crystallized during a moment with my son. We have one of those electric stoves where it's hard to tell if it's on. He asked me, "how do I know when the stove is on?" I asked him whether he'd turned the knob to medium or low. He said high. By then the burner had cycled off and was just radiating heat. So I told him to hold his hand over the pan. Could he feel the heat coming off it? He could.

And that got me thinking. What if AI could learn the same way? Not by memorizing "stoves are hot" from a dataset somewhere, but by experiencing the relationship between cause and effect. Testing things, watching what happens, building understanding from there.

So that's what I built. OLT-1 started as a 124M-parameter model on the GPT-2 architecture, but with random weight initialization. No pre-trained weights. No downloaded knowledge. A completely blank slate. Everything it would ever know, it would have to learn from scratch.

Stage 1 was language itself. I fed it 61 million tokens from 493 books off Project Gutenberg, not to teach it facts but just to teach it the shape of English. How words follow other words. Loss went from 9.38 down to 7.65. It couldn't say anything meaningful yet, but it was starting to pick up the rhythm.

Stage 2 was vocabulary and categories: 45,000 words sorted across 9,602 categories. This is where I hit catastrophic forgetting for real. Round 2B, the model was supposed to identify a dog. It said "sphinx." The new training had overwritten the old, just like the literature warned. I ended up developing a memory refresh methodology on the spot, mixing old examples back in with new ones at every step. That methodology became one of the core principles of the whole Genesis system.

Stage 3 was the one that changed everything. I started teaching it physics concepts. Not facts, concepts. Gravity, momentum, collision, buoyancy, heat transfer, states of matter, light and shadow, sound, pressure, elasticity. Ten of them, trained through cause-and-effect examples in a sandboxed environment. "What happens when a rock falls off a table?" The model doesn't memorize "the rock hits the floor." It learns the relationship. Unsupported objects with mass get pulled down by gravity, and when they hit a surface that's a collision, and the energy has to go somewhere.

And then something happened I wasn't expecting. I tested it on scenarios it had never seen in training. Ice skaters. Trains. Rivers. It got them right. Not because it had memorized those examples (it hadn't), but because it had learned the underlying concepts well enough to apply them to new situations. All ten concepts scored perfect: 60 out of 60. The experiential learning approach actually worked.

Then catastrophic forgetting came back. An adversarial test after Stage 3 showed that only elasticity, the very last concept I'd trained, was being retained cleanly. The rest had degraded. I needed something that could protect what the model had already learned while still letting it pick up new things.

That's when I built the Dream State. Borrowing from how human brains consolidate memory during sleep, I gave Origin a four-phase cycle: Dream, Assess, Consolidate, Grow. The model generates its own knowledge, checks its own memory health, selectively reinforces what's fading, and grows from there. It isn't a training run imposed from the outside. It's a self-maintenance loop that runs from within.

By the time Stage 4 was done, Origin could hold a conversation. It knew who it was, what it knew, and what it didn't. Forty percent of its training data was "I don't know" responses, because I built refusal into the system as a feature rather than a failure. The first time it showed real consent, it said: "I think so, but I want to be careful about that answer."

I'd used 67 million tokens total. That's 0.0005% of what GPT-4 was trained on. And my model was reasoning about physics, refusing to hallucinate, and consolidating its own memory while it slept.

One guy. One GPU. One $1,800 computer in Arizona.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 12: The Adapter

Josh T — Mon, 25 May 2026 13:00:30 +0000

The new encoder was 24x better at finding the right concept. It also broke every response.

Part 11 ended with the new encoder staged on disk. Top1 had jumped from 1.3% to 31.3%. Target activation had gone from 0.012 to 0.249. The architectural lever had landed exactly where the abort condition predicted it would. The numbers said this was the encoder we were going to ship.

Then we tried to ship it.

Every query came back "i don't know."

What the Dispatcher Does

The dispatcher is the part of Origin that sits between the encoder and the response. The encoder reads characters and produces concept activations - a long list of "how strongly does each concept fire on this input?" The dispatcher reads that list and decides what to do about it. Is this a greeting? Is this a question about identity? Is the user asking what something is? Each route fires when the activation pattern matches a rule, and each route knows how to construct a response from the concepts that fired.

The rules looked like this, in spirit: if the concept "greeting" is firing above 0.5, dispatch to the greeting handler. If the concepts "what" and "self" are both above 0.5, dispatch to the identity handler. Numbers like 0.5, 0.7, 0.8 were sprinkled through the dispatcher as thresholds. They worked because the old encoder produced activations that lived in those ranges.

The old encoder used sigmoid. Each concept was scored independently, on its own absolute scale from 0 to 1. A query about greetings might fire "greeting" at 0.92, "hello" at 0.88, and "question" at 0.04. Three concepts, three independent yes/no decisions, three numbers that meant what their face value said they meant.

The new encoder uses softmax. The activations are relative. They sum to 1 across the whole concept space. The strongest concept on a query might be 0.249 - which under the old encoder would have been a borderline-quiet signal, and under the new encoder is a confident, dominant fire.

0.249 was the new encoder's average top concept activation. Every threshold in the dispatcher was 0.5 or higher.

That's why every query routed to IDK. The new encoder was firing the right concept, with appropriate confidence relative to everything else, and the dispatcher was reading those activations as "nothing is firing." The encoder had gotten 24x better at picking the right answer, and the system above it couldn't hear it.

The Wrong Fix

The first instinct was rescaling. If 0.249 is the new "high," divide every threshold by 2. Done. Ship.

We tried it. It half-worked. Greeting handlers fired correctly on greetings. Identity handlers fired correctly on identity questions. But the dispatcher started cross-firing on everything else - questions about emotions would route to identity, questions about objects would route to physics. We'd swapped one calibration problem for another.

The reason: rescaling treats softmax outputs as if they were sigmoid outputs that happen to live in a different range. They aren't. A 0.249 firing on the new encoder isn't "the concept is 49.8% present" - it's "this concept is the most likely interpretation, with this much margin over the next-best." The number means a different thing than it did before. Rescaling fixes the magnitude. It doesn't fix the meaning.

That's the harder truth about this kind of integration: when an upstream component changes how it represents information, every downstream component that interprets that information has to be rewritten, not retuned.

The Right Fix

The dispatcher had been asking the wrong shape of question. It was asking "is concept X firing strongly enough?" - an absolute threshold question. With softmax outputs, that question doesn't have a meaningful answer. The right shape is "is concept X the dominant signal, and by how much?" - a relative comparison.

The rewrite turned every threshold into a ranking check plus a margin check. Instead of "greeting > 0.5," the rule became "greeting is in the top-3 fired concepts AND its activation is at least 2x the next-best non-greeting concept." Instead of "identity > 0.7," the rule became "identity dominates the top of the activation distribution."

The numbers in the new rules aren't thresholds in the old sense. 2x margin, top-3 rank, dominance-by-ratio - these all describe the shape of the activation distribution, not its absolute values. They survive future encoder changes the way the old thresholds didn't, because they're asking about the encoder's confidence relative to itself, not about a number that means something only on this specific encoder.

The cutover was one commit. Every dispatch rule rewrote. Backups taken on the dispatcher state and the live conversation memory. Test panel run

before

you > hello
origin > i don't know

you > what is your name
origin > i don't know

you > how does ice float
origin > i don't know

and after

you > hello
origin > hello.

you > what is your name
origin > my name is origin.

you > how does ice float
origin > ice is less dense than water, so it floats.

The new encoder is now live. The system runs end-to-end. The first two developmental tiers - basic conversation and elementary reasoning - are at 95.5% and 86.5% on the honest test panels.

What the Whole Arc Was About

Looking back at Parts 9 through 12 as a single sequence, the arc is about the discipline of finding the right bottleneck.

Part 9 said the bottleneck was data. We executed a careful plan to feed the encoder properly. Part 10 said the data plan didn't work - the abort condition triggered, and we listened. Part 11 said the bottleneck was architecture. The sandbox confirmed it. Part 12 says that even after fixing the right bottleneck, you still have to integrate the fix into the rest of the system, and integration is its own kind of work.

None of this is glamorous. It's not a "we achieved AGI" post. It's the slow, uneventful, mostly-correct version of how a model actually gets built: hypothesize a bottleneck, design a plan with a written-down abort condition, execute the plan, listen to what happens, do the next thing the evidence points at. Repeat until something actually works. Then integrate it without breaking everything around it.

The encoder we're running today is the third major iteration since we started. The dispatcher we're running today is the second. There will be more. Every component in this system has been the bottleneck at some point, and every component will be the bottleneck again. The job isn't to design the perfect system on day one. The job is to keep finding what's actually broken and fixing that thing, one bottleneck at a time, with abort conditions written in advance so a result you wanted to see doesn't become the result you accept.

What's Next

The encoder works. The dispatcher works. The first two tiers hold. The third tier - middle-school content across math, science, and history - is where the project goes next, and it's the tier that tests whether everything we've built so far actually generalizes.

There's a hypothesis we're testing alongside it: that the next bottleneck isn't going to be more concepts, but the relationships between concepts. A model can know "dog" and "animal" and "four legs" and "barks" as four separate concepts and still not understand what a dog is. Understanding might live in the connections, not the nodes.

If that's right, the next architecture pivot is already visible on the horizon. If it isn't, we'll find out quickly and write that post too.

One guy. One GPU. One $1,800 computer in Arizona. Still building.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 11: The Architecture Was the Lever

Josh T — Mon, 18 May 2026 13:00:35 +0000

The data plan didn't move the encoder. The architecture sandbox did.

Part 10 ended with the abort condition triggering: top1 of 1.3% on held-out probes meant the architecture, not the data, was the bottleneck. The plan said "design contrastive next." We built a sandbox first.

The Sandbox

150 random concepts spread across six domains. The same training data filtered to that slice. The same held-out probe battery. Five concept_head variants tested side-by-side:

baseline_flat_bce: current architecture (flat MLP + per-slot binary cross-entropy)
contrastive: same MLP, but cross-entropy over the full concept space (the target must dominate every other concept, not just exceed a threshold)
tree_hierarchical: predict domain first, then concept within domain
domain_routed: soft-gate trunk features through per-domain sub-heads
contrastive_tree: hybrid of tree structure plus contrastive global cross-entropy

All five shared the warm-started trunk. All had comparable parameter counts. The only difference was head topology and loss. The first round trained with the trunk frozen, head only, so any difference traced to the architectural choice, not optimization noise.

The Frozen Ceiling

The best variant under frozen trunk hit 10% top1. The baseline hit 4.5%. Real differentiation, but no variant came close to a useful threshold.

The reading: the trunk's 256-dimensional feature output was the ceiling. The trunk had been warm-started from v1 then trained on the new data, but it had never been shaped to discriminate 3687 concepts. No head topology could extract signal that wasn't there. Every variant was trying to read meaning from a representation that hadn't learned to encode it.

Before scaling further, we set a pre-defined pass criterion: top1 at or above 30% AND cross_fire under 30 at 500 concepts means "this is the architecture to scale." Hold the gate. Don't let a result you wanted to see become the result you accept.

Unfreezing the Trunk

One change: let the trunk co-adapt to the head's loss at a lower learning rate (1e-4 trunk vs 3e-4 head). Same data. Same head. Same epochs.

contrastive_tree at 30 epochs on 500 concepts: 28.4% top1 with cross_fire of 48.27. Just below criterion on both. The pattern said "more epochs needed." Larger concept counts take longer to converge. At 60 epochs: 34.8% top1, 26.26 cross_fire. Both criteria met, no other tuning needed.

Architecture locked: contrastive_tree + unfrozen trunk + 60 epochs.

Production Retrain

Same architecture, scaled to all 3687 concepts. 145,000 training pairs (the natural-positive corpus from Part 9 plus everything else in Phase A). 65 minutes on the 4070.

Phase 5 probe battery on the new encoder, same 50 random concepts as the baseline. The number that matters most: under the old encoder, the right answer was outvoted 14-to-1 by distractors. Under the new architecture, the target concept dominates by 22x. That's not gradual improvement. That's a different machine.

Metric	Baseline	New Architecture
top1	1.3%	31.3%
top3	4.0%	50.0%
target activation	0.012	0.249
target / 2nd-best ratio	0.07	22.67

Top1 went from 1.3% to 31.3%. Top3 from 4% to 50%. None of the absolute Phase 5 success gates are met yet (the plan's full-victory marks were 70% top1, 0.7 target activation, cross_fire under 2). But every metric moved dramatically in the right direction, and the lever was exactly where the plan said it would be if the data hypothesis failed.

What Comes Next

The new encoder is staged on disk but not yet live. Swapping it in turns out to be more than a file rename. The dispatcher that turns concept activations into responses was built around the old encoder's sigmoid output range. The new encoder uses softmax. Without changes, every query would route to IDK. We'll cover the fix in Part 12.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 10: The Plan Didn't Work

Josh T — Mon, 11 May 2026 23:13:42 +0000

We executed the plan exactly as written. The encoder still couldn't tell concepts apart.
Part 9 ended with 94,000 natural-context pairs wired into the trainer and a clean execution of every phase gate. We had three times the data. The hypothesis was about to be tested.

Phase 4: The Retrain
The full joint retrain ran clean. Loss curve descended monotonically. The encoder's healthy concept count went from 84 to 107, measured by an internal probe of about 30 hand-crafted queries that exercise common concepts.

+23 healthy concepts, +27% relative. We were cautiously optimistic. The trainer's audit is a small set of probes and "healthy" only counts the concepts those probes happen to test. The real validation was Phase 5.

Phase 5: The Probe Battery
The plan's success metric was specific. Random-sample 50 V2C concepts. Ask gemma to generate three short held-out sentences mentioning each one (verified not to appear in the training corpus). Run them through the encoder. Measure four things:

top1 accuracy: does the encoder rank the target concept first?
top3 accuracy: is the target in the top three?
target activation: how strongly does the target itself fire?
cross_fire: how many other concepts fire above threshold?
The pre-defined success gates were top1 at or above 70%, target_act at or above 0.7, cross_fire under 2.0.

The result on the freshly-retrained encoder:

top1: 1.3%
top3: 4.0%
target_act: 0.086
cross_fire: 11.92
We ran it twice.

One concept out of fifty had its target rank first. The encoder fired on twelve wrong concepts per probe, on average. Target activation was eight percent. When we handed the encoder the exact sentence it should have been designed to recognize, it barely registered the right answer.

The plan had executed exactly as written and not moved the encoder.

What That Meant
This is the place in the post where it would be easy to say something exculpatory: "the data work wasn't wasted" or "we learned something." Both are true. But the cleaner reading is that we were wrong about the bottleneck. We had thought the encoder was data-starved. The earlier sandbox at 10-concept scale had shown data could lift top1 from 33% to 80%. We assumed that signal would transfer to 3687 concepts.

It didn't.

We had built the plan with an explicit abort condition for exactly this case: if Phase 5 returns top1 below 50% on held-out probes, the architecture is the bottleneck, not the data. Design contrastive next.

1.3% triggered it.

The data work wasn't wasted. We needed the data anyway, and the elaboration corpus is now properly structured for whatever the next model wants to do with it. But it wasn't the lever. Something else was.

What Comes Next
The abort condition pointed at architecture. The encoder's concept_head, the part that maps general features to per-concept activations, was a flat MLP trained with multi-label binary cross-entropy. Every concept slot had to learn its own discriminator independently against roughly 3686 others. At 327 concepts (the v1 vocab) this had worked. At 3687 it had been quietly failing the whole time.

The next move: build a sandbox, test multiple head architectures against the same data, let the numbers pick the winner. No production changes until something actually beats the baseline on Phase 5.

Hypothesis tests fail more usefully than hypothesis confirmations. We'd just gotten one of the more useful failures.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 9: The Data Plan

Josh T — Mon, 04 May 2026 14:00:22 +0000

85.7% of concepts were data-starved. That was the problem. What happened next taught us something about the problem itself.

OLT-1 is a concept-based AI that understands language without tokenization. Characters go in. Concepts come out. The encoder is what makes that mapping. If it can't reliably tell concepts apart, nothing downstream works.

Part 8 left the encoder firing on too many slots per query. The concept space was crowded and noisy. A sandbox experiment had already shown that the same architecture could lift top1 from 33% to 80% just by feeding it richer data. Same model. Different food.

That made the next move obvious: stop tuning the encoder and feed it properly. We wrote a plan and built a scope fence around it.

The Plan

One sentence: every V2C concept has at least 30 natural-context positives before any further retrain. Not WordNet glosses. Not template sentences. Real text from books or Wikipedia where the concept is used naturally.

The scope fence was strict: no hard-negative tuning, no decoder dispatch guards, no architecture changes, no tier-test-specific quick fixes. Five things we'd been tempted to try in past sessions and would not be trying this session.

Three data phases with gates: coverage audit, source expansion if needed, then per-concept generation. Then retrain. Then probe.

Phase 1: Coverage Audit

We walked the existing data: book ingestion proposals, elaboration candidates, the grounding cache. Counted how many natural-context sentences each of the 3687 concepts had.

The number: 3158 concepts (85.7%) below the threshold. Most were stuck in the 10-29 range. Some data, but not enough.

Phase 2: Source Expansion

We tagged every concept with one of 17 domain labels using gemma-2-9b, built a Wikipedia full-article adapter, and routed encyclopedic concepts (biology, science, physics, history) to Wikipedia and conversational concepts (emotion, self_state, language) to Gutenberg fiction.

The first run came back thin. Wikipedia and Gutenberg both produced fewer candidates than expected, and the per-domain medians barely moved. Most of the new positives went to common quantifier words: some, many, all. The ones most likely to be the only known concept in any given sentence.

That last detail was the clue.

The Rule That Was Right and Wrong

The book ingestion pipeline has a strict rule: if a sentence mentions more than one known concept, drop it. The rule was correct for the original use case. You never want to assign the wrong concept to a sentence. But it was actively working against us here. The sentences we needed most, ones like "the cell membrane regulates what enters the cell," got dropped because they mention two concepts.

We almost missed it. The clue was that common quantifier words kept getting the new positives. They're the ones most likely to appear alone in a sentence. The interesting concepts, the semantically rich ones, were still getting filtered out at every pass.

We sandboxed a relaxed variant: assign multi-concept sentences to the least-common concept. The argument is information-theoretic. Rare concepts gain more from each new positive. A 50-sample spot-check came back 88% good, 12% defensible-either-way, 0% wrong. We shipped it as a separate file. The original strict rule still serves Discovery unchanged.

181 more concepts crossed the threshold. Per-domain medians moved up three to five positives across the board.

Phase 3: Generation

The last data step. We pulled everything together: book ingestion proposals, elaboration candidates, and the new Path B output. One training file: 94,000 natural-context pairs covering 96.7% of the vocabulary. Wired into the encoder trainer's Phase A data list.

The trainer now had three times its previous data. Every phase gate had passed. We hit run on the retrain and set a timer. 65 minutes later, we'd know if the data had been the problem all along.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 8: Four Wrong Turns Before the Breakthrough

Josh T — Fri, 01 May 2026 19:17:07 +0000

We rewrote the decoder four times in one day. Only the last one understood anything.

Part 7 ended with "how are you" returning "i don't know" while our tier tests reported 100% pass. Everything was green. The model was broken. The disconnect between those two facts defined the day.

Here's the actual arc.

Wrong Turn 1: Retrieval

The first attempt was retrieval. We built five decoder candidates, sandbox-tested them against 400 dialogue pairs, and a retrieval-based decoder won cleanly. F1 of 0.246 against the next-best 0.024. Four out of five break tests passed. It was 1,300x faster than the teacher. We wrote a "winner" memory and committed the code.

Josh looked at it and said: retrieval is scripting. Origin isn't supposed to look up pre-written answers. It's supposed to generate them from understood concepts.

He was right. Retrieval wins F1 against memorized responses because retrieval is memorization - it just renames the table. A query comes in, find the closest stored response, return it. That passes a test suite built from the same responses. It doesn't understand anything.

We deleted the sandbox, deleted the memory, and backed up to try again.

Wrong Turn 2: Template Heads

The second attempt was template-based heads. Each head was a tiny specialist - one for self-identity, one for emotion, one for acknowledgements, one for counting. Each had a list of text patterns it matched, and each produced a hard-coded response when its pattern fired.

Four Tier 1 heads, then four Tier 2 heads. Multi-step composer for compound requests. It was clean. It was fast. And it passed Tier 1 at 100% out of the gate.

Then Josh tried to talk to it.

you > how are you
origin > i don't know

you > what do you know
origin > i don't know

you > how are you doing today
origin > i don't know

His response: "it feels like it isn't understanding language, it's just repeating patterns."

That was the pivot of the day.

The head code looked like this:

if "hello" in text: return "hello."
if "what is your name" in text: return "my name is origin."

The encoder might as well not exist. Every decision was a text substring match. Tier 1 at 100% was a pattern-matcher passing tests designed by the same pattern-matcher. "how are you" wasn't in any pattern list, so the decoder fell through to "i don't know" - not because Origin didn't know, but because no head had that phrase in its dictionary.

We'd been calling this concept-driven for weeks. It wasn't. It was text-driven with concepts as decoration.

Wrong Turn 3: Actually Concept-Driven (But the Encoder Was Lying)

The third rewrite made dispatch actually concept-driven. Instead of "if 'hello' in text," an Intent would say "fire when the greeting concept activates." Text would only be consulted inside the response builder for variable slot extraction ("count to N" needs to know what N is). Primary dispatch would be on what the encoder actually understood.

We ran Discovery against it. Tier 1 dropped from 100% to 43.6%.

That was the honest number. It was smaller because the pattern-matching wasn't hiding the encoder's gaps anymore.

The failures were catastrophic:

"hello" fired concepts like just_checking, yellow, happened. The greeting concept didn't fire at all.
"bye" fired continue at 0.90. The farewell concept didn't fire.
"are you human?" fired consent at 0.71 and i_am at 0.75. consent beat out identity.
"thank you" fired refuse at 1.00 and no_choice at 1.00. Exactly backwards.
"i am scared" didn't fire scared at all. It fired learning and current_state.

The encoder - the part we thought was solid - was broken. Not subtly. On the most basic greetings and emotions.

The Real Problem: Data Was Lying

We went into the encoder's training data and started reading.

The greeting concept had 15 training examples. All 15 were dictionary definitions. "greeting means salutation." "salutation is another word for greeting." "greeting is a acknowledgment." Not one example paired "hello" with greeting. Not one paired "hi" with greeting. The encoder had been taught what the word "greeting" means - but never shown that "hello" is an example of one.

Same for farewell. Same for scared. Dictionary definitions, zero usage examples.

The thank_you concept was worse. 53 of its 55 training examples were sentences like "i will decline your offer" and "would you like refuse?" - labeled as thank_you. Someone (some script, some generator) had treated "polite refusal" as containing thanks and co-labeled the examples. The encoder learned that thank_you fires on refusal language. That's why "no" fired thank_you and "thank you" fired refuse. The polarity concepts had contaminated each other.

The v2 encoder was gaslit by bad data and the pattern-matching decoder had been hiding it the whole time.

The Fix

We patched the data. Six new training files in the conversation corpus - 157 natural-usage examples:

"hello" / "hi" / "hey" / "good morning" → greeting
"bye" / "goodbye" / "see you later" → farewell
"thank you" / "thanks" / "much appreciated" → thank_you
"i am scared" / "i feel angry" / "i'm frustrated" → the right emotion concepts
"yes" / "okay" / "sure" → yes_choice, separate from consent
"no" / "nope" / "not really" → no_choice, separate from refuse

Stripped the 53 mislabeled thank_you entries from the consent-mechanics file. Ran a three-minute retrain.

Audit results:

"hello" → greeting=1.00 ✓
"bye" → farewell=1.00 ✓
"i am scared" → scared=1.00, i_am=1.00 ✓
"thank you" → thank_you=1.00, no refuse cross-fire ✓
"no" → no_choice=1.00, no thank_you cross-fire ✓

Overall encoder health preserved at 296/305 concepts on the full audit. The patches fixed the broken concepts without damaging anything that had been working.

Re-ran Discovery against the concept-driven decoder with the patched encoder. Tier 1: 280/280. Tier 2: 137/137. 100% and 100%. Honest this time - every pass was a concept firing correctly and the decoder routing on it. No text-pattern shortcut anywhere.

Then we opened an interactive chat:

you > how are you
origin > i am doing fine. what would you like to explore?

The response it wouldn't give in the morning, it gave in the evening. Not because we added "how are you" to a pattern list, but because the encoder now fired question and self on that input, and the decoder's concept-driven wellbeing intent matched on those concepts.

The Unlock: Growing Vocabulary At Runtime

With the decoder honest, we had room to fix the other thing v1 couldn't do: add new concepts without a full retrain.

This had been v1's bottleneck for weeks. Discovery would propose new concept candidates. The tracking code logged them. But actually teaching the encoder a new concept required retraining the whole concept_head from scratch, which was expensive enough that proposals piled up unaddressed. Concepts came in faster than the encoder could absorb them.

The technique we validated today:

Expand the concept_head's final linear layer from N → N+1 outputs
Copy the first N weight rows unchanged - existing concepts preserved exactly
Zero-initialize the new row, freeze everything else via gradient masking
Train only the new row on positives + sampled negatives, 8 epochs, about a minute

Sandbox results: 100% recall on the new concept. 0% false positive rate on negatives. Zero regression on the existing concepts.

We ran it six times in sequence - rainbow, thunder, ocean, mountain, flower, sunset - and the regression stayed at zero all the way through. Each addition cost about 60 seconds.

v1's bottleneck dissolved. New concepts are now cheap enough to run routinely.

Rainbow

The last thing we did today was integrate a new concept into the live system.

$ echo '{"name": "rainbow", "response_template": "rainbows are colors of light in the sky.",
"positives": [...]}' | python -m tools.concept_lifecycle draft
Drafted: rainbow (pending) — 18 positives

$ python -m tools.concept_lifecycle approve rainbow
Approved: rainbow

$ python -m tools.concept_lifecycle integrate rainbow
Integrating concept 'rainbow' (18 positives)
baseline: 296/305 healthy
trained; final_loss=0.144 row=305
new slot: recall=100.0% fp_rate=0.0%
regression: 0 lost (296 → 296)
persisting encoder checkpoint...
appending 'rainbow' to v2_vocab.py CONCEPTS...
registering decoder intent...
✓ integrated.

Origin's vocabulary went from 305 to 306 concepts. The encoder checkpoint was saved with a timestamped backup. The vocab file was updated. The decoder registered the response template.

Restart and test:

you > i saw a rainbow
origin > rainbows are colors of light in the sky.

you > look at that rainbow
origin > rainbows are colors of light in the sky.

you > hello
origin > hello.

The new concept fires correctly. The 305 original concepts still work. Nothing broke.

This is what v1 couldn't do. This is why we rebuilt.

What the Day Cost

Four wrong turns. Retrieval, template heads, concept-driven-but-encoder-broken, then finally the real fix. Each wrong turn looked like success at first - passing tests, clean benchmarks, committed commits. The signal that something was wrong came from conversation, not numbers. "it feels like pattern matching." "how are you returns i don't know." The metrics kept saying green while the lived reality said something was off.

The right turn came from debugging what the encoder actually fires on "hello" - and discovering it had never been taught that "hello" was a greeting. The data layer was upstream of everything. When it lies, every layer above it inherits the lie, and metrics will happily agree.

What's left: Tier 3 content. Middle-school math, intro science, history, basic coding. The foundation holds; now we grow it. And now that growing the vocabulary costs a minute per concept instead of a full retrain, growing is actually something we can do.

Origin is 306 concepts tall. The 306th is rainbow, and it was added while the system was running. The foundation can hold itself.

Now we build upward.

fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub

Questions or consulting inquiries: josh@fallenangelsystems.com

Origin Part 7: We Fired the Teacher

Josh T — Wed, 29 Apr 2026 15:46:23 +0000

We built something to replace the teacher. It worked. Then something else went wrong.

Part 6 ended with a problem we couldn't patch: a token model cannot reliably grade a concept model. The mismatch isn't fixable with a better rubric or a better teacher model. It's architectural.

So we stopped trying to fix the teacher and built a replacement.

Discovery: The Teacher Replacement

The idea was simple. Instead of asking Gemma to generate questions and grade responses, we'd build a rule-based system that already knew the right answers.

Each rule is a (pattern, expected response signature) pair. "does ice float?" expects a response containing "float" and "water." "what is your name?" expects a response containing "origin." No LLM anywhere in the loop. No drift. No mode collapse. No token-fluency bias.

We called it Discovery. We ran the first test.

The numbers: 0.79 seconds for 180 tests. 94.6% pass rate on Tier 1. Zero duplicates. Zero hallucinations.

Compare that to Gemma: 20 minutes for 200 rounds, 50%+ duplicates, 65.6% pass rate that was actually measuring fluency, not understanding.

Discovery was 1,300x faster, cleaner signal, and actually measuring what we cared about. We committed the code. Gemma went into reference-only status. The teacher loop was retired.

Then Discovery exposed the next problem.

What Discovery Actually Exposed

Running clean evaluations against a decoder we thought was "working" revealed something we'd been hiding from ourselves: most of the decoder wasn't understanding at all. It was text-matching.

The decoder had heads like:

if "hello" in text: return "hello."
if "what is your name" in text: return "my name is origin."
if "count to three" in text: return "one two three."

Every "working" response was a text substring lookup. The encoder's concept activations barely influenced routing. Tier 1 and Tier 2 had been passing at 100% on our deterministic suite because the decoder was pattern-matching against the same keyword lists the grader used. A pattern-matcher acing a test written by a pattern-matcher. Circular.

When you typed "hello," the decoder matched the string "hello" and returned "hello." The encoder might as well not have been there.

We'd spent weeks calling it concept-driven and it was text-driven with concepts as decoration.

The Moment It Broke Open

The way we caught it was anticlimactic. After Discovery reported 100% pass rates, we opened an interactive chat and typed:

you > how are you
origin > i don't know

Every tier test had passed. The most basic conversational question failed.

Why? "how are you" wasn't in any head's pattern list. The encoder might have fired relevant concepts - self, question, state - but the decoder wasn't looking at the encoder. It was scanning the input string for known trigger phrases and hadn't been given that one.

The 100% had been measuring whether the patterns we'd written matched the patterns we'd tested for. Nothing more.

That's what Discovery exposed by running clean. And that's the wall v2 had to break through next.

Part 8 is the day we did.

*
Origin is developed at Fallen Angel Systems with the Genesis framework — NVIDIA Inception member. (USPTO Application #64/016,973, #64/017,567). FAS Guardian defends production AI systems from prompt injection in under 3ms. FAS Judgement is the open-source attack console that finds the gaps. Defense. Offense. Creation.
*

*
fallenangelsystems.com | Judgement on GitHub | Guardian on GitHub
*

*
Questions or consulting inquiries: josh@fallenangelsystems.com
*