Slang Is a Dial: Training a 125M Language Model From Scratch on Free GPUs

TL;DR: I built a language model from scratch, end to end, on free GPUs: a 125.9M-parameter model that writes in a register you choose, with slang, emoji, abbreviation, elongation and formality as five dials. The idea is simple and testable: measure the register of every training document with code, write it as five control tokens in front of the text, and the model learns p(text | register). On 200 held-out requests the generated register lands within 0.146 of the request on average, against 0.646 for a shuffled-request baseline. Behind that number sit a dated slang lexicon, a deterministic register annotator, a data pipeline and tokenizer, a verified Llama-style model, a lease-based training relay that turns flaky free sessions into one continuous run, a regression-tested training stack, and a Llama-format export that loads in three lines of transformers. Five live sketches below let you poke each idea.

Model: Ravikishan/bussin-125m · Code: Ravikisha/Bussin

The numbers at a glance

Model	125.9M parameters, Llama architecture, 49,152-token vocabulary, 1,024 context
Training	About 7.8 billion tokens on free Kaggle T4 pairs, $0 spent
Register control	Register error 0.146 against 0.646 for a shuffled-request baseline (200 held-out requests)
One dial at a time	slang_3 measured 2.88, emoji_2 measured 2.00, abbrev_3 measured 2.40
Verification	Six behavioural test suites; export matches the original to 3.6e-05 in logits
Try it	Ravikishan/bussin-125m on the Hub, code on GitHub

---

Table of Contents

1. The idea: cringe is register mismatch
2. What I built
3. Register as an input
4. Measuring register with code
5. A dated lexicon that knows what is current
6. The data pipeline and tokenizer
7. The model
8. Training systems
9. The relay: continuous training on free compute
10. Engineering rigor
11. Results
12. Talking to it
13. Shipping it
14. Skills this project demonstrates
15. What is next

---

The idea: cringe is register mismatch

Ask a general-purpose model to "talk like Gen Z" and you get "Dear Sir, no cap, I hope this email finds you bussin." It knows the words. It has no model of when they fit.

That led to the central observation of this project: nobody cringes at a teenager saying "that's so cooked" to a friend. They cringe when a bank's Twitter account says it. So cringe is not slang. It is register mismatch: the words do not fit the setting.

If that is right, the average-of-all-text model is the wrong tool, because averaging the register of a contract and a Twitch chat produces exactly that blend. What you want is a model that treats register as something you can set, and a way to measure whether it did what you asked. Both turned out to be buildable with no human labelling at all.

What I built

This is a full-stack project, and every layer is code in one repository, with tests:

layer	what I built
Research	A 17-section specification: the thesis, the data tiers, the compute plan, the evaluation design
Lexicon	A dated slang dictionary with current / aging / dead status and emerging-term discovery
Annotator	A deterministic five-dimension register scorer that also grades model output
Data	Mining, cleaning, language routing, PII scrubbing, stage-ordered uint16 shards
Tokenizer	A 49,152-token byte-level BPE with the register control tokens built in
Model	A Llama-style 125.9M decoder with a verification suite
Training	WSD schedule, curriculum, multi-GPU trainer, exact token accounting, hashed checkpoints
Systems	A lease-based relay and a stateless orchestrator that run training across free sessions
Evaluation	Register Calibration Error, a paired likelihood test, single-dimension sweeps
Release	A Llama-format export with a logit-parity check, a model card, a repo with figures

Register as an input

Instead of p(text), learn p(text | register):

$$\mathcal{L}(\theta) = -\sum_{t} \log p_\theta\left(x_t \mid x_{\lt t},\, r\right)$$

where $r$ is a short vector describing the register of the whole document. In practice $r$ is five small integers, written as control tokens in front of every training document:

<|endoftext|><|reg|><|slang_3|><|emoji_1|><|abbrev_2|><|elong_0|><|formal_0|><|/reg|> ngl that movie was bussin

At inference you write the prefix yourself and the model continues in that register. Slang stops being a prompt trick and becomes a dial.

dimension	levels	what it measures
slang	0 to 4	weighted lexicon hits per 100 tokens
emoji	0 to 3	emoji and chat emotes per 100 tokens
abbrev	0 to 3	internet abbreviations (ngl, tbh, fr) per 100 tokens
elong	0 to 2	stretched words (sooooo) and keysmashes
formal	0 to 3	a 0 to 1 formality score from five signals

The register tokens are real vocabulary: 34 special tokens with fixed ids, including end-of-text, padding, register open and close, and every dimension level, so slang level 3 is always token 7. Drag the knobs to build a prefix and see exactly what the model is given, with the real token ids. Set a single dial to its sweep level to see what the trained model produced for it.

A frame from the interactive sketch in the original.

The whole pipeline, end to end:

Diagram from the original.

The last arrow is the part I like most. Because the register is measured by code and not labelled by people, the same annotator that labels the training corpus can score what the model writes. That makes "cringe" a number, which I call Register Calibration Error (RCE):

$$\mathrm{RCE} = \frac{1}{N}\sum_{n=1}^{N} \frac{1}{5}\sum_{d \in \text{dims}} \left| r^{\text{requested}}{n,d} - r^{\text{generated}}{n,d} \right|$$

It rises when the model under-uses slang and when it over-uses it, so it cannot be gamed by suppressing slang.

Measuring register with code

The annotator is deterministic code in bussin/data/register.py: no LLM calls, no human labelling, and cheap enough to run over a whole corpus (it is budgeted at about 30,000 documents per second per core). The thresholds that turn raw rates into levels:

dimension	cut points	unit
slang	0.5, 2, 5, 10	weighted hits / 100 tokens
emoji	0.5, 2, 6	emoji / 100 tokens
abbrev	2, 6, 14	abbreviations / 100 tokens
elong	1, 4	elongation events / 100 tokens
formal	0.30, 0.55, 0.78	score from 0 to 1

The design decisions are where the thinking is:

• Calibrated buckets. The cut points are chosen so every level holds a meaningful slice of the corpus. A short chat message is about 10 tokens, so a rate per 100 tokens would put a single slang word in the top bucket. Dividing by at least 30 tokens treats a short message as weak evidence, which gives the model a smooth gradient of levels to learn from instead of two extremes.
• Ambiguity handling. "Living" and "cap" are ordinary words and slang. Ambiguous words count at 30% weight unless the document also contains an unambiguous slang term, which anchors them at full weight. That trades a little recall for far fewer false positives on ordinary prose.
• One signal, one dimension. Emotes (KEKW) and abbreviations (ngl) feed their own dimensions and are excluded from the slang count, even though dictionaries list them too.
• Formality from five signals. Terminal punctuation, capitalised sentence starts, mean word length, formal function words (however, therefore) and the absence of chat markers, averaged with equal weight.

Here is a simplified port you can poke. It uses a small lexicon, but the thresholds, the 30-token floor, the anchoring rule and the five-signal formality score are the real ones. Click to change the message, and toggle the floor to see what it does to the one-word example.

A frame from the interactive sketch in the original — watch it run.

A dated lexicon that knows what is current

Slang needs a lexicon, and a lexicon needs a source. I built mine from the largest public Urban Dictionary dump plus curated lists. Every term carries a first-seen date and a status (current, aging or dead) that is refined from corpus evidence, and the dead terms double as a test set for outdated slang.

Then I stress-tested it against 19 contemporary terms and learned something useful about the structure of the problem: the newest definition in that dump is dated 2023-11-09, so a dated dictionary can say what is dead but structurally cannot say what is current. Lexicographers lag usage by years and the dumps lag the lexicographers. So the lexicon has three layers:

• Dated sources for the long tail and for status.
• CURRENT_SLANG: a curated set of unambiguous recent coinages (bussin, skibidi, fanum tax, mewing, cheugy) that doubles as the anchor set for the ambiguity rule above.
• DRIFTED_SENSES: ordinary words whose meaning moved, with a documented sense for each: cooked ("ruined, or doing extremely well"), ate ("performed excellently"), cap, bet, mid, no crumbs, hits different. A frequency filter normally removes these, and it is right to in general, so each exemption needs a written reason.
• Emerging-term discovery: a corpus pass that ranks words by recent-versus-baseline frequency against an English vocabulary, so new slang can enter the lexicon from the data itself.

The build turns 152,941 dictionary definitions into 55,499 terms plus 1,322 curated entries.

The data pipeline and tokenizer

The pipeline is a sequence of numbered stages, from building the lexicon through mining, tokenizer training, tokenization and sharding. A few choices worth calling out:

• Language routing, not language dropping. Statistical language ID is unreliable on short chat, so the router combines py3langid with marker lists and routes English and Hinglish into their own pools, dropping other languages.
• PII scrubbing on the body. A mandatory pass over the text itself, not just the schema, replaces handles, emails, phone numbers and similar patterns with placeholder tokens.
• Author columns dropped at ingest, before anything is written.
• Elongation capped but remembered. sooo and soooooo carry different register information, so the cleaner caps the run length and records the original length for the annotator.
• Stage-ordered shards. Tokens are written as uint16 shards tagged with a curriculum stage, so training order is a property of the data, not of a flag.
• A tokenizer gate. The 49,152-token byte-level BPE had to pass a round-trip check before any training: exact round trip (1.0000) and 3.37 bytes per token on Gen-Z text.

The model

A small Llama-style decoder. The point of choosing something standard is that every result is about the data and the objective, not the network.

Parameters	125,851,392
Layers / width	14 / 768
Attention	12 query heads, 4 KV heads (grouped-query), head dim 64
FFN	SwiGLU, hidden 2048
Positions	RoPE, theta 10,000
Norm	RMSNorm (computed in fp32)
Embeddings	tied; vocabulary 49,152 (byte-level BPE)
Context	1,024

Embeddings are $49152 \times 768 = 37{,}748{,}736$ parameters, about 30% of the model, which is why tying them pays off at this size. Three implementation details:

• Block-diagonal document mask. Training packs several short documents into one window. The mask is causal and limited to the same document, so nothing attends across a boundary.
• fp16 plus a gradient scaler. The free T4 GPUs have no bf16, so activations run in fp16 while logits and loss are computed in fp32, where a 49k-wide logit row does not saturate.
• z-loss. A small penalty keeps the log-partition from drifting:

$$\mathcal{L} = \mathrm{CE} + 10^{-4}\,\overline{\left(\log \sum_j e^{z_j}\right)^{2}}$$

The model ships with its own verification suite: parameter counts match the analytic formula exactly at all four planned sizes (125M, 400M, 1B and a smoke model); untrained cross-entropy lands at 10.95 against $\ln(49152) \approx 10.80$ ; and tests assert causality, document isolation, RoPE against an independent reference implementation, and grouped-query head grouping.

The same code scales: the repository carries configurations for 400M (396,418,176 parameters) and 1B (1,255,245,824), sized so that the 400M model reaches 75.7 tokens per parameter on the same free-compute plan.

Training systems

A schedule built for an unknown budget. The usual cosine schedule needs the total step count before step one. A free quota is genuinely unknown (Kaggle's own wording is "30 hours or sometimes higher depending on demand"), so I used Warmup-Stable-Decay: warm up, hold the peak for as long as the quota lasts, then decay whenever you decide to finish.

$$
\eta(s) =
\begin{cases}
\eta_{\max}\,\dfrac{s+1}{W} & s \lt W \[4pt]
\eta_{\max} & W \le s \lt S \[4pt]
\eta_{\min} + (\eta_{\max}-\eta_{\min})\left(1-\sqrt{\dfrac{s-S}{D}}\right) & s \ge S
\end{cases}
$$

The decay phase doubles as the Gen-Z anneal: the last steps run at a low learning rate over the most contemporary data, so register gets shaped without overwriting the general-English competence learned earlier. A stable-phase checkpoint stays valid, so a different ending never costs the whole run. Move your mouse across the plot to change when the quota ends:

A frame from the interactive sketch in the original — it does not render here.

Curriculum as data order. Shards are read in stage order (general, then a ramp, then the Gen-Z anneal), with shuffling inside a stage and never across one, so the curriculum is guaranteed by the loader.

Multi-GPU without the tax. The model is wrapped in DataParallel through a thin module that returns only scalar losses, which avoids gathering roughly 0.8 GB of logits on every micro-step. Measured throughput on two T4s is about 14,000 tokens per second, at a global batch of 262,144 tokens.

Exact token accounting. The trainer counts the tokens the loader actually served and refuses to run if that differs from what the plan assumes, so the "tokens seen" figure in every checkpoint is real, not an estimate.

Checkpoints you can trust. Each checkpoint (about 1.66 GB with optimizer state) holds weights, optimizer, scheduler, gradient scaler, RNG state and the data cursor, with a SHA-256 manifest verified on load. Local copies are removed after a verified push so a long session never fills the disk, and old checkpoints are pruned within a single run's namespace only.

The relay: continuous training on free compute

Free sessions end after at most 12 hours, or crash, or hit a quota. So training is a relay: each session resumes exactly where the last one stopped, with no server in the middle.

• Checkpoints go to the Hugging Face Hub.
• A single file, RUN_STATE.json, is the lease: who holds the run and until when.
• The lease is claimed by compare-and-swap on the Hub's commit SHA: read the file and its revision, write with that revision as the parent, and if anyone committed first the Hub answers 412 Precondition Failed. That rejection is the CAS.
• The holder heartbeats every 10 minutes, and each heartbeat re-reads the state to confirm it still owns the lease.
• A stateless orchestrator reads state and quota on a schedule and starts a session only when the lease is free or expired.

Diagram from the original.

This sketch simulates the protocol with illustrative timings (the real heartbeat is 10 minutes and checkpoints land about every 50). Watch a session end, the lease lapse and the next session win the claim, sometimes against a duplicate that loses the race. Click to end the current session.

A frame from the interactive sketch in the original — it does not render here.

Around that core sits a set of operational mechanisms:

• Divergence watchdog. It compares the median loss of the last 50 steps with the previous 50; a jump above 1.5x triggers a rollback to the last good checkpoint at a reduced learning rate. It resets at stage boundaries and treats a gradient-scaler overflow as a skipped step.
• Stall detector. Three consecutive sessions ending on the same step halt the orchestrator instead of spending quota.
• Quota ledger. A ledger tracks GPU-hour use per platform, reconciles it and closes orphaned sessions.
• Platform adapters. Dispatch adapters for Kaggle GPU and TPU, Colab and Lightning share one interface, with Kaggle GPU run end to end.
• History compaction. The checkpoint repository is squashed automatically when its commit history grows, only when no live lease exists.
• A status dashboard rendered to a static page after every tick.
• Bit-exact resume. An interrupted run reproduces an uninterrupted one with 0.0 relative divergence, which a test verifies.

Over the project the relay moved training across many sessions on free Kaggle T4 pairs: about 110 GPU-hours and 6.74 billion tokens in the pretraining phase, then 1.07 billion tokens (4,067 steps) in the register-curriculum anneal.

Engineering rigor

A model trained for days on free hardware cannot be debugged by re-running it, so the project leans on tests that assert on behaviour, not just on code paths:

test	what it checks
model_sanity_test	init loss near $\ln V$ , causality, document isolation, RoPE vs an independent reference, GQA grouping
objective_test	trains a tiny model through the real shard writer, loader and loss, then asserts on what it learned
accounting_test	13 checks: token accounting, simulated two-device runs, curriculum order, prune scope
lease_test	18 checks, including the two-worker race and heartbeat ownership
relay_e2e_test	16 checks across a full two-session hand-off, including no leftover local checkpoints
resume_test	interrupted vs uninterrupted run: 0.0 relative divergence

The most valuable of these is objective_test. During an audit of the training objective I found a loss-alignment defect (a duplicated label shift) that the usual signals do not reveal, because a healthy-looking loss curve cannot tell you which target the model is learning. I fixed it at the loss, wrote objective_test so the property is checked by what the model learns (it fails on the pre-fix loss and passes on the fix), and continued training under the corrected objective. Treating the loss contract as something to test, not assume, is a habit I would carry to any training stack.

Results

Two complementary tests.

1. A likelihood test. Score the same held-out text under its true register prefix, a shuffled one and none, paired by document. At step 3,314 a shuffled prefix cost +0.054 nats on average (95% CI +0.047 to +0.062), and the true prefix beat the shuffled one in 80.7% of documents: the model genuinely conditions on the register.

2. A generation test (RCE). I took 200 requests from the measured registers of real held-out documents, stratified so every slang level is requested. The model wrote 48 tokens from the prefix alone and the annotator measured the output. The shuffled-request baseline scores the same generations against another document's request, which is what ignoring the prefix looks like. Click the sketch to flip between the two pairings:

A frame from the interactive sketch in the original — it does not render here.

The model improves on the shuffled baseline on every dimension, with every 95% interval excluding zero, and its output lands on the requested register more tightly than a real continuation of the same document does (0.146 against 0.466). The cleanest demonstration is to turn one dial at a time, everything else at zero, 50 samples each:

Requesting slang_3 measured 2.88, emoji_2 measured exactly 2.00, abbrev_3 measured 2.40, elong_2 measured 1.80 and formal_3 measured 2.62. At level 0, every dimension measured between 0.00 and 0.02. The register is a control you can actually reach for.

A sample from the prefix slang_3 emoji_1 abbrev_2 (temperature 0.8, one word masked):

I actually thought the same, I was a lil disappointed as I was expecting some cool s*** for this movie, but then i actually liked it, I hope the director can find something funny to make a

Casual, hedged, slangy, in the register that was asked for.

Talking to it

bussin-125m is a base language model, not an assistant. It has no instruction tuning, so it does not answer questions: it continues text in the register you ask for. That makes the register prefix the whole interface, and the cleanest way to see it is to hold the message fixed and turn the dials. Same message, same sampling settings (outputs are cut at 60 tokens), different registers.

"thoughts on the new iphone"

register	dials	the model continues
formal	slang 0, emoji 0, abbrev 0, formal 3	I don't know much about phones, so I've been wondering how to connect my phone to the phone with a SIM. I was thinking of buying the new one, however I'm not quite sure how to connect to the new phone. Do you think the SIM connection is the problem?
chill	slang 1, abbrev 1	also if you know anyone who uses airpods, pls share them
casual	slang 3, emoji 1, abbrev 2	i don't know what the phone is worth rn but it's a nice looking phone that i wanna use for a couple of hours, but i dont know if it's worth anything at all to have a phone that's mid to not worth \$100 also, i feel like i would prefer it
hype	slang 4, emoji 2, abbrev 3, elong 1	i don't know what the phone is worth rn but if u wanna get a new phone lmk i wanna get a new iPhone idk if i should get the iphone 12 or the 12 pro i wanna keep my 16 pro btw 🙏

"is pineapple on pizza ok or nah" (the chill setting returned only a bare link for this one, so it is left out)

register	dials	the model continues
formal	slang 0, emoji 0, abbrev 0, formal 3	I've been meaning to ask if you're okay with this particular pizza and if I should try my best to make it better? I'm going to eat it.
casual	slang 3, emoji 1, abbrev 2	for me pineapple is literally the best pizza because its easy to get in and out of, and its not even half as good as the other two i would say pineapple on pizza is actually worse imo, u get the same amount of cooked pizza than pizza on pizza
hype	slang 4, emoji 2, abbrev 3, elong 1	for a reference i don't eat pineapple because i love eating pineapple but if u like pineapple on pizza, sure, u can eat pineapple on pizza bro so i'm not actually gonna say pineapple on pizza is bad so yeah, i love pineapple on pizza bro 😭😭😭

Read down the columns: formal gives capitalised, fully punctuated sentences, while casual and hype give lowercase text, rn, lmk, imo, u, and emoji at the high setting. Nothing about the message changed. That is the dial.

I also ran 50 everyday prompts through it: 20 casual, 15 neutral and 15 formal, each with a matching register prefix, same settings throughout. A representative selection (outputs are cut at 70 tokens, and ... marks where I shortened one further):

prompt	register	the model continues
i have an exam tomorrow and havent studied	casual	i dont wanna fail, i wanna get into clg (pls do tell me if it actually works) can u pls tell me how to study? i wanna study for 12 hrs everyday edit: how many days will be left?
can you recommend a good show to binge	casual	i kinda like this kinda vibe tho like if u guys want to start u can try it like naruto naruto is a good show but if u want more anime, pls give me recommendations if u have any suggestions pls give ...
what's the best game right now	casual	edit: actually the best one rn is smash bros but i'm gonna go with smash bros ultimate edit 2: i'm playing sonic 3 rn and i like the character so much that i'm kinda thinking of buying it, it's ...
should i cut my own hair	casual	i don't think my hair is actually that good tho, i have thick, wavy and straight hair and i've had a really bad time with frizz i want to be able to grow a nice hair idk ...
best way to save money as a student	chill	http try to avoid eating out tho
Explain the difference between weather and climate.	formal	Climate is a term used in studies that look at the climate of a particular region or area, especially in relation to weather patterns and the overall climate of a particular region or area.
What are the benefits of regular exercise?	formal	What kind of exercise is recommended to lose weight? How long does it usually take for your body to adjust? Also, what are the risks associated with increasing your activity level?
How should I prepare for a job interview?	formal	My interview went on for 5 hours. I don't want to waste any more time, so I was wondering how I can prepare accordingly.
What should I consider when buying a laptop?	formal	Parts list: GPU: Intel® NVIDIA GeForce RTX 5080 16 GB Video Card CPU: AMD Ryzen 7 9800X3D 3.7 GHz 8-Core Processor AIO: Thermalright Frozen Notte 360 ...

The pattern across all 50: the model holds a register for a whole passage and stays near the topic for a sentence or two, and it behaves like a forum or chat post on the subject rather than an answer to it. That is exactly what a base model trained on community text should do, and it is the foundation an instruction-tuning stage would sit on. Like any base model trained on raw community text, it can produce rough content, so the weights are released for research use.

Shipping it

The model is a standard Llama architecture, so publishing it for everyone is a matter of renaming tensors, and the export proves the rename is correct:

embed.weight                    -> model.embed_tokens.weight
blocks.N.attn.wq / wk / wv      -> model.layers.N.self_attn.q_proj / k_proj / v_proj
blocks.N.attn.wo                -> model.layers.N.self_attn.o_proj
blocks.N.ffn.w_gate / up / down -> model.layers.N.mlp.gate_proj / up_proj / down_proj
blocks.N.attn_norm / ffn_norm   -> input_layernorm / post_attention_layernorm
norm.weight                     -> model.norm.weight   (lm_head is tied, so it is dropped)

The export script loads both the original model and the converted LlamaForCausalLM and compares logits on random token ids: max difference 3.6e-05 and 100% argmax agreement. It refuses to report success otherwise. The result is on the Hub with a model card, a tokenizer and a generation config, and anyone can use it:

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "Ravikishan/bussin-125m"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo).eval()

# levels: slang 0-4, emoji 0-3, abbrev 0-3, elong 0-2, formal 0-3
prompt = "<|endoftext|><|reg|><|slang_3|><|emoji_1|><|abbrev_2|><|elong_0|><|formal_0|><|/reg|>"
ids = tok(prompt, return_tensors="pt", add_special_tokens=False)
out = model.generate(**ids, max_new_tokens=60, do_sample=True, temperature=0.8,
                     top_k=50, top_p=0.95, pad_token_id=1)
print(tok.decode(out[0][ids["input_ids"].shape[1]:]))

The repository carries the specification, the configs, the tests, the evaluation scripts and a figure generator that draws every chart in the README from the measured results, so the charts cannot drift from the numbers.

Skills this project demonstrates

• ML research and problem framing. Turning a fuzzy cultural idea ("cringe") into a conditional-modelling hypothesis and a metric (RCE) that is cheap, deterministic and hard to game.
• Data engineering. A dated lexicon, language routing, body-level PII scrubbing, stage-ordered sharding and a tokenizer gate.
• Model engineering. A Llama-style decoder from first principles: RoPE, grouped-query attention, SwiGLU, document masking, fp32 logits, z-loss, with an analytic parameter-count check.
• Training systems. WSD scheduling, multi-GPU data parallelism without the logit-gather tax, exact token accounting, hash-verified checkpoints and bit-exact resume.
• Distributed-systems thinking. A lease implemented as compare-and-swap on a commit SHA, heartbeats with ownership checks, stall detection and a stateless orchestrator.
• Evaluation design. Paired likelihood tests, shuffled-request baselines, single-dimension sweeps and bootstrap intervals.
• Testing discipline. Behavioural tests that fail on the old code, including a test of what the model learns.
• Release engineering. A tensor-renaming export with a logit-parity gate, a model card, a reproducible figure pipeline and a public repository.
• Resourcefulness. About 110 GPU-hours of training on free tiers, with no paid compute.

What is next

The natural next layer is instruction tuning, so the register dial sits on top of an assistant that answers questions. The scaffolding is built for a bigger run: the 400M and 1B configurations are written and sized, the lease relay is platform-agnostic, and the next steps are a Gen-Z validation set so the loss curve can see the anneal, human ratings to complement the annotator, a content-safety filter ahead of a larger corpus, and a from-scratch run of the audited training stack at 400M.

Everything is open:

• Model: huggingface.co/Ravikishan/bussin-125m
• Code and write-up: github.com/Ravikisha/Bussin
• Raw results: docs/results/rce_final-4067.txt in the repo

If you take one idea from this: make the thing you care about measurable, then make it an input. Once register was a number, it could be labelled by code, conditioned on, and evaluated, all without a single human annotation.