DEV Community: Senna

I Rebuilt Jev's Structure with Qwen (Not Its Capabilities)

Senna — Mon, 21 Sep 2026 03:28:45 +0000

Jev does not generate an answer string. It makes a typed decision in one forward pass.

So, what kind of model is Jev?

Jev takes two things: a state and a set of questions.

The state is the context each decision is based on. Each question then asks for one typed answer:

Noul: yes or no, represented by one probability
Choice: one item from a supplied set, plus a distribution over the options
Score: a value on a supplied scale, with a score, distribution, and confidence

The official API documentation has the concrete request and response shapes.

The striking part is the speed. TypeSafe's own comparison looks like this:

	Jev	Typical frontier LLM
Latency	70–500 ms	3–329 s

Why is that gap so large?

Jev only needs one forward pass.

A normal LLM builds an answer autoregressively. It produces a token, feeds that token back in, and repeats until it reaches an end token:

Autoregressive generation — one forward pass per output token

[state][question]                 -> Transformer -> "escal"
[state][question]"escal"          -> Transformer -> "ate"
[state][question]"escal""ate"     -> Transformer -> EOS

answer = "escalate"

Jev does something else. It runs the packed input through the transformer once and reads probabilities directly from its hidden states:

One-shot processing — one forward pass, regardless of question count

                                      ┌─ question 1 -> probabilities
[state][question 1][question 2][...] ─┼─ question 2 -> probabilities
                                      └─ question 3 -> probabilities
                    ↑
          one trip through the transformer
          no answer string is generated

That is the whole premise: no generation loop, just one forward pass and a typed readout.

Working backwards from the public clues

TypeSafe hasn't published Jev's full architecture, but the official material and Archer Hume's investigation provide enough constraints to make a useful hypothesis.

The base is probably an ordinary decoder LLM

“Jev's breadth of knowledge (84.6% on MMLU-Pro) requires frontier-scale pretraining, every model at that scale is a causal decoder, and TypeSafe describes RLCD as post-training a pretrained language model.”

— Archer Hume, Jev's Architecture Unmasked

That's a sensible starting point. Jev's MMLU-Pro score suggests broad pretraining, and TypeSafe describes RLCD as post-training a pretrained model. Hume's argument: a bidirectional encoder would either start from a weaker base or need an expensive conversion. Frontier-scale pretrained models, meanwhile, are all causal decoders.

A causal model also fits this serving pattern: compute a prefix once, then attach separate suffixes.

So my first assumption was boring on purpose: Jev is probably built on a normal causal-decoder LLM, not a mysterious new backbone.

Then it stops before the generation loop

TypeSafe is explicit about this part:

“Jev outputs all probabilities in parallel instead of autoregressively generating by token.”

— TypeSafe, Introducing System One Models and Jev

Hume found a second clue in the API's output_tokens field:

“The API still reports an output_tokens field, which sounds like a record of generation. It isn't one. For yes/no questions, the count fits exactly: 4 shared tokens, plus 15 per answer, plus the token length of each question's identifier.”

— Archer Hume

He also observed that 0.0 and 0.01 cost the same, and that a 200-option Choice has roughly the latency of a 2-option Choice. The field looks like accounting metadata, not a trace of sampled tokens.

So there is no reason to believe Jev is spelling out an answer one token at a time and then parsing it back into a number. The model can stop after one forward pass and a readout.

The mask has to make state public and questions private

Hume's proposed serving shape is concise:

“A prefix KV cache with separate causal suffixes is the natural implementation.”

— Archer Hume

There are two useful pieces of evidence behind that idea.

The first is token accounting. Hume reports a request with a 23,000-token state and 5,000 questions fitting under a 65,536-token request limit. That's difficult to explain if every question reprocesses the full state from scratch.

The second is a small but revealing visibility test:

“With the secret in the sibling question, its reported probability was 0.00. Removing that sibling produced the same result. Putting the declaration in the state instead raised it to 0.90–0.92.”

— Archer Hume

A secret in one question was invisible to a probe question. The same secret in shared state was visible.

That doesn't prove every implementation detail, but it points to one simple structure: questions are isolated from each other; state is shared.

That next step is mine, not Hume's. If I pack several branches into one normal causal sequence, a later question can attend to an earlier question. That violates the behavior above. So I need an additional tree mask:

A token can see:
  - the shared state
  - earlier tokens in its own question branch
  - never another question branch

I also reset RoPE position IDs at the beginning of every branch. Without that reset, inserting an unrelated question shifts the positions of later branches and changes their hidden states even if the attention mask is otherwise correct.

Leave the FFN alone

The attention path needs new visibility rules. The feed-forward layers don't.

So I left Qwen's FFN blocks untouched. Their job is still to transform each token's representation locally; they aren't where state sharing or branch isolation happens.

Choice and Score can share an output head; Noul can't

Hume found evidence that the options in a Choice aren't scored independently:

“mean log-odds fell from +0.38 to +0.11. Every block showed a decrease.”

— Archer Hume, after adding an irrelevant option across ten blocks

Why does that matter? If each option got an independent, unchanged logit and all logits were passed through one softmax, an added option would change the denominator but not the log-odds between two existing options. The denominator cancels.

But Jev's existing-option odds moved. The options are being considered together.

Hume narrowed the readout down to two candidates:

“Two readouts fit the evidence. A final-position head scores each option slot from the decision token's representation; a pointer-style scorer compares that representation with each option's own final hidden state. Both let options influence one another... Neither result is decisive.”

— Archer Hume

I picked the pointer form because it works for a variable number of options. A fixed-slot head needs an output shape sized for a maximum number of candidates. The pointer form doesn't.

For Choice and Score, my output head is:

z_i = (Wq * h_decision) dot (Wk * h_option_i) / sqrt(d)
p(option_i) = softmax(z)_i

<DECISION> provides the query. Each option provides a key. The final softmax gives a probability distribution over the supplied options.

Noul is different. It has no runtime-defined candidates to compare, so I use a plain linear head on <DECISION> followed by sigmoid:

p(yes) = sigmoid(w dot h_decision + b)

The architecture I ended up with

Putting those pieces together, my Jev hypothesis is:

Start with a normal causal-decoder LLM.
Pack state and question branches into one request, then run one forward pass.
Use a tree attention mask: all branches can see state, each branch can see itself, and branches can't see one another.
Reset each branch's RoPE position IDs immediately after state.
Keep the FFN path intact.
Use one pointer output head for Choice and Score; use a separate linear-plus-sigmoid head for Noul.

If that design is broadly right, I should see three Jev-like behaviors:

Adding, removing, or reordering questions should not materially change an existing question's answer.
Reordering otherwise identical options should change output probabilities.
Adding an irrelevant option should change the relative odds between existing options.

A Jev-like architecture reproduction built on Qwen2.5-0.5B

I wanted the whole experiment to run on my MacBook, so I built the reproduction on a small enough backbone: Qwen/Qwen2.5-0.5B.

Real Jev is almost certainly built on something much larger. That's fine for this specific test. The three behaviors above come from the mask, position IDs, and output-head shape, not from how much factual knowledge the backbone has.

I changed the following pieces. The reproduction code is at senna-lang/jev-repro.

Module	Change from stock Qwen
`backbone.py`	Loads Qwen's transformer backbone and removes the original vocabulary LM head
`packer.py`	Adds `<STATE_END>`, `<SEP>`, and `<DECISION>`, then packs state and question branches into one sequence
`tree_mask.py`	Implements the tree mask and per-branch position-ID resets
`readout.py`	Adds the pointer output head for Choice/Score and the linear Noul head
`forward.py`	Calls `backbone(...)` exactly once per request and never calls `generate()`

forward.py wires the other three pieces together.

Did the Jev-like behaviors show up?

Adding or inserting a question did not change another question's answer.

I changed question count and order, then compared the existing question's probabilities. The maximum difference was about 0.0006.

That's not bit-exact. Eager dense attention accumulates small floating-point rounding differences across Qwen's 24 layers. But the difference is small enough to treat unrelated branches as isolated for this experiment.

Reordering identical options changed the output probabilities.

I used four states, two option sets, and all 24 permutations of each four-option set. Before training, the mean probability movement was 0.97. After light training, it was still 0.94.

The more interesting number is the top option: after training, the highest-probability option changed in 17–58% of permutations. This is not just a softmax getting more or less confident. The answer itself moves when the list moves.

That's the same direction as Hume's option-order probe, where reversing options shifted a technical-support probability from roughly 0.84–0.89 to 0.93–0.96.

Adding an irrelevant option changed the odds between the existing options.

I saw the same behavior in the Jev-like reproduction. The direction and magnitude varied by run, but the existing odds moved. A comparison baseline that scores each candidate independently showed exactly zero movement.

The full behavioral checks are in the verification results.

Bonus: training the output head

All three structural checks pass before any training. This was just an extra check: can the new output head learn from an ordinary supervised signal at all?

TypeSafe says Jev is post-trained with RLCD, but the method isn't public. RL training usually updates far more than a small output head, though we don't know exactly what TypeSafe updates. I wasn't trying to reproduce that process.

Instead, I trained only the Choice/Score pointer head (Wq, Wk) with cross-entropy on AG News. Each news article becomes state; a fixed Choice question asks for its topic; World, Sports, Business, and Sci/Tech become the options.

The Qwen backbone stays frozen. The Noul head stays randomly initialized because AG News has no yes/no labels.

Item	Setup
Training target	Pointer output head only (`Wq`, `Wk`)
Training data	500 / 2,000 / 10,000 / 20,000 examples
Evaluation data	500 examples
Epochs	1 for every size
Loss	Cross-entropy
Optimizer	Adam, learning rate `1e-3`

Training examples	Accuracy	ECE
0	0.2260	0.5664
500	0.6200	0.3779
2,000	0.8180	0.1800
10,000	0.8300	0.1706
20,000	0.7720	0.2280

Accuracy and calibration both improved through 10,000 examples, then both got worse at 20,000. I used one epoch, batch size one, and a fixed learning rate, so extra examples also meant more noisy update steps. That's a limitation of this training setup, not a statement about Jev's limits.

At most, this shows that Qwen's frozen representations can support a small classification head for this four-way task. It says nothing about reproducing Jev's general decision-making ability.

I also tried the official Jev examples

For a final boundary check, I retrained the best 10,000-example setup with a fixed seed (accuracy 0.8040, ECE 0.1980) and ran the Choice and Score examples from TypeSafe's documentation through it. Noul was excluded because its head was never trained. The Jev values are documentation examples, not live API results.

Check	Result
Output structure	10 requests, 17 questions, one forward pass per request, 56 probabilities all inside `[0, 1]`
Choice answers	2 of 8 matched Jev; both were chance matches
Score top level	2 of 9 matched Jev; this head always saturated at the highest level, and both matches were cases where Jev did too

The structure behaved as intended. The answers themselves? About what you'd expect.

An AG News-trained output head on a frozen 0.5B model didn't transfer to support-ticket decisions. It was never supposed to.

Full outputs are in the comparison result.

Sources

Note: This article was originally written in Japanese and translated into English with AI.

Fable May Not Be the Best Choice for Some Engineers

Senna — Sun, 05 Jul 2026 00:44:36 +0000

Fable and Opus may not be the most comfortable tools for engineers who learned to code by hand.

I started thinking about this after reading Simon Willison's recent note. His point is simple: with a strong coding agent like Fable, it may be better to let the model exercise its own judgment than to spell out every condition yourself.

Instead of writing detailed rules like "run tests for larger features, but not for small copy changes, except for design changes...," you can simply say: write and run tests where appropriate.

The same applies to cost. Rather than deciding manually which tasks should go to which model, you can ask the agent to choose an appropriate lower-cost model and delegate the work to a subagent.

Manual cars and automatics

This is a rough analogy, but it feels similar to driving a car.

People who enjoy driving often like manual cars. They want to choose the gear themselves. They want to feel the engine speed and have the car respond directly to their intent.

For people who simply want to get somewhere, an automatic is easier.

Software engineers are similar. If you have written code professionally for a long time, you usually have your own way of working. You may want to get the types right first. You may prefer small diffs. You may have a specific sense for how granular tests should be. You may even have an order in which you like to read an unfamiliar codebase. (At least, I hope you do.)

For someone with that kind of style, a highly autonomous model like Fable or Opus can feel a little too automatic.

The stronger the model, the more small instructions get in the way

This is the same structure as management in human organizations.

A junior member needs concrete instructions: read this document from this angle and summarize it in this format.

A senior member can take a rougher assignment: I want to solve this problem, so investigate it, come up with an implementation plan, and move it forward.

Of course this does not mean throwing work over the wall. You still give the goal, constraints, and success conditions. You just don't dictate every step along the way.

I think instructions for Fable and Opus are moving closer to the second case.

Choose coding agents by delegation level, not just capability

The tricky part is that the strongest model is not always the best choice.

Choose your model by delegation level, not by intelligence.

If you already have a strong preference for how development should proceed, steering a mid-tier model like Sonnet yourself may feel better than handing the whole process to a fully autonomous flagship model.

If you want to decide the development process yourself, use a mid-tier model like Sonnet as a tool you steer closely.
If you want to delegate the implementation approach itself, use a highly autonomous model like Opus or Fable.
If your own development style is not fixed yet, it may be easier to delegate broadly to a flagship model.
For maintenance work or small fixes in an existing codebase, a short interaction with a mid-tier model is often more efficient.

The value of Fable and Opus is not only that they write code faster. It is that you can delegate the way development proceeds.

This is also why it pairs well with what people call vibe coding.

By vibe coding, I do not mean throwing vague prompts at a model and hoping for the best. I mean giving the model a goal like "I roughly want this kind of thing" and letting it handle larger decisions: task decomposition, how far to investigate, whether tests should be written, whether to delegate to subagents, and whether the final result is coherent.

If you want to make those decisions yourself, the model does not need to be that autonomous.

To be fair, sometimes the automatic wins

Don't get me wrong: even if you love steering, there are moments when Fable or Opus is simply the right tool. Large refactorings across an unfamiliar codebase. Exploratory work where you don't yet have an opinion about the approach. Prototypes you might throw away tomorrow.

The argument isn't "manual is better." It's that delegation level is a choice — and if you always default to the most autonomous model, you're skipping that choice without thinking about it.

Manual or automatic?

Flagship models like Fable and Opus are not always the best partners for engineers who came up in the handwritten-code era.

If you have your own style and want to keep your hands on the steering wheel, using a mid-tier model like Sonnet as an extension of your hands may feel better. If you want to delegate the development process itself, Fable and Opus start to show their real value.

In short: choose your model by delegation level, not by intelligence.

For someone who likes manual cars, even the best automatic car is not always the most fun. But if the goal is simply to arrive, automatic wins.

Fable and Opus are probably that kind of tool.

How do you choose your coding agent — by intelligence, or by how much you want to delegate? And if you learned to code by hand: are you actually comfortable handing the whole process over?

Note: This article was originally written in Japanese and translated into English with GPT.