<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Jan</title>
    <description>The latest articles on DEV Community by Jan (@janmusic).</description>
    <link>https://dev.to/janmusic</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4205033%2Fd8156ba6-95d8-461e-b85c-0fe044a096f2.png</url>
      <title>DEV Community: Jan</title>
      <link>https://dev.to/janmusic</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/janmusic"/>
    <language>en</language>
    <item>
      <title>Lower loss, worse stories: small NCMs trained on TinyStories</title>
      <dc:creator>Jan</dc:creator>
      <pubDate>Mon, 02 Nov 2026 21:39:15 +0000</pubDate>
      <link>https://dev.to/janmusic/lower-loss-worse-stories-small-ncms-trained-on-tinystories-6k3</link>
      <guid>https://dev.to/janmusic/lower-loss-worse-stories-small-ncms-trained-on-tinystories-6k3</guid>
      <description>&lt;p&gt;I trained small Next-Concept Modeling (NCM) models from scratch in MLX on TinyStories. They reconstruct held-out stories almost perfectly, but the text they write on their own is a lot worse than text from a plain autoregressive (AR) baseline built from the same transformer blocks. And the part I didn't expect: the training change that made the teacher-forced loss better made the generated text worse. Same result in two generator seeds.&lt;/p&gt;

&lt;p&gt;NCM is the method of &lt;a href="https://arxiv.org/abs/2610.14162" rel="noopener noreferrer"&gt;Kumbong et al.&lt;/a&gt;. My project, &lt;a href="https://github.com/janmus1c/ncm-mlx" rel="noopener noreferrer"&gt;ncm-mlx&lt;/a&gt;, is an independent MLX implementation of it. The models here are my own small training runs, not reproductions of the authors' models. In an &lt;a href="https://dev.to/janmusic/running-released-next-concept-modeling-checkpoints-on-apple-silicon-28mi"&gt;earlier post&lt;/a&gt; I wrote about running their released checkpoints in MLX. Below I only use their TinyStories checkpoint as a reference point.&lt;/p&gt;

&lt;h2&gt;
  
  
  The setup
&lt;/h2&gt;

&lt;p&gt;NCM doesn't write one token at a time. A codec turns a 128-token text window into a stack of discrete codes, from one coarse code at the top down to fine detail at the bottom. A generator then makes a new stack one level at a time, predicting all codes of a level in parallel, and the codec decodes the finished stack back to text. Here the hierarchy has 12 levels. Level 0 is sampled from a stored unigram distribution, then 11 generator forward passes predict the rest.&lt;/p&gt;

&lt;p&gt;What I trained:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;One codec (15.5M saved values), trained once and then frozen.&lt;/li&gt;
&lt;li&gt;Generators with 11.8M values, each trained on 960,000 story windows.&lt;/li&gt;
&lt;li&gt;An AR baseline with the same 6-layer, width-384 block stack (13.8M parameters), trained on the same 960,000 windows.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Data is TinyStories V2 (GPT-4 split) at a pinned revision. Generation is unconditional, so every model writes 128-token story openings with no prompt. GPT-2 large, also pinned, scores how likely the text looks. Each model writes 1,280 outputs (256 for each of five sampling seeds, temperature 1, no top-k or top-p), and the score is the mean perplexity over the five sampling seeds. Lower is better.&lt;/p&gt;

&lt;h2&gt;
  
  
  Reconstruction isn't the problem
&lt;/h2&gt;

&lt;p&gt;The codec gets 99.816% of non-padding tokens right on 4,096 held-out windows. So the weak numbers below come from the generator making up code stacks, not from the codec losing information.&lt;/p&gt;

&lt;h2&gt;
  
  
  The corruption ablation
&lt;/h2&gt;

&lt;p&gt;During generator training, part of the earlier-level codes the generator conditions on gets replaced with random codes. The targets stay clean. The idea is to prepare the generator for sampling time, where it conditions on its own earlier levels, and those can be wrong. I tried three schedules, with the corrupted fraction going down linearly from the first to the last level:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;strong: 0.75 to 0.25&lt;/li&gt;
&lt;li&gt;reduced: 0.30 to 0.05&lt;/li&gt;
&lt;li&gt;none: 0 to 0&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Everything else stayed fixed: the frozen codec, tokenizer, architecture, optimizer and number of training windows. Each schedule was trained twice, with generator seeds 0 and 1, on the same frozen codec.&lt;/p&gt;

&lt;p&gt;Two numbers matter per model. Clean code CE is the generator's cross-entropy when every earlier level comes from the real text (teacher forcing). Generative perplexity is the GPT-2 large score of text the model wrote by itself.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Clean code CE (nats/code), seed 0 / 1&lt;/th&gt;
&lt;th&gt;GPT-2 large perplexity, seed 0 / 1&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;NCM, strong corruption&lt;/td&gt;
&lt;td&gt;2.24875 / 2.24909&lt;/td&gt;
&lt;td&gt;47.897 / 45.568&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;NCM, reduced corruption&lt;/td&gt;
&lt;td&gt;1.66515 / 1.66106&lt;/td&gt;
&lt;td&gt;54.923 / 55.114&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;NCM, no corruption&lt;/td&gt;
&lt;td&gt;1.38995 / 1.38638&lt;/td&gt;
&lt;td&gt;97.776 / 93.833&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;AR baseline&lt;/td&gt;
&lt;td&gt;n/a&lt;/td&gt;
&lt;td&gt;14.676 / 14.311&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Released TinyStories checkpoint (Kumbong et al.)&lt;/td&gt;
&lt;td&gt;different codec&lt;/td&gt;
&lt;td&gt;16.259&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Held-out real stories&lt;/td&gt;
&lt;td&gt;n/a&lt;/td&gt;
&lt;td&gt;8.529&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fo31xlb9yrl89mvnbcfdu.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fo31xlb9yrl89mvnbcfdu.png" alt="Corruption ablation on TinyStories: clean teacher-forced code loss falls as corruption is reduced, while GPT-2 large generative perplexity rises, in both generator seeds" width="799" height="362"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The two columns go in opposite directions. With less corruption, clean code CE improves from about 2.25 to about 1.39 nats per code. Over the same change, generated text gets worse, from 47.9 to 97.8 perplexity in seed 0 and from 45.6 to 93.8 in seed 1. Same ordering in both seeds. The best NCM run is still about three times the AR baseline's perplexity (45.568 vs 14.311 in seed 1).&lt;/p&gt;

&lt;p&gt;To give a feel for what a perplexity near 48 means here, this is the start of the first output of the strong-corruption seed-0 model, not picked for quality:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Once there was a little boy called Jack. Joe was was three years old and discover. One One day, he wanted to go and look for exploring.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The story vocabulary is there, but broken grammar, repeated words and names that change mid-story are typical.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why lower loss can mean worse text
&lt;/h2&gt;

&lt;p&gt;Teacher-forced loss measures how well the generator predicts the next level when everything above it is correct. At sampling time nothing guarantees that. The generator conditions on levels it made itself, and a mistake in an early level gets carried into every later prediction. A model that only ever saw perfect earlier levels during training never learned what to do with its own mistakes. That train/test mismatch is usually called exposure bias, and it's the same problem scheduled sampling was proposed for in autoregressive models.&lt;/p&gt;

&lt;p&gt;The no-corruption model fits that picture: best teacher-forced loss and by far the worst text. Corruption shows the model imperfect context during training, and here more of it helped generation while hurting the loss.&lt;/p&gt;

&lt;p&gt;I think that's the most plausible reading, but these runs don't isolate it. Corruption strength, how it's spread across levels and optimization effects aren't separated, and I didn't measure per-level error with teacher-forced versus self-generated context.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this doesn't show
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;One codec.&lt;/strong&gt; Both generator seeds share the same frozen codec, so the repeat measures generator variability only, not the whole two-stage training.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Two seeds.&lt;/strong&gt; Between seeds, a schedule's perplexity moved by 2.3, 0.2 and 3.9. Enough to see the ordering held, not enough to estimate run-to-run variance.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Unmatched total compute.&lt;/strong&gt; The generator and the AR baseline see the same number of windows and use the same block stack, but NCM's teacher-forced input is 641 positions per window vs 128 for AR, and the codec is extra training and inference work on top. Total compute and total model size aren't matched.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Synthetic data.&lt;/strong&gt; TinyStories V2 is GPT-4-generated children's stories, not naturally occurring text.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Unconditional generation.&lt;/strong&gt; No prompt, continuation or other conditional task was tested.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A proxy scorer.&lt;/strong&gt; GPT-2 large perplexity is a fixed external probability score. It's not a human rating and doesn't directly measure coherence or grammar. Other sampling temperatures and truncation aren't part of these results.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A much bigger reference.&lt;/strong&gt; The released TinyStories checkpoint is far larger (36.4M codec and 88.6M generator values) and was trained by the authors with a much bigger budget. It's a reference point, not something I reproduced.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Checking the evidence
&lt;/h2&gt;

&lt;p&gt;The full write-up is the &lt;a href="https://janmus1c.github.io/ncm-mlx/research/e21-e23-assessment/" rel="noopener noreferrer"&gt;E21 to E23 assessment&lt;/a&gt; and Section 10 of &lt;a href="https://doi.org/10.5281/zenodo.23897923" rel="noopener noreferrer"&gt;working paper v0.12&lt;/a&gt;. The &lt;a href="https://github.com/janmus1c/ncm-mlx/releases/tag/paper-v0.12" rel="noopener noreferrer"&gt;paper-v0.12 release&lt;/a&gt; has two archives next to the paper editions:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;ncm-mlx-text-training-source-d323f4f.tar.gz&lt;/code&gt;: the 44 training-source files exactly as run.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;ncm-mlx-tinystories-evidence-e21-e24.tar.gz&lt;/code&gt;: the audited evaluation reports, all generated samples and the per-output scorer records for all six runs.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;code&gt;SHA256SUMS&lt;/code&gt; covers both. With them you can recompute every perplexity above, read every output or rescore the samples with another model. The text-training code is in the repository too. The evaluation driver that produced the reports isn't included, and I haven't verified a clean-checkout reproduction of the training runs yet.&lt;/p&gt;

&lt;h2&gt;
  
  
  What's next
&lt;/h2&gt;

&lt;p&gt;From the &lt;a href="https://github.com/janmus1c/ncm-mlx#roadmap" rel="noopener noreferrer"&gt;roadmap&lt;/a&gt;, in this order:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Close the NCM/AR gap at small scale first, under the same fixed scorer.&lt;/li&gt;
&lt;li&gt;Prompt-conditioned (prefix) training, and the authors' XSum and WMT14 codecs. Red-fuchs described an MLX port of both codecs and a prefix-conditioned training loop in &lt;a href="https://github.com/janmus1c/ncm-mlx/issues/3" rel="noopener noreferrer"&gt;issue #3&lt;/a&gt;, their code is in &lt;a href="https://github.com/Red-fuchs/ncm-modular" rel="noopener noreferrer"&gt;Red-fuchs/ncm-modular&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;Mixed-precision training throughput, measured before sizing any bigger run.&lt;/li&gt;
&lt;li&gt;A larger natural-text corpus, with its license and splits recorded.&lt;/li&gt;
&lt;li&gt;Bigger models only if NCM gets close to the AR baseline at matched compute.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Nothing so far shows NCM will get there, not sure yet. What these runs did give me is a clear reason to report reconstruction, teacher-forced loss and free-running quality as separate numbers, because here they disagreed.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;AI assistance disclosure: AI coding assistants helped with implementation, experiment analysis and drafting. They are not authors, and this assistance is not independent verification or human review.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>mlx</category>
      <category>machinelearning</category>
      <category>nlp</category>
      <category>research</category>
    </item>
    <item>
      <title>Running released Next-Concept Modeling checkpoints on Apple Silicon</title>
      <dc:creator>Jan</dc:creator>
      <pubDate>Fri, 30 Oct 2026 15:37:24 +0000</pubDate>
      <link>https://dev.to/janmusic/running-released-next-concept-modeling-checkpoints-on-apple-silicon-28mi</link>
      <guid>https://dev.to/janmusic/running-released-next-concept-modeling-checkpoints-on-apple-silicon-28mi</guid>
      <description>&lt;p&gt;You can run HazyResearch's released Next-Concept Modeling checkpoints on the Apple Silicon GPU without installing PyTorch. &lt;a href="https://pypi.org/project/ncm-mlx/0.2.0/" rel="noopener noreferrer"&gt;NCM-MLX 0.2.0&lt;/a&gt; supports unconditional TinyStories generation and the released easy, medium and hard Sudoku models. The project provides converted weights separately on Hugging Face, an independent MLX inference implementation and recorded comparisons with upstream PyTorch. Model weights are not included in the Python package.&lt;/p&gt;

&lt;p&gt;The research method and trained checkpoints are by &lt;a href="https://huggingface.co/papers/2610.14162" rel="noopener noreferrer"&gt;Kumbong et al.&lt;/a&gt;. NCM-MLX independently implements their inference computation; the original training results have not been reproduced here.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try a released checkpoint
&lt;/h2&gt;

&lt;p&gt;In a Python 3.12 environment on Apple Silicon:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;python &lt;span class="nt"&gt;-m&lt;/span&gt; pip &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="s2"&gt;"ncm-mlx==0.2.0"&lt;/span&gt; huggingface-hub
hf download janmusic/ncm-tinystories-mlx &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--revision&lt;/span&gt; be91c549d5336dd64a476ace9c27589a6f21a7fc &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--local-dir&lt;/span&gt; ncm-tinystories-mlx
python &lt;span class="nt"&gt;-m&lt;/span&gt; ncm_mlx official generate &lt;span class="nt"&gt;--task&lt;/span&gt; tinystories &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--weights&lt;/span&gt; ncm-tinystories-mlx &lt;span class="nt"&gt;--n&lt;/span&gt; 2 &lt;span class="nt"&gt;--levels&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;a href="https://huggingface.co/janmusic/ncm-tinystories-mlx" rel="noopener noreferrer"&gt;model card&lt;/a&gt; documents the downloaded files and their provenance. The model stores approximately 500 MB of float32 weights, plus tokenizer and metadata. It uses its own inference path; it is not a Transformers &lt;code&gt;AutoModelForCausalLM&lt;/code&gt; or an &lt;code&gt;mlx_lm&lt;/code&gt; checkpoint.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;--levels&lt;/code&gt; decodes the accumulated latent representation after each level. TinyStories uses a 12-level hierarchy: level 0 is drawn from a stored unigram distribution, then 11 generator forwards predict the remaining levels in parallel. Output is bounded to 128 BPE token positions. This checkpoint generates unconditionally from a start prefix; it does not accept instructions or continue a user prompt.&lt;/p&gt;

&lt;p&gt;For a preview without installation, the &lt;a href="https://huggingface.co/spaces/janmusic/ncm-refinement-demo" rel="noopener noreferrer"&gt;browser demo&lt;/a&gt; replays recorded refinement traces. It is a static replay of actual outputs, not live inference. Intermediate text can be incoherent: it shows partial reconstruction, not the model's reasoning.&lt;/p&gt;

&lt;h2&gt;
  
  
  Converting the checkpoints
&lt;/h2&gt;

&lt;p&gt;The upstream release contains PyTorch checkpoints and pickled tokenizer data. The &lt;a href="https://github.com/janmus1c/ncm-mlx/blob/v0.2.0/ncm_mlx/official/convert.py" rel="noopener noreferrer"&gt;converter&lt;/a&gt; pins checkpoint revision &lt;code&gt;d342ce8aaea58830eef3b8ad473664673e41f692&lt;/code&gt; and checks source-file SHA-256 digests before parsing. An alternative to downloading the converted weights is:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;python &lt;span class="nt"&gt;-m&lt;/span&gt; ncm_mlx official fetch &lt;span class="nt"&gt;--task&lt;/span&gt; tinystories
python &lt;span class="nt"&gt;-m&lt;/span&gt; ncm_mlx official convert &lt;span class="nt"&gt;--task&lt;/span&gt; tinystories
python &lt;span class="nt"&gt;-m&lt;/span&gt; ncm_mlx official generate &lt;span class="nt"&gt;--task&lt;/span&gt; tinystories &lt;span class="nt"&gt;--n&lt;/span&gt; 2
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A small &lt;a href="https://github.com/janmus1c/ncm-mlx/blob/v0.2.0/ncm_mlx/official/torch_reader.py" rel="noopener noreferrer"&gt;tensor reader&lt;/a&gt; reads the zip container, tensor-storage buffers and reconstruction metadata without importing PyTorch. Its restricted unpickler permits supported storage types, tensor reconstruction and an ordered dictionary, and rejects unrecognized pickle globals. The separate BPE reader refuses every global. This is a deliberately narrow reader for pinned files, not a general replacement for arbitrary checkpoint loading.&lt;/p&gt;

&lt;p&gt;Each source tensor must match exactly one conversion rule. The rule either maps it to an expected MLX parameter, transforms its layout, or explicitly identifies state that inference does not read. Ordinary convolution weights change from &lt;code&gt;[out, in, kernel]&lt;/code&gt; to &lt;code&gt;[out, kernel, in]&lt;/code&gt;; learned strided resampling weights use their own transpose-and-reshape rules. A shape-compatible but wrongly ordered tensor could otherwise produce plausible-looking failures.&lt;/p&gt;

&lt;p&gt;The converter checks missing and unexpected parameters and rejects shape mismatches. It also checks the stored level IDs and attention mask before dropping those derived buffers; EMA training state is omitted. The output is safetensors plus an architecture specification, tokenizer JSON and, for TinyStories, the level-0 distribution. This is a file-format and layout conversion, not quantization or additional training.&lt;/p&gt;

&lt;h2&gt;
  
  
  Checking tokenization
&lt;/h2&gt;

&lt;p&gt;TinyStories uses the released 8,192-token byte-pair tokenizer. Replacing it with another tokenizer of similar vocabulary size would change every subsequent input. The independent &lt;a href="https://github.com/janmus1c/ncm-mlx/blob/v0.2.0/ncm_mlx/official/bpe.py" rel="noopener noreferrer"&gt;BPE implementation&lt;/a&gt; reproduces pre-tokenization, byte handling, merges and special tokens.&lt;/p&gt;

&lt;p&gt;Thirteen retained parity cases, including Unicode, have identical pre-tokenization, token IDs and round trips. The checks do not exhaust Unicode inputs; rare whitespace classes remain a documented limitation.&lt;/p&gt;

&lt;h2&gt;
  
  
  Comparing outputs and logits
&lt;/h2&gt;

&lt;p&gt;The &lt;a href="https://github.com/janmus1c/ncm-mlx/blob/v0.2.0/docs/research/official-checkpoint-parity.json" rel="noopener noreferrer"&gt;parity report&lt;/a&gt; compares upstream code revision &lt;code&gt;c70274273f489abc73832b2b98584a8483b6b203&lt;/code&gt; with the MLX port on CPU, using identical inputs. It separates codec behavior, per-level generator logits and completed greedy outputs.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Check&lt;/th&gt;
&lt;th&gt;TinyStories&lt;/th&gt;
&lt;th&gt;Sudoku-easy&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Largest compared fp32 logit difference&lt;/td&gt;
&lt;td&gt;0.100&lt;/td&gt;
&lt;td&gt;0.0000725&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Composed exact-fp64 logit difference&lt;/td&gt;
&lt;td&gt;approximately 3.22e-10&lt;/td&gt;
&lt;td&gt;approximately 1.18e-13&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Greedy fixture outputs identical&lt;/td&gt;
&lt;td&gt;8/8&lt;/td&gt;
&lt;td&gt;64/64&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;TinyStories logits reach magnitudes around 853. Its tied distance head subtracts large squared-codebook terms, amplifying small upstream rounding differences. Upstream PyTorch's own fp32 result differs from the exact-fp64 reference by about 0.071. An absolute &lt;code&gt;1e-4&lt;/code&gt; threshold alone would therefore give a misleading diagnosis.&lt;/p&gt;

&lt;p&gt;The float64 row is a diagnostic computation. Upstream norms and positional tables that otherwise round internally are recomputed in float64. On the MLX side, composed operations replace primitives that do not preserve full float64 accuracy in this test, with selected functions evaluated on the host. Production inference remains float32. This supports agreement of the tested computations; it does not establish production bitwise identity.&lt;/p&gt;

&lt;p&gt;TinyStories codec code agreement is 97.4–100% per level on four input texts. Each first divergence is a near-tie in quantizer distance; changing that code changes later residuals. In the separate generation check, eight greedy fixtures have identical codes and text.&lt;/p&gt;

&lt;p&gt;The broader Sudoku evaluation provides another check: MLX solves 1,988, 1,926 and 1,724 of 2,000 published easy, medium and hard boards, respectively. Upstream PyTorch CPU and MLX produce identical outputs on all 2,000 easy boards. The full medium/hard sets were not also rerun in upstream PyTorch. These are results of the released upstream-trained models, not NCM-MLX's smaller independently trained generators.&lt;/p&gt;

&lt;h2&gt;
  
  
  Measuring latency
&lt;/h2&gt;

&lt;p&gt;On an Apple M5 Pro with 64 GB memory, the recorded eager TinyStories batch-one median is 39.8 ms on Metal versus 280 ms for upstream PyTorch on CPU. Both complete a 128-position sample, including decoder work and a host copy. MLX uses three warmups and twenty timed iterations; PyTorch uses two warmups and ten iterations. Desktop GPU activity was present. The &lt;a href="https://janmus1c.github.io/ncm-mlx/official-checkpoints/" rel="noopener noreferrer"&gt;methodology&lt;/a&gt; records hardware, precision and contention.&lt;/p&gt;

&lt;p&gt;This compares a working GPU port with a CPU reference, not two equally optimized accelerator implementations. The tested PyTorch MPS path rejects required non-divisible area interpolation. The timing demonstrates a useful execution option on that Mac; it does not measure an architecture-only speedup or promise another Mac's latency.&lt;/p&gt;

&lt;p&gt;Later repository work adds prefix caching and opt-in compilation; these options are not part of the pinned 0.2.0 installation above. Under the released attention contract, prefix tokens attend only to themselves and their positions restart consistently. Their per-layer keys and values can be computed once per generation call. The cache belongs to that call, so different requests cannot reuse stale prefix state.&lt;/p&gt;

&lt;p&gt;For Sudoku's 91-token prefix, recorded batch-16 caching improves median latency from 207.8 to 65.4 ms in one busy-desktop comparison. At batch one, caching alone is slower: 23.4 versus 20.5 ms. TinyStories has only a one-token prefix, leaving little reusable work. Compilation is opt-in and incurs first-call tracing costs.&lt;/p&gt;

&lt;h2&gt;
  
  
  Using the port
&lt;/h2&gt;

&lt;p&gt;The released port provides a reference for trying NCM, examining refinement and investigating inference changes on Apple Silicon. The conversion rules and parity fixtures make those changes easier to check against the released computation.&lt;/p&gt;

&lt;p&gt;Separate, preliminary TinyStories training experiments investigate smaller independently trained models. They do not inherit the released checkpoints' quality, and their ongoing comparisons are not a new capability claim here. Neither track establishes general-purpose chat or instruction following.&lt;/p&gt;

&lt;p&gt;The &lt;a href="https://github.com/janmus1c/ncm-mlx" rel="noopener noreferrer"&gt;source and reproduction instructions&lt;/a&gt; and &lt;a href="https://huggingface.co/collections/janmusic/next-concept-modeling-in-mlx-6ae3b295f2ed5265df0906bf" rel="noopener noreferrer"&gt;converted-model collection&lt;/a&gt; provide the next steps. Cite the original paper for NCM and the &lt;a href="https://doi.org/10.5281/zenodo.23781234" rel="noopener noreferrer"&gt;versioned software record&lt;/a&gt; for this implementation. Both codebases use MIT licensing; the converted models retain the upstream license and attribution.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;AI assistance disclosure: OpenAI Codex assisted with implementation, experiment analysis and drafting. It is not an author; this assistance does not constitute independent verification or human review.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>mlx</category>
      <category>opensource</category>
      <category>python</category>
      <category>machinelearning</category>
    </item>
  </channel>
</rss>
