DEV Community: Abhash Chakraborty

I built a remote for the coding agents in my tmux

Abhash Chakraborty — Thu, 15 Oct 2026 19:08:11 +0000

Inyon lets me answer the coding agents running on my machines from my phone. It's free, open source, and as of this week anyone can install it with one line or from npm.

Walking back to press Enter

I run Claude Code and Codex on a laptop and two servers, and each one is signed in to a different account. A long task usually stops at some point to ask something small: can I run the tests, can I edit this file, which of these two approaches. Until I answered, nothing moved.

The agents have their own remote features, but each follows one account at a time, and some only work if you started the agent their way. My agents were already running in tmux, often started over SSH an hour earlier. I didn't want to restart them inside something else. I wanted to answer them where they were.

What Inyon does

Inyon reads every agent session in tmux across my machines and shows each one as a chat: replies, tool calls, diffs, plans and to-dos. When an agent asks a question, the options arrive as buttons. I tap one, and Inyon types it into the pane, exactly as if I'd pressed the keys.

Codex asks before it runs the tests. The answer goes from the overview straight into its tmux pane.

Nothing changes at the keyboard. When I sit back down, the terminal is where I left it, and the agent never knew anything was different. Each machine keeps its own account, because Inyon never signs in to anything.

It works with Claude Code, Codex, Gemini CLI, OpenCode, GitHub Copilot CLI and Antigravity CLI, on Linux, macOS and Windows.

Reading an agent without wrapping it

Most remotes for agents start the agent themselves, so they see everything it does. I chose to read from outside instead, which is harder and more fragile, but it means any pane works, including one started before Inyon was installed.

Claude Code and Codex write their conversations to session files, which give the chat. Questions are harder: agents don't publish them anywhere. Each one draws its prompt on the screen in its own layout, and that layout changes between versions. So Inyon reads the pane's screen with a parser per agent, tested against prompts captured from real sessions. That's the honest weak spot: a new agent release can break a parser until I update it, and three of the six readers are still in beta.

The question on the pane's screen becomes buttons, and the tap is typed back into the pane

Reaching a laptop with no server

I didn't want to run a server for anyone, and a laptop usually sits behind a router with no open ports.

When a computer is set up for the phone, inyon opens a free Cloudflare quick tunnel. It signs the tunnel's current address with its own key and publishes it with pkarr, so the phone can always find it. The phone checks that signature and runs a Noise handshake with the computer's key, which it got from the QR code, and the computer only lets in phones it has paired. The tunnel only ever carries ciphertext. If the machines share a VPN like Tailscale or WireGuard, the Android app tries that address alongside the tunnel and keeps whichever answers first.

Pairing: a signed address, a free tunnel, and a handshake the tunnel can only carry, never read

The cost of having no server is notifications. Only a push service can wake a closed Android app, and I won't run one, so the app keeps a small "Inyon is connected" notification and gets no alerts once it's closed. In a browser, each computer sends standard Web Push itself.

Making it safe to install

Inyon types into the same panes as my agents, so I'm asking people to trust it with a lot. I spent more time than I expected on the install path.

Every release's checksum list is signed with minisign. inyon setup and inyon update refuse a list without a good signature before they download anything, and the one-line installer checks it too when minisign is installed.
The website's download mirror checks the same signature and every file, and publishes nothing if one fails.
On npm, the packages are published only by the release workflow, through npm's trusted publishing. There's no npm token to steal, each version carries a provenance statement, and nothing runs at install time.
Every third-party GitHub Action in the release pipeline is pinned to an exact commit.

None of this makes Inyon bug-free. It does mean the file you install is the one the release built.

What's rough

There's no iOS app yet. The Gemini, OpenCode and Copilot readers are in beta. On Windows, through psmux, the terminal view shows what's on screen without scrollback.

Try it

On each computer:

curl -fsSL https://inyon.abhashchakraborty.tech/get.sh | sh

or npm install -g @abhash-chakraborty/inyon, then inyon setup. Scan the QR code with the Android app from the download page, or run a hub and use a browser. The docs cover the rest, and the code is on GitHub.

The project page has more on how I built it. Next I want an iOS app and fewer beta labels, and I'd like to hear which agents you'd use it with.

Originally published at abhashchakraborty.tech.

Parakeet Redux: is a 178 MB speech model useful in the real world?

Abhash Chakraborty — Tue, 06 Oct 2026 19:08:12 +0000

Moondream recently released Parakeet Redux, a speech-to-text model that fits in 178 MB. It's NVIDIA's Parakeet TDT 0.6B v3, one of the strongest open speech models, with its weights cut down to three values each. The original is 1,256 MB, and the model card says accuracy stays close to it.

A 0.6B-parameter model in roughly the space of a 61M one would change where good speech recognition can run: on an ordinary CPU, offline, inside an app you download in seconds. So I wanted to know what it actually gives up. I tested it against its own original, a standard 8-bit copy of that original, four other open models and three cloud APIs. First on my laptop, then again on a cloud machine with an NVIDIA Tesla T4, to see how it does on a GPU.

The short version: Redux is 7× smaller than its original and was faster on every device I tried: 15× real time on six laptop CPU cores and 85× on the T4, about 15% ahead of the original there. In exchange it made 0.6 to 0.8 more errors per 100 words, mostly on accented speech and dictation. And the small file doesn't mean a small footprint: it used about 1 GB of RAM on a CPU and 2 GB of GPU memory on the T4. English only, measured on 5 and 6 October 2026.

What Parakeet Redux is

Redux doesn't change the model's design. It keeps the architecture of Parakeet TDT 0.6B v3, and its files name that model as the "teacher" it was trained against. What changes is how the weights are stored.

Read from the model's own config.json, ternary.json and weights file.

The input: 16 kHz audio turned into 128 mel bands, then shortened eight times by a subsampling stage, so the model sees one frame for every 80 ms of sound.
The encoder does the listening: 24 Conformer layers, each with self-attention (8 heads), a convolution (kernel 9) and two feed-forward blocks, 1,024 wide. This is where almost all of the weights live.
The decoder is a TDT (token-and-duration transducer): a small prediction network (a two-layer LSTM, 640 wide) and a joint network that emit a token together with how many frames to skip. Skipping ahead is a big part of why Parakeet models are fast.
A voice activity head: a tiny 0.2M-parameter side head that marks speech and silence every 80 ms, so the model can tell where the talking is on its own.

How it got so small

Normally each weight is a 16-bit number. Redux turns the encoder's 604 million weights into ternary values: every weight is −1, 0 or +1, and each group of 128 weights shares one scale. Three possible values need about 1.58 bits, and Redux packs five weights into a single byte (35 = 243 combinations, which fits in 256), so each weight costs 1.6 bits instead of 16. In the released file, 44% of those weights are exactly zero.

That shrinks the encoder to 121 MB. The parts that are small but sensitive stay in 16-bit: the decoder, the joint network, the token embeddings, the subsampling layers and the per-group scales, about 55 MB together. Add a couple of MB of normalization values and you get the 178 MB on disk.

Ternary weights also make the math simpler: multiplying by −1, 0 or +1 is just adding, subtracting or skipping. Moondream's Photon engine is built around that, with ternary matrix kernels for CPUs with AVX2, AVX-512 or AVX-VNNI, and a CUDA path documented for NVIDIA Ampere or newer GPUs. The model is released under CC-BY-4.0.

How I tested it

I wrote the method down before collecting a single result and logged every decision after that. The other nine systems:

Parakeet TDT 0.6B v3 at 16-bit (1,256 MB) and standard 8-bit (740 MB): Redux's original.
Parakeet Unified EN 0.6B, an English-only model that can stream (731 MB, 8-bit).
Whisper large-v3 (1.55B parameters, 1,669 MB) and Whisper large-v3-turbo (0.8B, 886 MB).
Moonshine base, a 61M-parameter model built for edge devices (132 MB).
Three cloud APIs: OpenAI gpt-transcribe, ElevenLabs scribe_v2 and Google's gemini-3.5-transcribe.

About 35 minutes of English audio, cut into 147 pieces, each with a reference transcript:

Real-world: six YouTube clips with human captions: a TED talk, a podcast, an MIT lecture, a news segment, a TEDx talk by an Indian-English speaker, and street interviews.
Benchmark sets: 50 utterances from LibriSpeech, Earnings-22, AMI meetings and VoxPopuli, from the Open ASR Leaderboard test sets.
Accents: 12 speakers with 12 different first languages reading the same passage, from the Speech Accent Archive (CC BY-NC-SA 2.0).
Noise: ten of the clean clips again, with pink noise at 5 dB.
Dictation: me reading four paragraphs (225 words) into the laptop microphone. I didn't read the script word for word, so its reference is the script corrected to what I actually said, using where the models agreed. That can favour the models, so treat dictation numbers as a hint.

Accuracy is word error rate (WER) after the Whisper text normalizer, on the 78 public clips (5,531 words), with 95% confidence intervals from 10,000 bootstrap resamples. 5% WER means about five wrong words in every 100. Speed comes from 20 short benchmark clips (3 minutes of audio): a warm-up, then the median of three timed passes, one clip at a time.

There were two setups, with the same audio, references and scoring:

My laptop: an AMD Ryzen 5 5600H (6 cores, 6 threads used), 15.3 GB of RAM and an NVIDIA GTX 1650 4 GB, on Windows 11. The other open models ran in transcribe-cpp with community GGUF conversions, on the GPU through Vulkan. Redux ran in Photon on the CPU only: its GPU path isn't documented for a card this old, so I didn't use it there.
A cloud T4: a Tesla T4 16 GB with a 2-core Xeon (4 threads) and 32 GB of RAM, on Ubuntu. Here each model ran in the engine people usually deploy it with: Photon for Redux, NVIDIA NeMo for the original Parakeets, faster-whisper for Whisper and Hugging Face transformers for Moonshine. The cloud APIs weren't run again.

Accuracy

Laptop run: 78 public clips, 5,531 words. Redux in green.

On the laptop Redux scored 5.33%, behind its original (4.54%) and Whisper large-v3 (5.04%), and well ahead of Moonshine (7.02%). The top of the table is a five-way tie: ElevenLabs (4.03%), OpenAI (4.09%), Parakeet Unified (4.16%), Whisper turbo (4.34%) and Gemini (4.38%), with confidence intervals that overlap almost completely. With 78 clips, gaps under about a point aren't reliable.

On the T4 the ranking of the open models held: Parakeet Unified 4.25%, Whisper turbo 4.56%, the original Parakeet 4.72%, Whisper large-v3 4.88%, Redux 5.32% on the GPU and 5.50% on the CPU, Moonshine 7.07%.

One result about compression on its own: the 8-bit original scored 4.52% against 4.54% for the 16-bit one. In this sample, going from 16-bit to 8-bit cost nothing measurable. Going from 8-bit to ternary is where a cost shows up.

Weights on disk against WER. The grey band is the three cloud APIs, whose sizes aren't published.

Plotted against size, Redux is the odd one out: everything in its accuracy range is four to nine times bigger. Size in general predicts little here; the 1.67 GB Whisper large-v3 scored worse than three models half its size.

Redux against its own original

Same audio, same architecture. On the laptop the original ran as GGUF; on the T4, in NeMo.

This is the comparison the whole test was built around. On identical clips Redux made +0.80 more errors per 100 words than its original on the laptop (95% interval −0.08 to +1.54) and +0.60 on the T4 (−0.25 to +1.26). The two setups agree on the direction and roughly the size. But both intervals include zero, and both reach past the +0.5 points I'd set in advance as "no meaningful loss", so 78 clips can't say whether the true gap is tiny or about a point and a half.

Where the gap lands is clearer than its size. On clean benchmark clips it was small, +0.2 on the laptop and +0.4 on the T4. On the 12 accented speakers it was about +1.5 on both, and on my dictation +1.8 and +2.7. Its mistakes were phonetic: "flight lines" for "lands", "revenue grow" for "grew". The model card reports a 6.55% average on seven Open ASR Leaderboard sets and no accent breakdown; on that kind of audio the gap here was small too.

Two setups

The T4 numbers come from two runs on the same machine type.

It runs on the T4. Photon documents its CUDA path for Ampere or newer, and the T4 is an older Turing card. It worked anyway, and fast. That's one card, though, not a promise for every Turing GPU.
The server CPU was slower than my laptop. Redux ran at 7.7× on the 2-core Xeon against 15.1× on six laptop cores. The Xeon has AVX-512 but not VNNI, so Photon used the same AVX2 kernel as on the laptop, and its fastest CPU path is still untested here.
Rankings shift with the machine. On the laptop the 8-bit original was faster than the 16-bit one; on the Xeon it was the other way round (4.8× against 6.6×). The machines differ in cores, clocks, engines and operating system, so I can't say which of those caused it.
The text changes a little too. Redux's transcripts were word-for-word identical on 134 of 147 pieces between the two CPUs, and on 127 of 147 between the Xeon's CPU and the T4. Accuracy stayed about the same, but don't expect the exact same output on every device.

Where it holds up, and where it slips

Laptop run. Each column has 147 to 715 words, so read one column as a hint.

Strong: earnings calls (5.5%, level with the best system), the podcast (3.0%), clean audiobooks (1.1%), meetings (8.2%) and noise (1.8%).
Weak: accented speakers (3.6% against 2.2% for its original on the laptop), my dictation, and the street interviews (11.9%, among the worst).
No system wins everywhere. ElevenLabs led on meetings, Whisper turbo on the lecture, Whisper large-v3 on accents and dictation, and Gemini in noise. Whisper large-v3 also lost points on the podcast for a reason that isn't really a mistake: it drops repeated fillers like "you know", which the captions keep.

Every one of the ten systems wrote the name "Ananya" in my dictation as "Aranya". I can't tell whether that's the models or my pronunciation, so it stays in the reference and counts as an error for all of them.

Speed

Seconds of audio per second of compute; median of 3 timed passes.

On the laptop CPU Redux ran at 15.1× real time, about four minutes for an hour of audio. That's 1.4× the 8-bit original in the same Linux container (10.7×) and twice the 16-bit one (7.6×), which makes it the fastest Parakeet on that CPU. It wasn't the fastest model overall: Moonshine ran at 20.3×, with clearly more errors. Whisper was slower than real time on this CPU.

On the T4 Redux was the fastest system I tested, at 85.0× against 73.8× for its original in NeMo and 46.1× for Parakeet Unified. At that rate an hour of audio would take about 42 seconds. That's worked out from short clips run one at a time, not a timed hour-long job.

Latency and memory

One 4 to 25 second clip at a time with the model loaded. Cloud times include the upload from my connection.

For dictation, the wait per sentence matters most. Redux took a median 0.09 s on the T4 and 0.48 s on the laptop CPU, quicker than every cloud API from my connection (1.03 s for ElevenLabs, 1.05 s for OpenAI, 3.03 s for Gemini). Whisper always processes a 30-second window, so even a short sentence took 11 to 14 seconds on the laptop CPU.

Memory is where the 178 MB figure misleads. Redux used about 980 MB of RAM on the laptop and 1.35 GB on the Xeon, not far from the 8-bit original. On the T4 it took about 2 GB of GPU memory, less than its original's 2.9 GB but nothing like the file size. The small download is real; a tiny runtime isn't, at least not with today's engine.

What it costs in real use

Two everyday jobs, with cloud list prices on 5 October 2026.

Two everyday jobs make the trade clearer. Dictating half an hour a day for a year is about 180 hours of speech: $40 with ElevenLabs, $49 with OpenAI and roughly $68 with Gemini at their list prices on 5 October, or no per-hour price with a local model. Transcribing an archive of 1,000 hours costs $220 to $370 in the cloud. With Redux the same archive would take about 66 hours on my laptop's CPU, or about 12 hours on a T4. Check current prices before you rely on these; they change.

OpenAI gpt-transcribe: $0.27 an hour. Per OpenAI's data controls, API audio isn't used for training by default.
ElevenLabs Scribe v2: $0.22 an hour. Audio is retained by default; zero retention is for enterprise plans.
Gemini 3.5 Transcribe: billed per token rather than per minute, which works out to roughly $0.37 an hour by my estimate.
Local models: no per-hour price, and the audio stays on your own machine. That isn't the same as free; I didn't measure hardware, server time or electricity.

Every model at a glance

Accuracy from the laptop run; speed in multiples of real time; the wait is the median on the laptop.

When it's worth using

English transcription on a CPU, on Linux. This is where it's strongest: the fastest Parakeet on both CPUs I tested, from a 178 MB download. Plan for about 1 to 1.4 GB of RAM and a small accuracy cost.
Apps that ship or download the model. 178 MB against 740 MB for the 8-bit original and 1,256 MB for the 16-bit one. I didn't measure the size of the engine itself, which you'd ship too.
GPU batch jobs. On the T4 it was the fastest system here, with somewhat lower accuracy than its original. It worked on this card despite not being documented for it; test yours.
Clean audio in bulk. The gap on benchmark-style clips was small. I didn't test many requests at once, so measure throughput under your own load.

Where to be careful:

Accented speakers, names and dictation. That's where the extra errors were. Test on the people who'll actually use it; 12 accented speakers and one dictating voice is a small sample.
Windows. On my AVX2 laptop the Windows build of Photon had no AVX2 kernel for this CPU, so Redux wouldn't run natively; it ran fine in a Linux container on the same machine. That's one version on one CPU, and may well change.
Small-memory devices. Phones and boards with little RAM weren't tested, and the 1 GB-plus runtime is the thing to check first.
If every word matters and you have the memory, the 8-bit original was more accurate, showed no measurable loss against 16-bit here, and runs in more places today.

What I took away

Ternary weights work. A 0.6B speech model in 178 MB, faster than its original on every device I tried, with a cost you mostly won't notice on clean audio. That's a real step for on-device speech.

Averages hide where compression hurts. The loss was small on benchmark-style audio and larger on accents and dictation, on both setups. If you ship a model like this, test it on your own speakers, not on leaderboard averages.

A small file isn't a small runtime. 178 MB of weights still needed a gigabyte of RAM on a CPU and two of GPU memory. Download size and memory are different budgets.

The engine matters as much as the weights. Most of Redux's practical limits were packaging, not the model: no native Windows kernel for my CPU, GPU support documented only for newer cards, and a runtime that uses most of a gigabyte. The T4 result shows the documented limits aren't the whole story either.

For English, local models have caught up with the cloud. On the laptop the best open models tied three commercial APIs on accuracy, and Redux sits within about a point of them while running offline with no per-hour price. The cloud still offers languages, speaker labels and other features I didn't test.

Limitations

Small samples. 78 public clips, 12 accented speakers who read the same passage, and 225 dictated words from one speaker. The ten noisy clips are copies of clean ones, which the intervals don't account for, so treat them as rough.
English only. The original Parakeet TDT v3 is multilingual; I didn't test other languages.
Engines are mixed in. Redux in Photon against its original in NeMo or GGUF compares two deployments, not compression alone. Start-up times weren't measured under equal conditions, so I haven't compared them, and memory figures are sampled while running, not exact.
Hardware I didn't have. No CPU with VNNI, no Ampere-or-newer GPU, no Apple Silicon or ARM board.
Cloud results are from 5 October 2026, from one network, with that day's models and prices.

What I'd test next

A CPU with VNNI and a GPU Photon officially supports, recording which kernel and precision it actually uses.
Apple Silicon and an ARM board, where small models matter most.
Accents at scale, 100+ speakers, and the other languages Parakeet supports.
Native Windows on an AVX2-only CPU, once a new Photon build is out.

Thanks to Moondream for Parakeet Redux, NVIDIA for Parakeet, OpenAI for Whisper, Useful Sensors for Moonshine, the Open ASR Leaderboard maintainers for the test sets, and Steven Weinberger and George Mason University for the Speech Accent Archive.

Originally published at abhashchakraborty.tech.

Fine-tuning a 0.8B model to beat hosted LLMs at one job

Abhash Chakraborty — Mon, 05 Oct 2026 16:07:46 +0000

I needed a model for one narrow decision. When a new fact about someone comes in, does it replace something already known? "I moved to Berlin" should retire "I live in Pune". "Watched two episodes of The Bear" should leave "works at Infosys" alone.

It sounds trivial, but any assistant with long-term memory makes this call all the time, and both kinds of mistake hurt. Miss an update and it keeps repeating stale facts. Replace too eagerly and it throws away things that are still true.

This post covers the models I compared, how I fine-tuned the best starting point, how I tested it, and what it took to train. The short version: a 0.8B open model, fine-tuned on 6,000 examples on a single T4, beat every hosted LLM I tested at this one job.

Every model on the same held-out tests. For my definition of an update, not a general ranking.

Why a System-One model

The obvious approach is to prompt an LLM and parse what comes back. That works, but it's slow, every decision is an API call, and the answer is free text that you then have to interpret.

TypeSafe's Jev takes a different approach. It's a "System One" model: instead of writing text, it answers typed questions with calibrated probabilities, like a choice from a list or a yes/no. For this task I send it the new fact and up to eight existing ones, and ask two questions about each existing fact:

How does the new fact relate to it: duplicate, updates, elaborates, related or unrelated?
Is the existing fact now outdated?

The score I use is the average of the two: p_update = (P(updates) + P(outdated)) / 2. At 0.70 or above, the old fact is replaced.

The candidates

Jev (hosted, by TypeSafe). Solid, but tuned for general decisions rather than my definition of an update.
Laya, an open 421M ModernBERT model that answers in Jev's format. Fast and light, but off the shelf it replaced too many everyday events.
Kev, Jared Palmer's open-weight model, compatible with Jev. I tried two sizes. Kev-0.8B was the best of everything as released on stated updates. Kev-4B scored lower on that, and on the T4 it ran out of memory on longer requests and took 5.5 seconds per decision.

For reference I also ran a zero-shot NLI model (DeBERTa-v3-large) and several hosted LLMs, each prompted with the same definition of an update. Kev-0.8B was the clear starting point: the best as released, and the only size that both trains and runs on a 16 GB T4.

The data

Each training example is one request: a new fact, up to eight existing facts, and the typed questions, with a label for every question. I built them from two pools:

6,818 logged Jev decisions. Training on these is distillation: Kev learns to make Jev's calls on real requests.
10,219 decisions with ground truth. I simulated 48 lives of a year each. People move city, change jobs, partners and managers, switch gyms, cars, courses, diets and laptops, reschedule appointments, mention crucial facts and make small talk. Because the simulator knows what actually changed, every pair has the right answer: same attribute with a new value is an update, the same value is a duplicate, anything else is related or unrelated. That's 53,870 fact pairs, 2,298 of them real updates.

The 6,000 training examples took every decision that contains a real update first, then filled up from the shuffled mix. Nothing from the test data went in.

Stage 1: supervised fine-tuning

LoRA on Qwen3.5-0.8B-Base, starting from Kev's released weights: rank 16, alpha 32, dropout 0.05, on every linear and linear-attention projection, plus a small decision head. One epoch over the 6,000 examples at a learning rate of 5e-5, with label smoothing of 0.05.

The GPU was a single NVIDIA T4 with 16 GB. It has no bf16, so everything ran in fp32, with batch 1, gradient accumulation of 8 and gradient checkpointing. The requests are short (87 tokens on average, 138 at the 95th percentile), so nothing was truncated. That came to 750 optimizer steps and 5.52M tokens in 6.55 hours, peaking at 5.09 GB of GPU memory.

Stage 2: reward-ranked RL (ReST)

Supervised training teaches the format and the definition, but it doesn't know which mistakes matter more. So I let the stage-1 model make the real decision, replace or keep at the 0.70 threshold, on 2,000 training decisions (10,574 pairs), and scored every pair:

+1 for catching a real update
−1 for missing one
−2 for replacing a fact that was still true
+0.25 for correctly keeping a fact

The mean reward was 0.265. There were 15 wrong replacements and 65 missed updates, spread over 52 decisions with a negative reward. Those 52 were relabelled with the truth (repeated twice when they included a −2) and mixed with replayed stage-1 examples, for 366 records in total. A second, gentler pass at a learning rate of 1e-5 took 46 steps, 0.34M tokens and 23 minutes, peaking at 4.98 GB.

Both stages on one T4. About 12 GPU-hours for the whole project, baseline tests included.

I picked the final checkpoint on six separate validation lives (400 decisions), where it scored 0.9991 AUC against 0.9983 for stage 1. The test sets were never used to choose anything.

How I tested it

Two held-out tests, both kept out of training:

A two-year simulated life with template text: 122 decisions, 428 pairs, 186 updates, 45 of them stated outright.
A live run where an LLM wrote the facts from a three-month simulated life, so they read more like real conversation: 127 decisions, 498 pairs, 104 updates, 36 stated. This is the harder and more realistic one.

A stated update says it directly ("I moved to Berlin"). An implied one doesn't, like a new address in another city with no mention of moving. I report both.

The main metric is ROC AUC. Take one real update and one non-update at random: AUC is the chance the model scores the real update higher. 0.5 is a coin flip and 1.0 is perfect. It isn't accuracy, so 0.954 doesn't mean "right 95% of the time". I also check precision and recall at the 0.70 threshold, because that's where the model actually acts.

Results

The live test. Most of the gain comes from SFT; ReST makes the model more careful about replacing.

On the live test, stated updates went from 0.752 as released to 0.929 after stage 1 and 0.945 after stage 2. For any update, including implied ones, it went from 0.708 to 0.873 and 0.872. Jev scores 0.781 and 0.570 on the same test. On the two-year life the stated score went from 0.952 to 0.962, and for any update from 0.583 to 0.840.

At the 0.70 threshold on the live test, precision was 1.00: it never replaced a fact that was still true. The cost is a recall of 0.14, so it only replaces when it's very sure. Kev and Laya as released never crossed the threshold at all. For this job, a cautious model is the right default, and a lower threshold chosen on validation data recovers more of the misses.

Averaged over both tests on stated updates, with every hosted LLM given the same definition:

my fine-tuned Kev (kev-mem): 0.954
Gemini 3.7 Flash: 0.892
Gemma 4 31B: 0.887
GPT-5.4 mini: 0.873
Kev as released: 0.852
Jev: 0.834
DeBERTa-v3-large NLI, zero-shot: 0.815
Laya: 0.711

Size and cost

The model has about 885M parameters: 873.4M in the base, 10.8M in the LoRA adapter and 0.5M in the decision head, so only 11.3M were trained. The adapter is 43 MB. It runs in about 4 GB of GPU memory and takes 0.81 seconds per decision on the T4, without any of the optimised kernels. Counting the baseline tests and a second stage-1 run, the whole project used about 12 GPU-hours.

Caveats

This is my use case and my definition of an update. Jev and Kev are general-purpose models and the hosted LLMs were only prompted, so a lot of the gain comes from specialising. That's the point, but it isn't a general ranking.
These are single training runs, with no confidence intervals yet.
Jev answered 234 of the 249 test decisions; the rest failed after retries.
The test lives are simulated, apart from the LLM-written text of the live run.

Try it

kev-mem-0.8b on Hugging Face, served through Kev's own server, so any Jev client works with it
kev-mem on Kaggle and the benchmark notebook
The memory-update detection benchmark for hosted LLMs

Thanks to TypeSafe for Jev, Jared Palmer for Kev, the Laya authors for open-sourcing their work (Apache-2.0), and the Qwen team for Qwen3.5.

Originally published at abhashchakraborty.tech.

Building a portfolio that feels fast

Abhash Chakraborty — Mon, 05 Oct 2026 12:05:04 +0000

I started rebuilding my portfolio this summer with a short brief: it should feel like mine, it should open fast, and I should be able to change any of it without a deploy. The first two pull against each other. The third turns what could have been a static site into one with a backend.

This is a write-up of how the pieces fit together, the problems that took longest, and what I settled on. Most of it applies to any content site that wants to be quick without being plain.

The shape of the system

Almost every request is someone reading, so I optimised for reads and accepted a bit more work on writes. Pages are React Server Components rendered on Vercel and cached as static HTML. A visitor's browser never talks to the database. It gets HTML, a few small interactive islands, and images from a separate storage domain.

Content lives in Convex: my profile, projects, posts, likes and the game leaderboard. Images and files live in Cloudflare R2 on their own domain. A private editor, which I call the Studio, writes to both. Product analytics end up in PostHog, but only after passing through the site's own pipeline.

Reads come from cached HTML, writes come from the Studio, and events pass through an outbox before they leave.

A rich design on a light page

The design leans on texture: brushed silver plates, a monochrome portrait that picks up colour on hover, and a holographic glint on project cards. Each of those could easily have cost a library and a few hundred kilobytes.

The rule I followed was simple. If something can be HTML and CSS, it is. JavaScript runs only in small client components that need state: the header, the command palette, the live status strip and the terminal. Each game is its own chunk and loads when you start it. Everything else ships as markup.

The tokens behind the look: eight colours, the holo foil that only appears on hover, one type family and three radii.

The rainbow foil shows how to keep an effect cheap. It's a conic gradient seen through a striped mask, and hovering only moves one pre-drawn layer with a transform, so nothing repaints while the pointer moves. On touch screens, where there's no hover, the same layers play as a slow pulse.

Scrolling was the subtle part. A few sections animate as they scroll into view, which means the browser marks styles dirty on every frame. Early on, the scroll rail read element positions inside its scroll handler, which forced a full layout each frame and made the top of the page stutter. The fix was to measure once on load and on resize, cache the numbers, and only ever animate opacity and transforms so the compositor does the work.

Order matters as much as size. The hero portrait and the status request are preloaded so they download alongside the HTML, and anything optional, like analytics, waits until the browser is idle. On a cold load from India, the first byte arrives in 83 ms and the largest paint lands at about 0.3 seconds.

One cold load in Edge with an empty cache, in milliseconds from the start of navigation.

Editing without redeploying

I wanted to write and fix things from a browser, but I didn't want page views to depend on a database call. The answer is a cache that the editor is allowed to break.

All public reads go through one small content layer. Each query is cached under a shared tag and revalidated hourly. When I save something, the save clears that tag, so the next request renders fresh HTML while every other request keeps coming from cache. If the database is unreachable, the profile and projects fall back to copies that ship with the code, so the site stays up even when the backend doesn't.

What one save does: sanitize once, write once, and invalidate only the cached content that changed.

Posts and project write-ups use a Tiptap editor. It stores two things: the editor's structured document, which is what I edit, and HTML, which is what the site renders. The HTML is sanitized on the server against an allowlist on every save, so the site can render it directly without trusting the editor.

Charts, metrics, flow diagrams and galleries were harder. They're React components, and I didn't want stored HTML to be able to render arbitrary components. In storage each one is an empty placeholder carrying its type and a small JSON payload. When a page renders, the HTML is split at those placeholders, the payload is validated, and the real component goes in its place. A malformed block renders nothing instead of breaking the page.

Previews that never leak

The Studio shows the real page next to the editor, not a lookalike. That uses the framework's draft mode, which skips the cache and lets pages read unpublished content. The risk is obvious: draft mode is carried by the browser, and anything a browser carries can be copied.

So draft mode alone isn't enough. A page only reads drafts when the request also carries a valid editor session, and the public queries never return unpublished posts at all. Previews are also scoped to the tab that asked for one, so opening the site in another tab after an editing session shows exactly what visitors see.

The Site editor: each section's controls on the left, the real page in draft mode on the right.

A terminal that owns the keyboard

The site has a terminal. Pages behave like folders, so cd projects takes you there, and it has six games: Snake, Space Defender, Type Racer, 2048, Minesweeper and a daily 2048 challenge.

The game picker. Each card acts out its game on hover.

The hardest bug came from a browser extension. Some copy-paste "unlocker" extensions stop every key press before the page sees it. In Edge that meant Space scrolled the page and the games ignored the arrow keys. Listening on the window in the capture phase fixed it, because those listeners run before anything attached to the document. Focus needed the same care: games take focus without scrolling, and the terminal never calls scrollIntoView, which moves the page as well as the container.

Scores taught me about leaving. A run used to count only when it ended, so closing the tab mid-game threw it away. Now each game saves its score so far when you exit, switch tabs or close the window. Posting lives in a module that outlives the terminal, keeps a verification token ready while you play, and sends the last score with a keepalive request so it survives the page closing. The server still checks every score against the game's rules.

Analytics without tracking people

I want to know which pages get read and where people give up, not who they are. That rules out most off-the-shelf setups, so the site runs its own small pipeline and only forwards anonymised events.

From a page view to a chart. Nothing that identifies a person is stored along the way.

The browser sends a page view once the page is idle, then the time the page was actually on screen and how far down it was read. Read depth comes from four invisible markers watched by an IntersectionObserver, so nothing measures the layout while you scroll. On the way out, a beacon carries the last update, so closing the tab doesn't lose it.

On the server a visitor is a one-way hash. By default it's built from values that change every day, so it can count visits without following anyone from one day to the next, and the inputs themselves are never stored. Visitors who choose to be remembered get a random id instead, which is what makes returning-visitor numbers possible. Each event is written to the database together with the counters it affects, in one transaction. A scheduled job then delivers it to PostHog and backs off and retries if PostHog is having a bad day.

Counting correctly took more thought than collecting. A reload shouldn't add a reader, so each post keeps one row per reader per day, and later updates only add the difference. The number shown under a post counts each reader once. Shares count when someone actually opens the shared link rather than when a button is pressed, because copying a link isn't sharing it.

What I'd keep

Most of the speed came from deciding what not to do on the hot path: no database on reads, no layout work while scrolling, nothing optional before the first paint. Most of the reliability came from writing things down before acting on them: content into a cache that's cleared on purpose, events into an outbox before they leave.

If I started again I'd keep the same order: settle a few visual rules, make reading fast, then add the fun parts where they don't get in the way of either.

You can browse the projects or open the terminal.

Originally published at abhashchakraborty.tech.