DEV Community: Kasper

284B model on one graphics card, because most of it is asleep

Kasper — Sat, 08 Aug 2026 11:50:16 +0000

A mixture-of-experts model only wakes a fraction of itself per token. Six experts out of 256, in each of 43 layers. So the question stops being how to fit 96 GiB into 32 GiB of VRAM and becomes how fast you can read the missing pieces off a drive.

Crow keeps the always-active part on the card, holds the 64 most useful experts per layer beside it, and streams whatever is missing while the GPU works. Peak host memory is 1.28 GiB. Decode is around 12 tok/s at a 200k context.

The interesting part is not the throughput. It is that 79% of a request is the decode thread waiting on the drive, so this is a latency problem wearing a bandwidth costume, and everything that helped acts on latency.

Model: DeepSeek V4 Flash
Full details on Git: Crow on Github

Giving Claude a brain

Kasper — Sat, 01 Aug 2026 18:05:12 +0000

I use Claude across a bunch of projects. Every one of them needs some place for Claude to write down stuff that isn't in the code. A decision I made three weeks ago, why a bug actually happened, a workflow nobody ever wrote down. I kept solving that by hand, per project, and after a while none of my setups looked the same and I trusted none of them fully.

So I built claude-obsidian-vault-kit. It's not a plugin, it's basically a setup contract disguised as a Markdown file. You hand it to Claude, answer a few questions, and it builds you a proper Obsidian vault: folders, an index, and some tools that check everything stays sane.

How it works

Download claude-obsidian-vault-kit.md, one file, that's it.
Drop it into a Claude conversation.
Say "set this up for me."

Claude then just asks you stuff first, like whether you already use Obsidian, if you want a throwaway test vault to poke around in before it's real, which projects you have, how you want to back things up. Only after that does it actually write anything to your disk.

What you end up with is completely empty. No fake example notes, nothing invented, just the folder structure, three small tools, and a few pages explaining how it all works.

The part I actually built this for

Every note gets a small header at the top (title, summary, project, topic, date) and the tool that builds your index only reads that header. It never opens the note itself. That's all: your index can't get cluttered with random text because there's no code path that lets it happen. You never edit the index by hand either, you edit the note and rerun the tool, and if a note's header is broken or missing, it doesn't just get quietly skipped. It tells you, loudly.

There's also a /vaultkit command that gets installed for you and running it rebuilds the index and checks for broken links, duplicate notes, and stale scheduled stuff. All in one go.

Where it stands right now

I've tested pretty thoroughly on Windows with PowerShell and Git Bash. macOS hasn't been tried ones (I don't a Macbook), and Linux was only tested once on an older version of the kit, so treat both as "probably fine, not promised." If you try either, opening an issue with what happened would genuinely help.

Try it

git clone https://github.com/nibor1896/claude-obsidian-vault-kit
or just grab the claude-obsidian-vault-kit.md file straight from the repo and drop it into a chat with Claude.

Result

F5-TTS in .NET, v0.3.0: streaming, numbers that read right, and the feature I measured away

Kasper — Sun, 19 Jul 2026 09:46:50 +0000

Horus.F5Tts.Onnx
Here's what landed in v0.3.0:

Streaming: hear the first sentence sooner

For a paragraph, SynthesizeLongAsync gives you nothing until every sentence is done. The wait that actually hurts in an interactive app is time-to-first-audio — and that's the one v0.3.0 fixes.

await foreach (var chunk in model.SynthesizeStreamAsync(reference, refText, paragraph))
    player.Write(chunk.Samples);   // sentence chunk.Index + 1 of chunk.Count

The bit I care about: it's not a second rendering. The chunks are the same sentence-level pieces, cross-faded the same way, so concatenating every streamed chunk is byte-for-byte identical to SynthesizeLongAsync for the same inputs and seed. I proved that twice — as pure cross-fade math with no models, and end-to-end against the model — and then confirmed it live: the streamed and non-streamed WAVs share one SHA-256.

It's chunk-granularity, not frame-level (F5 renders each chunk as a whole), so the win is the first chunk arriving early, and it grows with text length.

Text normalizers: numbers that read right

Checkpoints are trained on normalized text. Feed the model a raw 50 %, 1.000 €, z.B., 3.8.2026 or 14:30 and it swallows or mumbles them — they're out-of-distribution. v0.3.0 ships ready normalizers for German and English:

options.TextNormalizer = GermanTextNormalizer.Normalize;
// "am 3.8.2026 um 14:30 Uhr" -> "am dritten August zweitausendsechsundzwanzig um vierzehn Uhr dreißig"

options.TextNormalizer = EnglishTextNormalizer.Normalize;
// "I saved $1,000 (50%) by the 3rd" -> "I saved one thousand dollars (fifty percent) by the third"

They cover cardinal numbers, percent, currency with cents, decimals and thousands (the ,/. convention is swapped between the two languages), dates, clock times, ordinals (German inflects from the leading word — am → -ten, der → -te, none → -ter), abbreviations and a few symbols. Only recognized patterns are touched; prose — and the name "Max." — is left alone.

The difference is audible. The same German sentence ran 5.2 s raw vs 8.3 s normalized — the extra seconds are the numbers actually being spoken instead of skipped.

And where a reading would be a guess, I left it alone on purpose: German bare ordinals at a sentence boundary stay cardinals, and English numeric dates (3/8/2026 — is that March 8 or 3 August?) are untouched. A wrong reading is worse than a plain one.

PreparedVoice — and the feature I measured away

When one voice speaks many lines, you can now bind the reference once:

var voice = model.PrepareVoiceFromWav("reference.wav", refText);
var a = await voice.SynthesizeAsync("First line.");
var b = await voice.SynthesizeLongAsync(wholeParagraph);

I originally wanted this as a performance feature — cache the reference processing across calls. Then I measured one synthesis (NFE 32):

phase	time	share
preprocess (the only reference-dependent step)	50 ms	0.3 %
transformer denoising loop (31 steps)	16,156 ms	99.5 %
decode	36 ms	0.2 %

Caching the reference would save ~0.3 % — and it can't even be cached, because F5 fuses the reference into the same graph pass as the generated text. So PreparedVoice ships as ergonomics only, and its docs say exactly that. The genuine latency win this release is streaming (time-to-first-audio), not throughput.

That's the rule I keep for this library: measure before you build, and don't sell a number you don't have.

Also in 0.3.0

IProgress<F5TtsProgress> progress reporting that spans the whole request, and 238 tests (the streaming guarantee and ~124 normalizer cases included).

📦 NuGet: dotnet add package Horus.F5Tts.Onnx
📝 Changelog
🤗 Models: German · English

Feedback, issues and PRs welcome. 🙌

v0.2.0 - Horus.F5Tts.Onnx - MASSIVE improvements

Kasper — Thu, 16 Jul 2026 14:26:20 +0000

HORUS.F5TTS.ONNX
HORUS.F5TTS.ONNX - NuGet

v0.2.0 ships FP16 → 10× faster on GPU.
60 ms per denoising step, down from 617. Measured, not vibes.

Plus: text of any length, real cancellation, resampling built in.

MIT · 90 tests
dotnet add package Horus.F5Tts.Onnx

Shipping the first .NET F5-TTS library — and the ONNX bug I had to fix first

Kasper — Mon, 13 Jul 2026 17:24:49 +0000

F5-TTS is one of the better open text-to-speech and voice-cloning models. There's just one catch if you live in the .NET world: running it means Python and PyTorch. No native story, no NuGet package, nothing you can drop into a desktop app without shipping an interpreter alongside it.

I wanted German (and any-language) TTS inside a native Windows app — so I set out to run F5-TTS on ONNX Runtime, which .NET already has first-class bindings for. This is the story of the bug that stood in the way, and the library that came out of fixing it.

The garbled-German problem

DakeQQ/F5-TTS-ONNX is an excellent project that exports F5-TTS to three ONNX graphs (preprocess → transformer → decode). It works great — for the default (v1) base checkpoint.

Point it at a non-English community fine-tune, though, and the output is fluent-but-wrong speech: the right voice, the right language, complete nonsense words. A classic sign that the text conditioning is broken while the acoustic model is fine.

Root cause: v0 vs v1

Many non-English fine-tunes (like hvoss-techfak/F5-TTS-German) are built on the older F5TTS_Base (v0) architecture, which differs from v1 in two config flags:

param	v1 (default)	v0
`pe_attn_head`	`None` (RoPE on all heads)	`1` (RoPE on the first head only)
`text_mask_padding`	`True`	`False`

The exporter's model reimplementation was hard-wired for v1: it applied RoPE to every attention head and always zeroed out padded text positions. On a v0 checkpoint, both corrupt the conditioning — and you get word-salad.

The fix was small and I contributed it upstream in PR #74 (merged): honor pe_attn_head in the attention processor, and honor mask_padding in the text embedding.

Wrapping it in a .NET API

With correct ONNX models in hand, the wrapper is tiny — all the heavy signal processing (STFT, the diffusion transformer, the vocoder) lives inside the graphs. The library just marshals tensors and runs the NFE loop:

using Horus.F5Tts.Onnx;

using var model = F5TtsModel.Load(
    "F5_Preprocess.onnx", "F5_Transformer.onnx", "F5_Decode.onnx", "vocab.txt",
    configureSession: o => o.AppendExecutionProvider_DML(0)); // CPU / DirectML / CUDA

var (reference, _) = WavAudio.ReadPcm16("voice.wav"); // 24 kHz mono
var result = model.Synthesize(reference, "Transcript of the clip.", "Hello from .NET!");
File.WriteAllBytes("out.wav", result.ToWav());

Design choices worth calling out:

No forced runtime. The package only references the ONNX Runtime managed API; the consumer adds the CPU / DirectML / CUDA native package and picks the provider via a session hook.
Character-level tokenizer. For Latin-script languages you don't need jieba/pinyin at all — a plain char→vocab-index mapping is enough (verified end-to-end).
Dependency-free WAV helpers, so the surface stays tiny.

Verifying it actually works

Compiling isn't shipping. I ran the whole pipeline against real models and transcribed the output with faster-whisper large-v3: correct language, exact transcript, confidence 1.00. (The smoke test caught a real runtime bug too — the NFE loop ran one step too many and overran the transformer's time-step table. CI + a smoke test earn their keep.)

Shipping it

NuGet: dotnet add package Horus.F5Tts.Onnx
GitHub: nibor1896/Horus.F5Tts.Onnx (MIT)
Model: nibor1896/F5-TTS-German-ONNX (CC-BY-NC-4.0)
Site: nibor1896.github.io/Horus.F5Tts.Onnx

As far as I can tell, it's the first library that runs F5-TTS natively from .NET. If you're building something that needs local, offline, natural-sounding TTS on Windows/.NET, give it a spin — and open an issue if you hit anything.