If you self-host Ripple-65B, the first decision is not about benchmarks. It is which file you download, because the model reads that file from GPU memory for every token it writes. A token is a chunk of text, roughly a word.
We ran Ricursive's open coder on our own cloud, on one RTX A6000 (48 GB) and on a real RTX 3090 (24 GB), through llama.cpp, a free program for running models on your own card. The tables and commands live in the full Ripple-65B test on one 48 GB card.
Dense means your ceiling is memory bandwidth, not compute
The models Ricursive compares against are all mixtures of experts. That design wakes only part of the network per token, 23B to 55B parameters. Ripple is dense: all 65B parameters take part in every token.
The file sizes come from quantization, storing each weight in fewer bits: about 130 GB at full precision, 38.8 GB at 4 bits. The card streams most of that file per token. On the A6000 that gave 14.3 to 16.3 tokens per second. The limit is how fast the card moves bytes out of its memory (its bandwidth), not how fast it does math.
What that feels like: a 400-token reply takes close to half a minute, while a short question starts answering in under a second. Fine for one developer steering an agent, not fast.
Long prompts are cheap because of the layer layout
While reading your prompt, a model saves per-token results so it need not redo them. That store, the KV cache, usually grows fast with the prompt. Ripple keeps it small. Of its 80 layers, 64 only look back over a window of 1,024 tokens. Just 16 track the full prompt, costing 64 KiB per token at 16-bit precision, plus the window.
So a 256K-token prompt fit on the same 48 GB card once we also quantized the cache. In llama.cpp, -c sets the context length and -ctk/-ctv set the cache precision:
-c 131072 -ctk q8_0 -ctv q8_0 # 128K, 44.4 GB peak
-c 262144 -ctk q4_0 -ctv q4_0 # 256K, 45.6 GB peak
We hid a passphrase at token 237,532, and the model returned it. Reading that prompt took about 13 minutes. One test per length proves retrieval of a fact, not reasoning across a codebase. The full million-token window does not fit on one 48 GB card.
The trap: your llama.cpp build has to know the architecture
Ripple landed in llama.cpp through PR #31692, merged on 5 November. Nik Bogatyrev submitted it, and AtomicChat's model card says AtomicChat wrote it. AtomicChat also published GGUF files, the single-file format llama.cpp loads, from 129.6 GB down to 18.5 GB. Older builds cannot read the ricursive architecture, so pin a release that can. We used release b12167, which also comes with prebuilt CUDA tarballs.
That matters for automated deployments. Our worker, the small program on the GPU machine that takes orders from our platform, compiles llama.cpp from source on NVIDIA by default, 5 to 20 minutes in earlier tests. With the prebuilt CUDA asset pinned instead, the model was serving 19.0 seconds after the request on the A6000 and 19.3 on the 3090, with the file already downloaded. Changing that default is still open on our side; the SDK example pins the asset for you.
Agents need tool calls: the model asks your code to run something, such as a shell command. With --jinja and --reasoning-format deepseek, the server turned the chat template's tool calls into the standard OpenAI tool_calls field. All of ours were valid.
What a small quant breaks, and why your tests may miss it
On the 3090 we used the 2-bit file (20.0 GB). Fewer bytes per token meant more speed: 23 to 31 tokens per second. Our sandboxed agent job, a toy repo with a planted off-by-one bug, passed with every file size.
That pass hides the problem: the fix was one changed comparison, with no fresh tests to write. Fresh code is where the smaller files went wrong. On the 3090, one of three coding answers had a correct function but a test expecting the wrong result. In a run on the A6000 capped to a 24 GB budget, both tests passed, yet one function never sorted its input. AtomicChat's own measurements agree:
| File | Size | Disagrees with full model on next token |
|---|---|---|
| 4-bit (Q4_K_M) | 38.8 GB | about 7 in 100 |
| 3-bit | 28.0 GB | about 9 in 100 |
| 2-bit | 20.0 GB | almost 20 in 100 |
When you evaluate a quant, make it write new code and new tests, not just turn one red test green. The memory ladder and per-file results for Ripple-65B list each run.
What this does not show
Every request ran alone, a handful of times at most, on prompts we wrote. We did not reproduce Ricursive's benchmarks, their SGLang serving setup or their DFlash2 drafter. Ricursive labels the release a Preview and describes its posttraining as "relatively light". For comparison, our test of Nanbeige4.3-33B, a dense agent model on 24 GB, half the size, wrote about 42 tokens per second on an RTX 4090.
Credit where it belongs: Ricursive Intelligence trained the model and released it under Apache 2.0. Nik Bogatyrev submitted the llama.cpp port. AtomicChat published the GGUF files and their quality table. The llama.cpp maintainers shipped the CUDA builds and the tool-call parsing. AI Search's news video is where we spotted it. Our part: the deployment, the measurements on two cards and the SDK example.
The launch command, timings and step-by-step setup are in the complete Ripple-65B write-up on inferencekey.com.

Top comments (0)