DEV Community

MoonsvnLyn
MoonsvnLyn

Posted on

PSA: if you're on an Intel hybrid CPU, run Strata's calibrate - it nearly tripled my decode speed (IQ3_S at 256K, 16 GB card)

This is the third version. In the third version, I added P-core thread pinning.

log before/after

setup: 5070 ti 16gb, 96gb ram, i7-14700kf, windows. qwen3.8-flash-next iq3_s on strata, 262k context.

first test: ~17 tok/s at 256k. log screenshot attached, before lines are stock settings.

then i changed 3 things: pool workers 13 instead of 19 (e-cores were stalling every verify window on my 14700kf). spec 6 + spec-min-p 0.7, pcie-frac 0. all measured by the built in calibrator, i didn't hand tune anything.

now 256k sits around 55 tok/s warm (prefix cached, thats how agent sessions actually run). cold is 43.

affinity test

if you're on a 12th-14th gen intel cpu just run the calibrator, the defaults were measured on a 6-core ryzen with no e-cores.

full numbers: https://github.com/JiuYue0820/Strata/blob/docs-256k-tuning/docs/TUNING-256K-16GB.md

Top comments (0)