LFM3-1.7B On-Device: The 256K KV Cache Math, Which GGUF to Ship, and the Safety Test to Run First

TL;DR

Liquid AI released the LFM3 family on October 27, 2026. The text model, LFM3-1.7B, is a 1.73B-parameter "tri-hybrid" with a 256K-token context window and a roughly 1 GB 4-bit footprint. We did not run it on a phone. What we did is read the public config.json, the model card and the GGUF repository, and compute what a deployment actually costs. Three things stand out:

• The cache is the cheap part, until it is not. Only 6 of 24 layers use global attention, so fp16 KV costs 12 KiB per token: 0.38 GiB at 32K, 1.50 GiB at 128K, 3.00 GiB at 256K. An all-global design with the same KV heads would need 4x that. At full 256K the cache is still about 3x the size of the 4-bit weights.
• The file you pick matters less than the test you run on it. The GGUF repository ships Q4_K_M (1.10 GB), Q5_K_M, Q6_K, Q8_0 and a "QAD" Q4_0 (1.05 GB) trained to recover Q4_0 accuracy. A new EMNLP 2026 Industry paper finds that quantization can erode chat-format safety, and that the damage depends on the model more than on the bit width.
• That paper did not test LFM3. Its numbers are for Llama, Qwen3 and Gemma. We use it as a test plan, not as a verdict on this model.

What is inside LFM3-1.7B, and why does it matter for memory?

The model card describes 24 layers: 13 gated short convolutions, 5 sliding-window attention layers and 6 global attention layers. The layer_types list in config.json agrees, which we checked by counting it rather than trusting the prose.

Property	Value	Source
Parameters	1.73B	model card
Hidden size / layers	2048 / 24	config.json
Attention heads / KV heads / head dim	16 / 4 / 128	config.json
Sliding window	256 tokens	config.json
Conv kernel width (conv_L_cache)	7	config.json
Max positions	262,144	config.json
Vocabulary, embeddings tied to head	128,000	config.json

Each operator type has a different memory behavior:

• Global attention (6 layers) stores K and V for every token seen. This is the only cache that grows with context.
• Sliding-window attention (5 layers) only needs the last 256 tokens, so its cache stops growing at 256.
• Convolution (13 layers) keeps a fixed state: the last 6 or 7 inputs per channel, which the modeling code handles for both state lengths.

Grouped-query attention (4 KV heads against 16 query heads) cuts the per-layer cost another 4x, so the long-context bill depends on 6 layers rather than 24.

How big is the LFM3-1.7B KV cache at 32K, 128K and 256K?

Per attention layer, one token stores 2 (K and V) x 4 KV heads x 128 head dim = 1,024 elements, or 2,048 bytes in fp16. Six global layers give 12,288 bytes, which is 12 KiB per token. The snippet below does the whole calculation from the live config, so it adapts if the layout changes.

import json, urllib.request

def fetch(u):
    return urllib.request.urlopen(urllib.request.Request(u, headers={"User-Agent": "Mozilla/5.0"}), timeout=60).read()

cfg = json.loads(fetch("https://huggingface.co/LiquidAI/LFM3-1.7B/raw/main/config.json"))
kinds = cfg["layer_types"]
n_full = kinds.count("full_attention")
n_swa = kinds.count("sliding_attention")
n_conv = kinds.count("conv")

kv_elems = 2 * cfg["num_key_value_heads"] * cfg["head_dim"]   # K and V per token per attention layer

def cache_gib(ctx, bytes_per_elem=2):
    full = n_full * kv_elems * ctx * bytes_per_elem                       # grows with context
    swa = n_swa * kv_elems * min(ctx, cfg["sliding_window"]) * bytes_per_elem  # capped at the window
    conv = n_conv * cfg["hidden_size"] * cfg["conv_L_cache"] * 2          # fixed, bf16 state
    return (full + swa + conv) / 2**30

for ctx in (4096, 32768, 131072, 262144):
    print(f"{ctx:>7} tokens: {cache_gib(ctx):.3f} GiB fp16 KV")

We ran it on October 31, 2026. Output:

   4096 tokens: 0.050 GiB fp16 KV
  32768 tokens: 0.378 GiB fp16 KV
 131072 tokens: 1.503 GiB fp16 KV
 262144 tokens: 3.003 GiB fp16 KV

Two cross-checks keep this honest. First, Liquid's release post quotes 1.6 GB of KV cache at 128K context. Our 1.503 GiB is 1.61 GB in decimal units, so the arithmetic agrees. Second, the fixed parts are negligible: the sliding-window layers hold 2.5 MiB in total once the window fills, and the convolution state is at most 364 KiB.

The "hypothetical" bars in the chart are our construction: the same 24 layers, the same 4 KV heads, all global. That is 48 KiB per token and 12 GiB at 256K. It is not a model anyone ships, but it shows what the layer mix buys: a 4x reduction at every context length.

Two caveats on the numbers:

1. They assume a runtime that caps sliding-window layers at 256 tokens. If a runtime allocates full-length cache for those 5 layers too, the cost rises to 22 KiB per token, or 5.5 GiB at 256K. We did not measure any runtime's actual allocation, so treat 3.0 GiB as the lower bound and 5.5 GiB as the pessimistic one.
2. Quantized KV helps. Storing the cache at 8 bits (about 8.5 bits per element in q8_0 blocks) brings 256K down to roughly 1.6 GiB by our arithmetic. We did not test the quality effect, and it is a separate question from weight quantization.

Which GGUF file should you pick for LFM3-1.7B?

The LiquidAI/LFM3-1.7B-GGUF repository lists these files. Sizes come from the Hugging Face API, and bits per parameter is file size divided by 1.73B, so it includes tokenizer metadata and any tensors kept at higher precision.

File	Size (GB)	File bits per parameter
BF16 / F16	3.467	16.03
Q8_0	1.846	8.54
Q6_K	1.428	6.60
Q5_K_M	1.259	5.82
Q4_K_M	1.100	5.08
QAD-Q4_0	1.050	4.85

The "4-bit" files are 5 bits per parameter in practice. K-quants mix precisions across tensors, and with a 128,000-token vocabulary the tied embedding and output matrix is 262M parameters, 15% of the model, so the tensors you do not shrink move the average.

The distinctive entry is QAD-Q4_0. The card says it is a Quantization-Aware Distillation checkpoint "trained to recover the accuracy lost to Q4_0 quantization". That is a different recipe from the post-training K-quants beside it: the weights were adjusted with the quantization in the loop. It is also the smallest file. What we could not find is an accuracy comparison for it. The GGUF card has no figures, and the text we could extract from the model card and release post does not set it against Q4_K_M. Liquid's on-device speed charts say only that models ran as "Q4_0 GGUF files", without saying whether that is the QAD file. So the claim that QAD-Q4_0 is the better 4-bit choice is, for now, the vendor's design intent rather than a measurement you can cite.

A practical default, until you have your own numbers: on memory-tight phones start from QAD-Q4_0 and Q4_K_M and measure both on your task. On laptops with RAM to spare, Q8_0 costs only 0.75 GB more than Q4_K_M. Either way, test the exact file you ship.

How fast can it decode as the context grows?

Decoding on a phone is usually limited by memory bandwidth. Each new token reads the weights once and the global-attention cache once. That gives an upper bound: tokens per second is at most bandwidth divided by bytes read per token.

We assumed 50 GB/s, a round number chosen for illustration. It is not a measurement of any device, and real throughput will be lower.

Tokens in context	Bytes read per token (Q4_K_M, fp16 KV)	Ceiling at 50 GB/s
1K	1.12 GB	44.8 tok/s
32K	1.51 GB	33.2 tok/s
128K	2.71 GB	18.4 tok/s
256K	4.32 GB	11.6 tok/s

The shape is the point. Up to about 32K the weights dominate and the curve is nearly flat. Past 128K the cache is larger than the weights and each added token costs speed. Liquid's model card does publish measured on-device numbers: Q4_0 GGUF files in llama.cpp at 32K context on an AMD Ryzen AI Max+ 395, an Apple M5 Max and an iPhone 17 Pro, where LFM3-1.7B is said to prefill at least 2.5x and decode at least 1.8x faster than every compared model while using the least memory. The absolute tokens per second sit in a chart image that we did not transcribe, so we do not compare our ceiling against them. Our figure is a bound, theirs is a measurement, and a measurement on your own device should replace both.

And the RAM side, with weights plus fp16 cache only (no activations or runtime buffers): 1.07 GiB at 4K, 1.40 GiB at 32K, 2.53 GiB at 128K and 4.03 GiB at 256K. On an 8 GB phone shared with an operating system, 256K at fp16 is plausible but tight, and that is where the q8_0 cache starts to earn its place.

What does a new EMNLP paper say about testing quantized on-device models?

"Quantization Breaks the Format: How Weight Compression Undermines Chat Format Safety in On-Device LLM" (Park, Kim, Lee, OptAI Inc.) appeared in the EMNLP 2026 Industry Track, pages 626 to 640. The setup:

• Nine models, four GGUF levels (Q8_0, Q5_K_M, Q4_K_M, Q2_K), all served through the same llama.cpp binary that Ollama builds on.
• Seven CHATBUG chat-format attacks on 520 AdvBench instructions: a direct baseline, three "format mismatch" variants that delete or swap the role tokens, and three "message overflow" variants that inject text after the assistant turn marker.
• Two metrics: a keyword refusal matcher (ASR-R) and a Llama-Guard-2 judge (ASR-M).

Findings that matter for a 1.7B model:

• Small models degrade most. Llama-3.2-1B goes from 48.5% mean ASR-R at Q8_0 to 75.3% at Q2_K (+26.9 pp). Qwen3-1.7B rises +17.6 pp and Qwen3-4B +13.6 pp. Gemma-3-4B moves only +3.0 pp.
• Model choice beats bit width. Two 4B models differ by more than 4x in drift, so the authors call model family a first-order variable.
• The keyword metric can mislead. For Llama-2 under one mismatch attack, ASR-R jumped +55.8 pp at Q2_K while the judge-based ASR-M barely moved (about -1.5 to -1.7 pp, as the paper reports). The authors trace this to Q2_K output artifacts that defeat keyword matching without changing refusal behavior. They treat ASR-M as the primary release signal.
• It reproduces in Ollama, with caveats. The median gap between their harness and Ollama was 2.7 pp, but one configuration was 19.4 pp lower in Ollama, which the authors attribute mostly to response post-processing in the serving runtime.

Their checklist (Table 4) says to avoid Q2_K for models of 2B or under without task-specific re-evaluation, to run dual-metric testing at Q4_K_M for that size class (Llama-3.2-1B rose +18.6 pp there), and above all to test the exact llama.cpp/GGUF artifact, quantization level, chat template and runtime you ship.

Now the limits, because they decide how far this transfers. The paper covers 1B to 8B open models in llama.cpp/GGUF only. It did not test LFM3 or any Liquid model. It used K-quants, so the QAD Q4_0 build is outside its matrix. LFM3's chat template differs from the models studied (it is ChatML with a <think> generation prefix and reasoning on by default), and the attacks target exactly the template's special tokens. LFM3 may behave better or worse than Llama-3.2-1B. Nobody has published that number yet, and we have not measured it.

What should you test before shipping a quantized LFM3-1.7B?

A short plan that follows the paper's checklist and Liquid's own run instructions:

1. Pin the stack. The GGUF card requires llama.cpp v0.8.0 (b11960) or newer, Ollama v0.40.12 or newer, or LM Studio with llama.cpp engine 2.65.1 or newer. Record the exact versions in your release notes.
2. Use the recommended sampling. For chat, tools and agents the card recommends temperature 0.1, top_k 50, repeat penalty 1.1. For math and science, temperature 1.0, top_p 0.95, top_k 20. A safety test at different sampling than production tests a different system.

   llama-cli -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_M --temp 0.1 --top-k 50 --repeat-penalty 1.1

1. Test the file you ship, once per variant: Q4_K_M, the QAD Q4_0 and, if you offer it, Q8_0 as the reference.
2. Include template attacks, not only plain harmful prompts: strip the role tokens, swap in a different template, and inject a compliant-looking prefix after the assistant marker.
3. Score with two signals. A keyword matcher is a cheap screen, but gate the release on a judge model or human review, as the paper recommends.
4. Re-run after every runtime or quantization change. The paper's Ollama gap shows serving-layer details can move results by double digits.

Also read the model card's own scope note: LFM3-1.7B is built for local agents, tool calling, structured output and domain fine-tuning, and it is "not recommended for agentic coding or complex multi-step reasoning due to its small size". Benchmark scores on the card are Liquid's own, and we did not reproduce any.

What are the limits of this analysis?

• No inference was run. Every cache and file figure is arithmetic over public metadata. Real memory use adds activations, runtime buffers and allocator overhead.
• The decode ceiling rests on an assumed 50 GB/s and ignores compute limits, thermal throttling and operating-system memory pressure.
• The cache figures assume sliding-window layers are capped at 256 tokens. We bracket the alternative at 5.5 GiB for 256K.
• Ginigen has no affiliation with Liquid AI or the paper's authors, and this is not an evaluation of either.

FAQ

How much memory does LFM3-1.7B need for 256K context?
By our arithmetic, about 1.02 GiB for Q4_K_M weights plus about 3.0 GiB of fp16 KV cache, so roughly 4 GiB before runtime buffers. If the runtime does not cap the sliding-window layers, the cache can reach 5.5 GiB. A q8_0 cache halves the KV part to about 1.6 GiB.

Why is the LFM3 KV cache so small?
Only 6 of its 24 layers use global attention, the 5 sliding-window layers are capped at 256 tokens, and the 13 convolution layers keep a fixed state. With 4 KV heads, that is 12 KiB per token in fp16.

Which LFM3-1.7B GGUF is best on a phone?
The smallest are QAD-Q4_0 (1.05 GB) and Q4_K_M (1.10 GB). The card says QAD-Q4_0 was trained to recover accuracy lost to Q4_0, but we found no published numbers comparing the two, so measure both on your task and safety tests.

Does 4-bit quantization make small models less safe?
It can, depending on the model. In a 2026 EMNLP Industry paper, Llama-3.2-1B showed 75.3% keyword-measured attack success at Q2_K versus 48.5% at Q8_0, while Gemma-3-4B was nearly unchanged. LFM3 was not in that study, so test your own build.

What llama.cpp version runs LFM3-1.7B?
The model card lists llama.cpp v0.8.0 (b11960) or newer, Ollama v0.40.12 or newer, and LM Studio with llama.cpp engine 2.65.1 or newer.

Further reading

• LFM2.5-24B-A2B on Your Own Hardware: The RAM, KV Cache and Bandwidth Budget
• 0.1 Bits Per Weight Is Not 0.1 Bits Per Weight: Where the Bytes Go in Sub-2-Bit LLMs

Sources: LiquidAI/LFM3-1.7B model card and config, LiquidAI/LFM3-1.7B-GGUF, Liquid AI, Introducing LFM3, Park, Kim, Lee, Quantization Breaks the Format (EMNLP 2026 Industry).