DEV Community

Nerav Doshi
Nerav Doshi

Posted on Originally published at pipelineandprompts.com

Compared Quantization Levels: Q4 vs Q8 vs FP16 on llama3.2:1b

Context: A model's weights — the numbers it uses to reason — are normally stored at high precision, like FP16 (16-bit floating point). Quantization compresses those numbers down to fewer bits (Q8, Q4, etc.) to shrink the file size and memory footprint, trading some numerical precision for big space savings. It's a similar idea to compressing a high-resolution photo into a smaller JPEG: the file shrinks a lot, some detail is technically lost, but for most practical purposes it's hard to tell the difference.

Ran: Before pulling anything new, ran ollama show llama3.2:1b to check what Entry 01's baseline model actually was — and it turned out to already be Q8_0, not an unspecified "default" as I'd assumed back in Entry 01. Worth a correction: Entry 01 and Entry 03's 1b numbers were Q8, not full precision. Pulled two more variants for comparison: llama3.2:1b-instruct-q4_K_M (an explicit 4-bit quantization) and llama3.2:1b-instruct-fp16 (full, uncompressed precision). Loaded each in turn, sent the same test question, and captured ollama ps / ps aux for all three quantization levels.

Result:

Q4_K_M Q8_0 (Entry 01 baseline) FP16
Download size 807 MB 1.3 GB 2.5 GB
ollama ps loaded size 997 MB 1.5 GB 2.7 GB
Process RSS ~0.94 GB ~1.24 GB ~2.57 GB

Memory scales cleanly with bit-width across all three: roughly 0.94 GB → 1.24 GB → 2.57 GB as precision doubles from 4-bit to 8-bit to 16-bit. That's a tidier relationship than I expected going in — quantization level is a solid, predictable lever for memory sizing.

For quality, I asked all three quantization levels the same real question: "what are good free AI tools that can create simple Google Slides using instructions?" None gave a clean answer. All three produced long lists mixing legitimate tools (Canva) with tools that have nothing to do with making slides — Q4 suggested Midjourney and DALL-E, FP16 added Artbreeder, Deep Dream Generator, and Prisma to the same mix. Full precision didn't fix this: FP16's list was just as padded with irrelevant suggestions as Q4's, and Q4 wasn't noticeably worse.

Takeaway: Memory scaled cleanly and predictably with quantization level — roughly doubling at each step from Q4 to Q8 to FP16, matching the bit-width math almost exactly. Answer quality told a different story: going all the way to full precision didn't reduce hallucinated suggestions on this open-ended recommendation question, and Q4 wasn't visibly worse than FP16. That's not proof quantization is "safe" in general — this was one soft question, not a rigorous eval — but it's a reminder that the real quality gap probably shows up on harder tasks (math, code, precise instruction-following), not casual recommendation queries. Also worth remembering going forward: check ollama show <model> before assuming what quantization a "default" pull actually gives you.

$ ollama show llama3.2:1b
  quantization        Q8_0

$ ollama show llama3.2:1b-instruct-q4_K_M
  quantization        Q4_K_M

$ ollama ps
NAME                         ID              SIZE      PROCESSOR    CONTEXT    UNTIL
llama3.2:1b-instruct-fp16    2887c3d03e74    2.7 GB    100% GPU     4096       4 minutes from now

$ ps aux | grep ollama
flyers  10763  0.3  15.7  438101456  2630128  ??  S  llama-server --model ... -c 4096
flyers   2256  0.0   0.3  436826160    56448  ??  S  ollama serve

$ ollama ps
NAME                           ID              SIZE      PROCESSOR    CONTEXT    UNTIL
llama3.2:1b-instruct-q4_K_M    22bc6b92eb01    997 MB    100% GPU     4096       4 minutes from now

$ ps aux | grep ollama
flyers  10786  0.2  5.7  436484688  963808  ??  S  llama-server --model ... -c 4096
flyers   2256  0.0  0.4  436826160   60400  ??  S  ollama serve
Enter fullscreen mode Exit fullscreen mode

Top comments (0)