DEV Community

#llamacpp

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Gemma 4 at Over 70 Tokens/s on a 2021 Laptop's 4 GB GPU: The Live Demo, Step by Step
xbill profile

Gemma 4 at Over 70 Tokens/s on a 2021 Laptop's 4 GB GPU: The Live Demo, Step by Step

#gemma #llamacpp #cuda #huggingface
9
Comments
12 min read
Inside My llama.cpp Setup: Tuning Qwen 3.8 27B for 512K Context
dmitryame profile

Inside My llama.cpp Setup: Tuning Qwen 3.8 27B for 512K Context

#ai #llm #localllm #llamacpp
Comments
7 min read
llama-server's sleep mode loses or crashes on a request that arrives just before it sleeps
homelabpm profile

llama-server's sleep mode loses or crashes on a request that arrives just before it sleeps

#llm #llamacpp #selfhosted #debugging
1
Comments 1
7 min read
When Chat Templates Go Wrong
pielounw profile

When Chat Templates Go Wrong

#llm #opensource #ai #llamacpp
Comments
7 min read
llama.cpp vs Ollama in 2026: Which Runtime Should You Run?
rosgluk profile
Rost

llama.cpp vs Ollama in 2026: Which Runtime Should You Run?

#llamacpp #ollama #llm #selfhosting
Comments
18 min read
A 4 GB Laptop GPU vs a 6-Core CPU on Gemma 4, Re-Measured in ABBA Order: 4.1x
xbill profile

A 4 GB Laptop GPU vs a 6-Core CPU on Gemma 4, Re-Measured in ABBA Order: 4.1x

#gemma #llamacpp #cuda #benchmarking
10
Comments 2
11 min read
ROCm vs Vulkan for AMD Local LLM Hosting: 2026 Guide
rosgluk profile
Rost

ROCm vs Vulkan for AMD Local LLM Hosting: 2026 Guide

#llm #selfhosting #llamacpp #ollama
Comments 2
20 min read
KV Cache on 16 GB GPUs: Making Long Context Actually Fit
rosgluk profile
Rost

KV Cache on 16 GB GPUs: Making Long Context Actually Fit

#llm #llamacpp #vllm #ollama
Comments 1
22 min read
How to pick --n-cpu-moe in llama.cpp: Qwen3.6 35B-A3B on 12, 16 and 24 GB GPUs
donald_lee_707b59c456e304 profile

How to pick --n-cpu-moe in llama.cpp: Qwen3.6 35B-A3B on 12, 16 and 24 GB GPUs

#llm #llamacpp #localai #gpu
1
Comments 2
3 min read
Running llama.cpp on a 32 GB MacBook Air: A Direct Comparison with Ollama
vishnuhd profile

Running llama.cpp on a 32 GB MacBook Air: A Direct Comparison with Ollama

#ai #localllm #llamacpp #ollama
Comments 1
9 min read
Fix Local LLM Quality: Context Stacking & Rope Freq Tweaks
umair24171 profile

Fix Local LLM Quality: Context Stacking & Rope Freq Tweaks

#localllms #aiagents #ollama #llamacpp
Comments
8 min read
Ollama vs LM Studio 2026: Ollama Wins for Devs
shaam_ai profile

Ollama vs LM Studio 2026: Ollama Wins for Devs

#ollama #lmstudio #localllm #llamacpp
Comments 1
7 min read
"V cache quantization requires flash_attn" — the llama.cpp error that quietly halves your context window
dreamdeck profile

"V cache quantization requires flash_attn" — the llama.cpp error that quietly halves your context window

#llamacpp #llm #performance #debugging
Comments
10 min read
llama.cpp vs Ollama: Which Should You Run in 2026?
mrsaynothing profile

llama.cpp vs Ollama: Which Should You Run in 2026?

#ai #llamacpp #ollama #localllm
Comments 1
6 min read
How to Run GGUF Models Locally: Ollama, llama.cpp & vLLM
mrsaynothing profile

How to Run GGUF Models Locally: Ollama, llama.cpp & vLLM

#ai #gguf #ollama #llamacpp
Comments 1
5 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.