DEV Community

#llamacpp

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Can a small language model run on a budget Android phone with 4 GB RAM?
neykosa profile

Can a small language model run on a budget Android phone with 4 GB RAM?

#android #llm #llamacpp #machinelearning
Comments
4 min read
111GB 모델 파일이 램 78.8GB로 도는 이유: POCKET GGUF를 비트 수·KV 캐시·토큰당 읽는 가중치로 분해한 램 예산표
vidraft_lab profile

111GB 모델 파일이 램 78.8GB로 도는 이유: POCKET GGUF를 비트 수·KV 캐시·토큰당 읽는 가중치로 분해한 램 예산표

#ai #machinelearning #llm #llamacpp
Comments
5 min read
Jan open source ChatGPT alternative: offline local AI
spanobserver profile

Jan open source ChatGPT alternative: offline local AI

#llamacpp #llm #localai #mcp
Comments
7 min read
How to Use a Remote Ollama or llama.cpp Server From a Desktop AI App on Windows in 2026
alichherawalla profile

How to Use a Remote Ollama or llama.cpp Server From a Desktop AI App on Windows in 2026

#ollama #llamacpp #windows #ai
Comments
6 min read
Six Serving Engines, One 8 GB GPU: Gemma 4 E2B on an RTX 3050
xbill profile

Six Serving Engines, One 8 GB GPU: Gemma 4 E2B on an RTX 3050

#gemma #vllm #llamacpp #nvidia
23
Comments 1
12 min read
Extending llama.cpp to Support Reflection Beam-1.0-501B-A23B: Handling Complex MoE and Attention Architectures
pdrojack profile

Extending llama.cpp to Support Reflection Beam-1.0-501B-A23B: Handling Complex MoE and Attention Architectures

#llamacpp #mixtureofexperts #attention #modelconversion
Comments
3 min read
One Word, Two GPUs: Why Gemma 4 Answers the Same Question Differently on a Laptop and a Tesla T4

Hardware differences subtly alter benchmark grades
xbill profile

One Word, Two GPUs: Why Gemma 4 Answers the Same Question Differently on a Laptop and a Tesla T4

#gemma #llamacpp #cuda #nvidia
28
Comments 7
12 min read
Running MoE LLMs Larger Than Your RAM: A Practical Guide to mmap, Expert Offloading and SSD Streaming with llama.cpp
eme_gug_0821b41b948be6516 profile

Running MoE LLMs Larger Than Your RAM: A Practical Guide to mmap, Expert Offloading and SSD Streaming with llama.cpp

#llm #llamacpp #moe #performance
Comments
8 min read
Four GPUs, One Quantization Ladder: Gemma 4 E2B From a 4 GB Laptop to an NVIDIA L4
xbill profile

Four GPUs, One Quantization Ladder: Gemma 4 E2B From a 4 GB Laptop to an NVIDIA L4

#gemma #llamacpp #quantization #nvidia
13
Comments 2
11 min read
llama-server's prompt cache reuses KV computed under a different LoRA scale
homelabpm profile

llama-server's prompt cache reuses KV computed under a different LoRA scale

#llamacpp #llm #selfhosted #lora
Comments
6 min read
Performance vs Quality: Gemma 4 E2B Quantization on an NVIDIA L4 and a Tesla T4
xbill profile

Performance vs Quality: Gemma 4 E2B Quantization on an NVIDIA L4 and a Tesla T4

#gemma #llamacpp #aws #quantization
Comments
11 min read
Ollama vs. llama.cpp on the Same GGUF: The 15% Speed Gap Was the Defaults, Not the Engine
maruhana profile

Ollama vs. llama.cpp on the Same GGUF: The 15% Speed Gap Was the Defaults, Not the Engine

#ai #llm #ollama #llamacpp
Comments 3
7 min read
Running 100B+ MoE Models on a Single RTX 4090: A Practical Guide to Expert Offloading with llama.cpp
eme_gug_0821b41b948be6516 profile

Running 100B+ MoE Models on a Single RTX 4090: A Practical Guide to Expert Offloading with llama.cpp

#llm #llamacpp #gpu #machinelearning
Comments
8 min read
Running 100B+ MoE LLMs on a Single RTX 4090: A Practical Guide to Expert Offloading with llama.cpp
eme_gug_0821b41b948be6516 profile

Running 100B+ MoE LLMs on a Single RTX 4090: A Practical Guide to Expert Offloading with llama.cpp

#llm #llamacpp #gpu #selfhosted
Comments
7 min read
Accuracy vs Speed: Gemma 4 E2B Quantization Tradeoffs, BF16 to Q2_K

Hacktoberfest Open-Source AI Challenge: Ghost in the Machine Submission 👻🌼
xbill profile

Accuracy vs Speed: Gemma 4 E2B Quantization Tradeoffs, BF16 to Q2_K

#devchallenge #hf26challenge #gemma #llamacpp
11
Comments 10
15 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.