DEV Community

#cuda

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Can a QAT 12B Fit an 8 GB GPU? Gemma 4 12B on an RTX 3050
xbill profile

Can a QAT 12B Fit an 8 GB GPU? Gemma 4 12B on an RTX 3050

#gemma #llamacpp #cuda #benchmarking
8
Comments
15 min read
Zero-Cloud AI Code Review: Connecting Self-Hosted GitLab to Local CUDA LLM Runtimes
hrlpavan profile

Zero-Cloud AI Code Review: Connecting Self-Hosted GitLab to Local CUDA LLM Runtimes

#gitlab #ai #cuda #selfhosted
Comments
4 min read
Reverse-Engineering NVIDIA: Modifying a CUDA binary
stjepan86 profile

Reverse-Engineering NVIDIA: Modifying a CUDA binary

#nvidia #gpu #cuda #hex
Comments
4 min read
Gemma 4 at Over 70 Tokens/s on a 2021 Laptop's 4 GB GPU: The Live Demo, Step by Step
xbill profile

Gemma 4 at Over 70 Tokens/s on a 2021 Laptop's 4 GB GPU: The Live Demo, Step by Step

#gemma #llamacpp #cuda #huggingface
9
Comments 2
12 min read
Gemma 4 on a Tesla T4, Part 3: Int4 Embeddings Serve E2B in 2.86 GiB at 2.30x bf16
xbill profile

Gemma 4 on a Tesla T4, Part 3: Int4 Embeddings Serve E2B in 2.86 GiB at 2.30x bf16

#gemma #vllm #cuda #machinelearning
9
Comments 2
11 min read
A 4 GB Laptop GPU vs a 6-Core CPU on Gemma 4, Re-Measured in ABBA Order: 4.1x
xbill profile

A 4 GB Laptop GPU vs a 6-Core CPU on Gemma 4, Re-Measured in ABBA Order: 4.1x

#gemma #llamacpp #cuda #benchmarking
10
Comments 2
11 min read
Gemma 4 on a Tesla T4, Part 2: The Minimum GCE VM and a Script to Drive It
xbill profile

Gemma 4 on a Tesla T4, Part 2: The Minimum GCE VM and a Script to Drive It

#gemma #vllm #gcp #cuda
13
Comments
13 min read
Finding a Random Island with Geometry and CUDA
adilaidev profile

Finding a Random Island with Geometry and CUDA

#gpucomputing #geospatial #cuda #geometry
Comments
4 min read
Gemma 4 on a Tesla T4: QAT Weights Decode 1.79x Faster Than bf16
xbill profile

Gemma 4 on a Tesla T4: QAT Weights Decode 1.79x Faster Than bf16

#gemma #vllm #cuda #machinelearning
8
Comments 1
9 min read
Same nvJPEG2000, different numbers: timer boundaries and frames in flight
fastvideo profile

Same nvJPEG2000, different numbers: timer boundaries and frames in flight

#cuda #gpu #performance #cpp
Comments
13 min read
Qwen3.8 27B at 256K: 50 TPS on a 24 GB GPU
pich profile

Qwen3.8 27B at 256K: 50 TPS on a 24 GB GPU

#aiinfrastructure #llm #cuda #performance
1
Comments 1
11 min read
Understanding GPU Memory: VRAM, Bandwidth, and Why Your Model Won't Fit
3ni8ma profile

Understanding GPU Memory: VRAM, Bandwidth, and Why Your Model Won't Fit

#gpu #cuda #vram #memory
1
Comments
2 min read
Gemma 4 on an 2021 4 GB Laptop GPU: QAT Takes It From 9.5 GiB to 1.6
xbill profile

Gemma 4 on an 2021 4 GB Laptop GPU: QAT Takes It From 9.5 GiB to 1.6

#gemma #llamacpp #mcp #cuda
11
Comments 3
13 min read
The sm_120 shared-memory cliff: why FP8 KV cache crashes vLLM on workstation Blackwell
conatusai profile
Jahn

The sm_120 shared-memory cliff: why FP8 KV cache crashes vLLM on workstation Blackwell

#cuda #llm #gpu #performance
Comments
3 min read
g5g vs g6 for LLM Serving: the Same Code, and 3.7x the Throughput
xbill profile

g5g vs g6 for LLM Serving: the Same Code, and 3.7x the Throughput

#aws #jax #cuda #machinelearning
5
Comments 2
6 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.