DEV Community

#cuda

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Size Your Model to Where You Run It: Gemma 4 E2B and E4B on an AWS L4 and Two Small GPUs

Budget GPUs achieve surprisingly high performance
xbill profile

Size Your Model to Where You Run It: Gemma 4 E2B and E4B on an AWS L4 and Two Small GPUs

#gemma #llamacpp #aws #cuda
3
Comments 2
10 min read
How Fast Can a 421M-Parameter Decision Model Run? I Benchmarked Laya Across NVIDIA GPUs
cookies_c9dc8b91f33d29250 profile

How Fast Can a 421M-Parameter Decision Model Run? I Benchmarked Laya Across NVIDIA GPUs

#machinelearning #cuda #performance #opensource
Comments 1
6 min read
Can a 26B MoE Run on an 8 GB GPU? Gemma 4 26B A4B on an RTX 3050

RAM bottlenecks and speeds inside
xbill profile

Can a 26B MoE Run on an 8 GB GPU? Gemma 4 26B A4B on an RTX 3050

#gemma #llamacpp #cuda #benchmarking
12
Comments 11
11 min read
I read a CUDA inference engine's symbols with Ghidra, then made it 12% faster on my 3090
prodbymarcu profile

I read a CUDA inference engine's symbols with Ghidra, then made it 12% faster on my 3090

#python #cuda #llm #inference
Comments
5 min read
How long until a rented GPU runs CUDA? Launch to first matmul on six GPU types, timed
runcrate profile

How long until a rented GPU runs CUDA? Launch to first matmul on six GPU types, timed

#gpu #cuda #pytorch #devops
Comments
5 min read
How Much Context Fits Beside a 12B on an 8 GB GPU? Gemma 4 12B's Context Cache on an RTX 3050
xbill profile

How Much Context Fits Beside a 12B on an 8 GB GPU? Gemma 4 12B's Context Cache on an RTX 3050

#gemma #llamacpp #cuda #benchmarking
7
Comments
9 min read
Qwen and Gemma With Thinking On: Thinking Pays Qwen on Tool Calls, Gemma Still Leads
xbill profile

Qwen and Gemma With Thinking On: Thinking Pays Qwen on Tool Calls, Gemma Still Leads

#gemma #qwen #llamacpp #cuda
6
Comments 1
9 min read
A Dense 31B on an 8 GB GPU: Gemma 4 31B on an RTX 3050
xbill profile

A Dense 31B on an 8 GB GPU: Gemma 4 31B on an RTX 3050

#gemma #llamacpp #cuda #benchmarking
10
Comments
10 min read
Gemma 4 on a Used 8 GB RTX 3050: 116 Tokens/s for One User, 618 for 64

Uses a 2014 desktop and custom MCP tools
xbill profile

Gemma 4 on a Used 8 GB RTX 3050: 116 Tokens/s for One User, 618 for 64

#gemma #llamacpp #cuda #localllm
13
Comments 6
13 min read
Exploring Result Visibility of Fixed-Latency Instructions on the SM120 Architecture
comradepenguin1917 profile

Exploring Result Visibility of Fixed-Latency Instructions on the SM120 Architecture

#cuda #nvidia #gpu
1
Comments
10 min read
Nvidia Just Let Rust Into CUDA. Here's Why That's a Bigger Deal Than It Sounds
ashraf_chowdury09 profile

Nvidia Just Let Rust Into CUDA. Here's Why That's a Bigger Deal Than It Sounds

#rust #ai #cuda #programming
1
Comments
5 min read
Do Drafters Work on Small Models? Gemma 4 E2B and E4B on Two Small GPUs

Acceptance rates hold steady across hardware
xbill profile

Do Drafters Work on Small Models? Gemma 4 E2B and E4B on Two Small GPUs

#gemma #llamacpp #cuda #benchmarking
10
Comments 3
19 min read
Comparing Qwen and Gemma on the Low End: Qwen Is Faster Without Drafting, Gemma Has Better Quality
xbill profile

Comparing Qwen and Gemma on the Low End: Qwen Is Faster Without Drafting, Gemma Has Better Quality

#gemma #qwen #llamacpp #cuda
15
Comments
13 min read
How vLLM CUDA Kernels Write and Read the Paged KV Cache
yuan_lei_e631e36865ed370b profile

How vLLM CUDA Kernels Write and Read the Paged KV Cache

#vllm #cuda #llm #ai
Comments 2
5 min read
CUDA Rust: two native tracks, not a wrapper over C++
jtorchia profile

CUDA Rust: two native tracks, not a wrapper over C++

#english #rust #cuda #gpuprogramming
1
Comments
5 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.