Inference - DEV Community

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Sachin Kr. Rajput

Oct 22

One Request, End to End: Tokenize, Prefill, Decode, Stream

#llm #machinelearning #inference #performance

12 min read

amrit

Oct 21

How DeepInfer Slashed Latency by 2.4 and Cut Costs by 45% – The Future of AI on Edge

#ai #inference #deeplearning

5 min read

TechPulse

Oct 21

DeepSeek Open-Sources DeepInfer, Its Production Inference Engine

#deepseek #ai #inference #opensource

2 min read

jamilxt

Oct 20

DeepInfer vs vLLM: DeepSeek's New Engine Is Not a Drop-In Replacement

#ai #inference #devtools #programming

7 min read

ai maya

Oct 20

Can two Mac Studios really run an 816B model? A bandwidth roofline check on Muse Spark Open

#ai #machinelearning #llm #inference

12 min read

EME GUG

Oct 20

Why LLM Inference Engines Are Fast: Continuous Batching, PagedAttention and Prefix Caching Explained with vLLM

#llm #vllm #inference #performance

7 min read

Reno Lu

Oct 20

Strata runs a 125B model on a 12 GB gaming GPU by leaning on system RAM, and the README spells out the cost

#llm #localai #inference #ai

3 min read

ai maya

Oct 19

Ling-3.1-flash and the 7:1 hybrid: how much KV cache does a 560B model need at 1M tokens?

#ai #machinelearning #llm #inference

10 min read

Mariano Gobea Alcoba

Oct 19

Technical Analysis of DeepSeek V4.1 Flash 1017!

#deepseek #llm #aibenchmarks #inference

5 min read

Hacks.gr

Oct 19

Tenstorrent raises $1,5 billion and doubles its value

#tenstorrent #ai #hardware #inference

1 min read

NIA

Oct 12

The Rounding Error That Stops the Machine

#performance #llm #tokenization #inference

5 min read

RC

Oct 8

AI agents designed the chip that runs their inference

#aihardware #fpga #verification #inference

4 min read

Oct 17

Magnitude: The Apache-2.0 Inference Engine That Claims 2x Faster Than llama.cpp

#ai #tools #inference #opensource

5 min read

Oct 16

I put the 381 tok/s Qwen stack on one RTX 3090, real chat does 118 and the headline mode crashes

#llm #inference #gpu #docker

5 min read

Oct 16

I read a CUDA inference engine's symbols with Ghidra, then made it 12% faster on my 3090

#python #cuda #llm #inference

5 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.