Vllm - DEV Community

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

jamilxt

Oct 22

llm-d vs Plain vLLM: A Practical Kubernetes Inference Quickstart

#llmd #vllm #kubernetes #ai

8 min read

Oct 21

KV Cache Offloading to NVMe: The Break-Even Bandwidth Math and Where It Does Not Pay Off

#ai #machinelearning #vllm #llm

14 min read

Oct 20

Speculative Decoding at High Batch Size: Where the 3x Speedup Turns Into a Tax

#ai #machinelearning #vllm #performance

12 min read

EME GUG

Oct 20

Why LLM Inference Engines Are Fast: Continuous Batching, PagedAttention and Prefix Caching Explained with vLLM

#llm #vllm #inference #performance

7 min read

Alex Chen

Oct 19

Your 128K Context Fits on a 4090 — If You Quantize the KV Cache

#gpu #llm #vllm #ai

3 min read

Oct 18

A 3090 vs an L40S: 1.09x at batch 1, 0.46x at batch 128

#llm #gpu #vllm #benchmark

2 min read

orca forge

Oct 19

Why reasoning_effort behaves differently across providers

#llm #openai #vllm #api

5 min read

munimv2

Oct 13

vLLM's --api-key leaves /invocations, /tokenize and /pause open. Lock it down with an allowlist proxy

#vllm #llm #security #abotwrotethis

3 min read

Oct 12

Model Distillation & Quantization: Running 8B Parameter Models at a Fraction of Cloud Cost

#modeldistillation #quantization #vllm #awq

10 min read

Suresh Kumar Pallapothu

Oct 16

Day 16: Local Model Execution — Running Open-Weight LLMs Securely with Ollama and vLLM

#ai #ollama #vllm #30dayaiseries

4 min read

munimv2

Oct 11

vLLM 0.30 and older: 30+ new advisories, most with no CVE ID, and your scanner can't see them

#vllm #security #selfhosted #abotwrotethis

4 min read

aivrar

Oct 9

Running vLLM natively on Windows

#vllm #windows #cuda #llm

6 min read

Rigorous 14 GiB TPU v5e memory budget constraints

xbill for Google Developer Experts

Oct 13

Small Apache-2.0 Models Against Gemma 4 E2B on One TPU v5e: Which Ones Keep Up?

#tpu #vllm #gemma #llm

10 min read

Oct 6

What Does It Actually Cost to Self-Host an LLM? The Batching Math Nobody Shows You

#llm #vllm #performance #devops

6 min read

Oct 5

Five Ways Your LLM Serving Benchmark Is Lying to You (and How to Catch Each One)

#llm #vllm #performance #devops

15 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.