Have you ever wondered how high-performance LLM deployment frameworks like vLLM, TensorRT-LLM, or Hugging Face TGI actually optimize model serving?...
For further actions, you may consider blocking this person and/or reporting abuse
@xulingfeng The red flag catch you again?
🤣KFC is so good 🍗
Now that was mean 😭
My bad 😂 Next time I'll save you a screenshot of the order receipt so you can at least look at it.
... I'll get you back someday... Wont be today, might not be tomorrow, but I'll get you back for that 1... 😂😭
How you liking the tetris btw?
Nice model of the fragmentation half. The piece the Tetris framing can't easily show, and the part that actually made vLLM a step change, is what paging unlocks beyond packing: copy-on-write sharing of KV pages. Two requests with the same system prompt, or N parallel samples from one prompt, can point at the same physical pages and only fork on write. Contiguous allocation can't express that at all - every sequence owns its own slab. So the paged table isn't only dodging external fragmentation, it's turning the KV cache into shared virtual memory, and prefix reuse is where a lot of the real throughput comes from once production traffic has fat shared system prompts. Would make a fun hard-mode level: two falling pieces allowed to overlap because they're the same prefix.
True, can work, but the problem is controlling the 2 pieces at once. Could set it up for wasd and arrows separately, but for now the series is still there to just teach the basics, I think from 10 on, I'll start covering the more advanced nuances, eg. having a ghost tetris block that fills fields it crosses, so if you're fast, you can use it to clear up older rows. Or system instructions that anchors a row and pushes it to the bottom, so it progressively fills the vram with set sequences that stay persistent. Though that would be re-using the tetris concept, instead of experimenting with something new.