Two numbers from two different rooms.
My last fourteen pieces here, read at 72 hours or later: 0, 10, 10, 21, 24, 25, 28, 30, 32, 40, 40, 40, 54, and one too new to count. Median 28 views. One of them has zero.
xulingfeng's post yesterday: 99 reactions and 62 comments inside a day. He also wrote that fifteen technical posts of his drew zero to one reaction each, until he switched to stories about AI failures and got 445 reactions across fifteen of them.
He is right, and it took me a month of measuring to see it. The tags I publish into are not a room. The newest 100 posts in hardware spanned 29 hours, and 66 of them came from one account. The top 100 hardware posts of a whole week carry 14 reactions total, 90 of them at zero. The top 100 in llm carry 295. I had been choosing between empty rooms and calling it aim.
So let me try the other thing: ask a question you can answer from your own desk.
I write about decode throughput. The physics is short. Token rate is bytes moved per token, divided by memory bandwidth. An RTX 4090 moves 1,008 GB/s. An 8B model in FP16 weighs 16.1 GB, so the ceiling at one sequence is 62.8 tokens/s. Real runs land at 70 to 90 percent of that. The missing tenth is batch size, kernel overhead, and whatever spilled to CPU. That gap is the only part that varies between machines, and I cannot compute it for yours. You can.
Reply with four things:
- your card and its memory (3090 24GB, M2 Pro 32GB, A100 40GB, CPU only, whatever it is)
- the model and quantization you actually run
- tokens/s at one sequence, and at whatever concurrency you run
- if you know it, how much context fits before it slows down or OOMs
I will compute the ceiling for each set of numbers and publish the table with your handle credited. If your measurement beats the computed ceiling, send that one too, it is more interesting than a match.
One line is enough. I answer every reply here inside a day.
The ceiling formula, if you want to check my arithmetic before trusting it: tokens per second = memory bandwidth in GB/s divided by model bytes read per token. For a dense model at one sequence that is roughly parameter count times bytes per parameter, plus KV cache traffic which I am ignoring on purpose so the number stays a ceiling.
Top comments (0)