VIDRAFT's vidraft-darwin Hits 510.58 TPS on The Fast Gemma Challenge — ~6× Baseline Speed with PPL 2.39
TL;DR: Korean AI startup VIDRAFT achieved the top verified score in Google and Hugging Face's The Fast Gemma Challenge inference optimization competition, reaching 510.58 tokens per second and a perplexity of 2.39 on the same hardware every other participant used — through software-only tuning. The result represents roughly 6× the throughput of the base model at comparable output quality, making VIDRAFT's inference optimization stack worth watching if you care about LLM serving costs.
What it is
The Fast Gemma Challenge is a joint competition hosted by Google and Hugging Face focused exclusively on inference-time optimization. The rules are deliberately constrained to create a fair, reproducible comparison:
-
Fixed model: Google's
gemma-4-E4B-it - Fixed hardware: NVIDIA A10G GPUs (identical for all participants)
- Optimization surface: Software tuning only — no model swaps, no hardware upgrades
- Verification requirement: Self-reported numbers don't count. Only results that pass the organizers' independent, private reproduction process earn the "VERIFIED" badge and appear on the official leaderboard
VIDRAFT (대표/CEO: 김민식), a Korean deep-tech AI startup, entered with an agent they call vidraft-darwin and claimed the top spot in the verified category.
How it works
VIDRAFT applied two internally developed technology components to squeeze maximum throughput out of the fixed hardware and model:
VK-series inference engine — VIDRAFT's proprietary inference engine family. At a conceptual level, inference engines in this class typically apply techniques such as continuous batching, kernel-level attention optimizations, quantization strategies, and memory scheduling to reduce per-token latency and increase hardware utilization. VIDRAFT has not disclosed specific implementation details beyond the engine's name and general purpose.
POCKET platform — Described as a large-model hosting and serving platform, POCKET is the layer through which
vidraft-darwinwas deployed and orchestrated during the challenge. Think of it as the serving infrastructure glue that sits between raw inference kernels and the benchmark workload.
The key engineering constraint worth emphasizing: no model weights were changed. Every optimization had to live in the software stack — scheduling, batching, memory layout, kernel dispatch — rather than in the model itself. Achieving a ~6× throughput gain under those conditions is a meaningful engineering result, since it means identical serving costs could support six times the request volume, or equivalent volume at roughly one-sixth the compute cost.
Benchmarks & Results
All numbers below come directly from VIDRAFT's public announcement and the challenge leaderboard:
| Metric | Value |
|---|---|
| Throughput (TPS) | 510.58 tokens/second |
| Perplexity (PPL) | 2.3929 |
| Speedup vs. base model | ~6× |
| Verification status | VERIFIED (organizer-reproduced) |
| Model | gemma-4-E4B-it |
| Hardware | NVIDIA A10G (competition-standard) |
A few things worth noting for engineers interpreting these numbers:
- PPL 2.39 matters as much as TPS. Throughput gains that trash output quality are easy to achieve (e.g., aggressive truncation or degraded sampling). The perplexity figure is the challenge's quality gate — a low PPL alongside high TPS indicates the optimization preserves generation fidelity.
- The VERIFIED label is the meaningful one. Any team can claim a self-measured number. Passing the organizers' blind reproduction is the bar that makes this result comparable across all participants.
- Results are publicly visible on the official The Fast Gemma Challenge leaderboard.
How to try it
The source article does not announce a public Hugging Face model release, GitHub repository, or OpenAI-compatible API endpoint for vidraft-darwin or the VK inference engine at this time.
If VIDRAFT publishes public access channels, they would likely appear on:
- Hugging Face: VIDRAFT org page (check for availability)
- VIDRAFT's official channels for API access announcements
For now, the competition result and leaderboard entry are the publicly verifiable artifacts. Watch VIDRAFT's announcements for any developer-facing release.
FAQ
Q: Is the ~6× speedup over the raw gemma-4-E4B-it baseline, or over another optimized baseline?
A: Per the announcement, the ~6× figure is relative to the base (unoptimized) model running on the same A10G hardware. The challenge's fixed-hardware constraint means all participants start from the same baseline, so the multiplier reflects pure software optimization gain.
Q: Does a PPL of 2.39 mean the outputs are actually good, or is this a narrow benchmark artifact?
A: Perplexity on a held-out validation set is the quality metric the challenge organizers chose. A PPL of 2.39 on gemma-4-E4B-it's validation set indicates the model's output distribution is well-preserved — it's not a perfect proxy for downstream task quality, but it's a meaningful guard against the kind of degenerate optimizations (e.g., always-greedy, truncated generation) that inflate TPS at the cost of coherence.
Q: Can I reproduce this result myself?
A: The base model (gemma-4-E4B-it) is publicly available via Hugging Face, and A10G instances are available through major cloud providers. VIDRAFT's specific VK engine and POCKET platform are proprietary and not publicly released as of this writing, so exact reproduction of their stack is not possible without their tooling.
Originally reported by 한국경제TV (2026-08-03) — source article.
Top comments (0)