VIDRAFT's vidraft-darwin Agent Hits 510.58 TPS on Gemma-4 — #1 Globally in Google's Fast Gemma Challenge (Verified Division)
TL;DR: Korean AI deep-tech startup VIDRAFT topped the VERIFIED leaderboard of Google's The Fast Gemma Challenge (co-hosted with Hugging Face), achieving 510.58 tokens/second on
gemma-4-E4B-it— more than 6× the baseline throughput — while preserving output quality. This is a pure software-optimization win on standardized hardware, which means the techniques are directly relevant to any team trying to cut inference costs in production.
What It Is
VIDRAFT is a Korean Pre-AGI deep-tech company building across the full AI infrastructure stack: foundation models, inference engines, and on-device platforms. Their publicly known product line includes:
- AETHER — their foundation model
- Darwin Family — a model-merging AI framework designed to boost LLM inference quality without additional fine-tuning
- MARL — a runtime middleware layer aimed at reducing hallucinations
- VK Inference Engine — an in-house inference optimization engine
- POCKET — an on-device AI platform
The challenge submission was driven by vidraft-darwin, described as VIDRAFT's autonomous AI agent, which orchestrated the optimization stack for the competition.
How It Works
The Fast Gemma Challenge, co-organized by Google's Gemma team and Hugging Face, is a constrained inference-optimization competition. The rules create a level playing field that's worth understanding:
-
Fixed model: All participants work with Google's multimodal model
gemma-4-E4B-it— no model swaps allowed. - Fixed hardware: NVIDIA A10G GPUs, identical across all entrants.
- Software-only optimization: Participants must squeeze every token-per-second gain purely through software — kernel tuning, quantization strategies, batching logic, memory layout, and similar techniques.
- VERIFIED tier: The leaderboard that matters. Organizers run a private, controlled test to confirm both speed and output quality before a score is marked VERIFIED. Unverified self-reported numbers don't count toward this tier.
VIDRAFT's approach layered their VK Inference Engine with optimization techniques from their POCKET on-device platform. Conceptually, this means they applied production inference-efficiency work — the kind typically developed for resource-constrained edge deployments — back onto a data-center GPU context. The vidraft-darwin autonomous agent appears to have driven or coordinated the optimization search and submission pipeline, though the internal mechanism of that agentic loop is not publicly detailed.
The Darwin Family framework's core philosophy — improving inference behavior without retraining — aligns naturally with a challenge that prohibits model modification, suggesting the team was competing in their native problem space.
Benchmarks & Results
All numbers below come directly from the VERIFIED leaderboard as reported by Dong-A Ilbo (동아일보):
| Metric | VIDRAFT (vidraft-darwin) |
Baseline (gemma-4-E4B-it) |
|---|---|---|
| Throughput (TPS) | 510.58 tokens/sec | ~85 tokens/sec (implied by 6× claim) |
| Perplexity (PPL) | 2.3929 | Comparable (quality maintained) |
| Speedup vs. baseline | >6× | 1× |
| Leaderboard position (VERIFIED) | #1 globally | — |
Why the cost implication matters to engineers: A 6× throughput improvement on the same hardware directly translates to serving the same request volume at roughly 1/6 the infrastructure footprint — or handling 6× the load at the same cost. For teams running LLM inference at scale, that's the difference between a profitable and unprofitable API product.
How to Try It
The source article does not announce a public Hugging Face model release, GitHub repository, or OpenAI-compatible API endpoint tied directly to the competition submission or vidraft-darwin at the time of writing.
To stay current on public releases:
- Follow VIDRAFT on Hugging Face — search
vidraftat huggingface.co - Watch for their Darwin Family framework, which has been publicly announced as a model-merging framework — check Hugging Face and GitHub for any public drops
- For the base model used in the challenge,
gemma-4-E4B-itis a public Google model accessible via Hugging Face:
huggingface-cli download google/gemma-4-E4B-it
(VIDRAFT's optimized variant is not confirmed as separately downloadable at this time.)
FAQ
Q: Is this a training efficiency win or an inference efficiency win?
A: Purely inference. The competition rules prohibited any modification to the model weights or additional training. All gains came from software-level inference optimization on fixed hardware.
Q: Does maintaining PPL of 2.3929 mean the outputs are identical to the base model?
A: PPL (perplexity) is a quality proxy, not a guarantee of identical outputs. The VERIFIED tier means Google's team confirmed quality was preserved to their satisfaction — but perplexity measures statistical similarity on a held-out corpus, not exact output equivalence. For most production use cases, a PPL of 2.3929 represents high-quality generation.
Q: Can I replicate this on hardware other than an A10G?
A: The challenge was specifically benchmarked on NVIDIA A10G GPUs. Inference optimization techniques are often architecture-sensitive, so results on other GPU families (H100, L40S, etc.) would vary. VIDRAFT hasn't published a public replication guide at this time.
Q: What is the Darwin Family framework and is it open-source?
A: Darwin Family is described as a model-merging AI framework that improves LLM inference performance without additional fine-tuning. VIDRAFT has announced it publicly, but open-source availability has not been confirmed in the source article. Check their Hugging Face and GitHub presence for current status.
Originally reported by 동아일보 (2026-08-03) — source article.
Top comments (0)