VIDRAFT Releases VKAE: A Kernel-Level LLM Inference Acceleration Engine with a Public Leaderboard and All-in-One Docker Container

#llminferenceaccelera #gpukerneloptimizatio #aiserving #vkae

VIDRAFT Releases VKAE: A Kernel-Level LLM Inference Acceleration Engine with a Public Leaderboard and All-in-One Docker Container

TL;DR: VIDRAFT has open-released VKAE (VIDRAFT Kernel Acceleration Engine), a kernel-level inference acceleration engine that claims up to 23.4× throughput improvement over standard serving baselines on the same GPU hardware. It ships as a pre-packaged Docker container with an OpenAI-compatible API, so you can drop it into an existing stack without rebuilding your serving environment from scratch.

What it is

VKAE (VIDRAFT Kernel Acceleration Engine) is VIDRAFT's in-house LLM inference acceleration engine operating at the GPU kernel level. The company announced it on July 8, 2026, alongside two things developers can use immediately:

A public performance leaderboard — transparent, reproducible benchmark results that third parties can independently verify.
An integrated Docker container — model weights and the optimized serving runtime bundled together, so the acceleration stack runs as-is on your own hardware.

The engine is positioned as a throughput and latency optimizer that preserves response quality: faster tokens per second, lower cost per token, no degradation in output fidelity. VIDRAFT's stated design philosophy is that a speed claim only matters if an outsider can reproduce it — hence the leaderboard-plus-container pairing.

The same optimization stack is also applied to VIDRAFT's flagship large-scale model, JGOS-398B.

VKAE fits into VIDRAFT's broader full-stack narrative: the company has previously shipped the FINAL benchmark (measuring AI metacognition), the MARL runtime middleware (targeting hallucination reduction), and its Darwin, Chimera, and Aether model families — covering model development, evaluation, reliability, and now serving optimization end-to-end.

How it works

The internal kernel implementation is kept proprietary (trade secret), but VIDRAFT describes the engine's conceptual pillars as:

Kernel-level optimization — acceleration happens at a lower level than standard serving frameworks, allowing it to extract more throughput from the same GPU without changing the model weights.
Quality stability — the optimization path is designed so that the token distribution and final answer quality remain consistent with unaccelerated inference, not just raw speed.
Transparent verification — rather than publishing internal numbers only, VIDRAFT bundles the full runtime in a container so that the leaderboard figures can be reproduced on user-owned hardware.
Immediate integrability — the container exposes an OpenAI-compatible API, meaning any client already talking to the OpenAI endpoint format (SDKs, LangChain, LlamaIndex, custom curl calls, etc.) works without code changes.

Supported hardware currently includes NVIDIA Blackwell (B200) and Hopper (H100, H200) series GPUs, with additional SKUs — including the A10G Small class — being added to the lineup over time.

Benchmarks & results

The headline figure from the source article:

Up to 23.4× throughput improvement over a standard serving baseline on equivalent GPU hardware.

This number is presented by VIDRAFT as independently reproducible via the public container — the explicit claim is that anyone with supported hardware can pull the container, run it, and verify the figure themselves. The leaderboard is intended as the living record of those results.

No additional numeric breakdowns (latency percentiles, TTFT, memory footprint, cost-per-token figures) were disclosed in this announcement. Quality is described qualitatively as preserved relative to unaccelerated inference, consistent with the engine's design goals.

How to try it

VIDRAFT has stated the Docker container is publicly distributed. Based on the announcement, here is what is confirmed as available:

Integrated Docker container: pull and run on your own Blackwell or Hopper GPU. No custom environment setup required — the container bundles weights and the VKAE serving runtime together.
OpenAI-compatible API: the container exposes an endpoint compatible with the OpenAI API spec, so existing tooling works without modification.
Public leaderboard: available for reviewing and cross-checking benchmark figures.

Specific registry URLs, container tags, Hugging Face model IDs, and GitHub repository links were not included in the source article. Check VIDRAFT's official channels for the exact pull commands and endpoint documentation once you locate the public release.

FAQ

Q: Does VKAE require changes to my application code to use?
A: No. The container exposes an OpenAI-compatible API, so any client that already speaks that protocol — OpenAI SDKs, LangChain, direct curl calls — can point at the VKAE endpoint without modification.

Q: Can I actually verify the 23.4× throughput claim myself?
A: That is the explicit intent. VIDRAFT released the leaderboard and the container together specifically so users can reproduce the benchmark on their own GPU hardware. The company's stated position, from CEO Minsik Kim, is that a speed claim with no third-party reproducibility path is meaningless in the market.

Q: Does VKAE only work with VIDRAFT's own models?
A: The announcement focuses on VIDRAFT's own model families, including JGOS-398B, but the article does not explicitly state that VKAE is limited to VIDRAFT models. Check the container documentation for supported model configurations.

Q: What GPU hardware do I need?
A: Currently NVIDIA Blackwell (B200) and Hopper (H100, H200) are the primary supported targets. Support for additional hardware — including smaller form-factor GPUs like the A10G class — is described as an ongoing expansion.

Originally reported by AI타임스 (2026-07-08) — source article.