VIDRAFT Launches VKAE: A Kernel-Level LLM Inference Acceleration Engine with Public Leaderboard and Unified Docker Container

TL;DR: VIDRAFT has open-sourced a unified Docker container bundling model weights and a pre-tuned serving stack for VKAE (VIDRAFT Kernel Acceleration Engine), its in-house kernel-level LLM inference accelerator. On identical GPU hardware and conditions, VKAE has recorded up to 23.4× higher throughput than standard serving baselines — without degrading output quality. The container ships with an OpenAI-compatible API, so existing applications can plug in with minimal code changes.

---

What it is

VKAE (VIDRAFT Kernel Acceleration Engine) is a kernel-level inference acceleration engine built by VIDRAFT, a Seoul-based Pre-AGI AI startup founded in 2024. It targets large language model (LLM) serving and is designed to increase throughput and reduce latency on the same GPU hardware you already own — with no sacrifice to model output quality.

Alongside the engine itself, VIDRAFT is publishing two complementary artifacts:

• A public performance leaderboard — reproducible benchmark results that users can verify on their own hardware.
• A unified Docker container — model weights and the optimized serving environment are packaged together, eliminating complex multi-step installation.

The container exposes an OpenAI-compatible API, meaning any application already talking to OpenAI's chat completion endpoints can be pointed at VKAE with trivial configuration changes.

VKAE is positioned as a key pillar of VIDRAFT's broader full-stack AI strategy, sitting alongside its model families (Darwin, Chimera, Aether), the FINAL Bench metacognitive evaluation suite, and the MARL runtime middleware for hallucination reduction.

---

How it works

VIDRAFT has not disclosed the internal kernel implementation of VKAE — that remains proprietary — but the publicly stated approach is to optimize at the kernel level rather than the framework or orchestration level. The key design principles, as stated by the company, are:

• Kernel-level optimization: Acceleration happens below the framework layer, targeting the low-level GPU execution primitives that drive token generation.
• Throughput-first, quality-preserved: The engine is designed to increase the number of tokens processed per second while keeping the model's generation outputs numerically equivalent to unaccelerated serving.
• Hardware-specific tuning: Primary support targets NVIDIA Blackwell (B200) and Hopper (H100, H200) architectures, with the company actively expanding coverage to additional GPU classes including smaller configurations such as A10G.
• Unified packaging: By bundling weights and the serving environment into a single container, VIDRAFT removes the reproducibility gap that often exists between a lab benchmark and a user's deployment.

The philosophy behind the leaderboard release is explicit in the CEO's statement: speed claims that cannot be independently reproduced are meaningless. Publishing a container that lets any engineer run the same benchmark on their own hardware is the core verification mechanism.

---

Benchmarks & results

All numbers below are VIDRAFT's own measurements, published as part of the leaderboard release:

Scenario	Model	Hardware	Precision	Result
Single-stream generation	Qwen3.5-35B-A3B	NVIDIA B200	FP8	Up to 601 tokens/sec
Multi-request throughput	Qwen3.5-35B-A3B	NVIDIA B200	FP8	Over 10,000 tokens/sec
Speedup vs. standard serving	Same model, same GPU	Same conditions	—	Up to 23.4×

The same family of optimizations has also been applied to VIDRAFT's ultra-large JGOS-398B model, though specific throughput figures for that model were not separately itemized in this release.

VIDRAFT's stated rationale for opening the leaderboard is fourfold:

1. Transparent verification — users reproduce the numbers on their own hardware.
2. Cost reduction — higher throughput on the same GPU directly lowers per-token serving cost.
3. Stability — quality is preserved through the acceleration process.
4. Immediate integrability — the OpenAI-compatible API means no rewriting of existing application logic.

---

How to try it

VIDRAFT has announced that the unified Docker container is publicly available. The container includes model weights and the optimized serving stack. Because it exposes an OpenAI-compatible API, integration looks like any standard OpenAI client call — just swap the base_url:

# Generic OpenAI-compatible curl — replace HOST and PORT with your container's address
curl http://<HOST>:<PORT>/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-name",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

For the specific container image name, pull command, and leaderboard URL, refer to VIDRAFT's official release channels. The source article does not specify a public Hugging Face repository or GitHub URL at this time — check VIDRAFT's official site and announcements for the exact registry location.

---

FAQ

Q: Does VKAE require model retraining or fine-tuning?
A: No. VKAE operates at the kernel/serving layer. The model weights are the same; the engine changes how inference is executed on the GPU, not the model itself. Output quality is stated to be preserved.

Q: Which GPUs are supported today?
A: VIDRAFT's primary targets are NVIDIA Blackwell (B200) and Hopper (H100, H200) GPUs. Support for additional GPU classes, including smaller configurations like the A10G, is actively being expanded.

Q: Can I use VKAE with my existing OpenAI-based application?
A: Yes. VKAE's container exposes an OpenAI-compatible API, so the change is typically limited to updating the base_url in your existing client configuration — no endpoint-specific code changes required.

Q: Are the kernel internals open-source?
A: No. The internal kernel implementation is VIDRAFT's proprietary IP and is not disclosed. What is public is the leaderboard (with reproducible benchmark conditions) and the Docker container, which lets you verify performance claims on your own hardware.

---

Originally reported by 디지털타임스 (2026-07-06) — source article.