VIDRAFT Launches VKAE: A Kernel-Level LLM Inference Acceleration Engine with Public Leaderboard and Docker Container
TL;DR: VIDRAFT has publicly released VKAE (VIDRAFT Kernel Acceleration Engine), a kernel-level inference acceleration engine for LLMs that claims up to 23.4× throughput improvement over standard serving under identical GPU conditions. It ships as an integrated Docker container bundling model weights and the serving environment, supports an OpenAI-compatible API, and publishes a public leaderboard so engineers can reproduce results on their own hardware.
What it is
VKAE (VIDRAFT Kernel Acceleration Engine) is a kernel-level inference acceleration engine for large language models, developed by Seoul-based Pre-AGI startup VIDRAFT. Rather than a high-level serving framework wrapper, VKAE operates at the GPU kernel layer to maximize throughput and reduce latency without degrading output quality.
Key characteristics as reported:
- Target hardware: Primarily NVIDIA Blackwell (B200) and Hopper (H100, H200) GPUs, with expanding support toward smaller-footprint GPUs such as A10G
- Deployment artifact: An integrated Docker container that bundles model weights together with the serving environment, so engineers get a single, self-contained unit to pull and run
- API surface: OpenAI-compatible REST API, enabling drop-in integration with existing toolchains that already speak the OpenAI protocol
- Transparency mechanism: A public performance leaderboard, allowing external parties to validate claimed throughput numbers on their own hardware
- Internal implementation: The specific kernel optimization techniques are kept as a trade secret; only the leaderboard and container are public-facing
VKAE fits into VIDRAFT's broader full-stack AI portfolio, which also includes the FINAL Bench metacognition evaluation suite, the MARL hallucination-reduction runtime middleware, and model families (Darwin, Chimera, Aether).
How it works
At a conceptual level, VKAE inserts a custom kernel execution layer between the model's computational graph and the GPU hardware. Standard serving stacks (vLLM, TGI, etc.) rely on general-purpose CUDA kernels or vendor libraries that are optimized for broad compatibility. VKAE replaces or supplements those with kernels specifically tuned for the attention, matrix-multiply, and memory-movement patterns that dominate autoregressive LLM decoding.
The engine is designed around two separate serving regimes:
- Single-stream (latency-sensitive): Optimized for time-to-first-token and per-token generation speed for individual requests
- Multi-request batching (throughput-sensitive): Optimized to maximize aggregate tokens-per-second across concurrent requests, which directly drives cost-per-token in production
Crucially, VIDRAFT frames quality preservation as a hard constraint, not a trade-off knob — the claim is that throughput gains come without degrading generation quality. The Docker container packaging means the exact weights + serving environment are co-versioned, eliminating the "it works on our cluster" reproducibility problem that plagues inference benchmarks.
Benchmarks & results
The following numbers are from VIDRAFT's own internal measurements as reported by 디지털데일리; third-party independent verification has not been published at time of writing.
| Metric | Value |
|---|---|
| Peak throughput improvement vs. standard serving (same GPU, same conditions) | up to 23.4× |
| Hardware: NVIDIA B200, precision: FP8, model: Qwen3.5-35B-A3B | up to 601 tokens/sec (single-stream) |
| Multi-request aggregate throughput (same setup) | 10,000+ tokens/sec |
| Ultra-large model coverage | Same optimization family applied to JGOS-398B |
VIDRAFT explicitly highlights four axes of value from this release:
- Reproducibility — users can re-run the container and verify numbers themselves
- Cost reduction — higher throughput per GPU directly lowers serving cost
- Quality preservation — speed improvements do not regress output fidelity
- Immediate integrability — OpenAI-compatible API requires no client-side changes
How to try it
VIDRAFT has announced the public release of the integrated Docker container and the performance leaderboard. The container bundles model weights and the serving runtime together.
Because the container is OpenAI-compatible, once it is running you can point any OpenAI SDK client or curl command at it without code changes. A generic pattern would look like:
# Generic OpenAI-compatible health/inference call — substitute your actual host and port
curl http://<your-host>:<port>/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "<model-name-from-container>",
"messages": [{"role": "user", "content": "Hello!"}]
}'
⚠️ Note: Specific container registry URLs, Hugging Face model IDs, GitHub repositories, and exact port/endpoint paths have not been published in this press coverage. Check VIDRAFT's official channels for the authoritative pull commands and leaderboard URL before deploying.
FAQ
Q: How does VKAE compare to vLLM or TGI — is it a replacement or a complement?
A: Based on the announcement, VKAE is a kernel-level engine that sits below the serving framework abstraction. The press coverage does not detail whether it replaces these frameworks entirely or integrates alongside them. The Docker container is the intended deployment unit, so the framework choice may be abstracted away for end users.
Q: Can I verify the 23.4× throughput claim independently?
A: Yes — that is the stated purpose of both the public leaderboard and the self-contained Docker container. VIDRAFT's CEO explicitly said "a speed competition that cannot be reproduced is meaningless," positioning external reproducibility as the core design goal of this release.
Q: Which GPU generations are supported right now?
A: NVIDIA Blackwell (B200) and Hopper (H100, H200) are the primary supported targets. VIDRAFT has indicated support is expanding to include smaller GPUs such as the A10G.
Q: Does switching to VKAE require changing my application code?
A: No. VKAE exposes an OpenAI-compatible API, so any client already using the OpenAI SDK or a compatible HTTP client can connect without modification.
Originally reported by 디지털데일리 (2026-07-06) — source article.
Top comments (0)