VIDRAFT's VKAE × VKUE Dual-Engine Strategy: Pushing 35B-Parameter LLM Inference from B200 Data Centers to CPU-Only Laptops
TL;DR: Seoul-based deep-tech startup VIDRAFT has packaged two complementary inference engines — VKAE (a kernel-level acceleration engine) and VKUE (a lightweight ubiquitous runtime) — into a single infrastructure stack targeting AI data center operators. Together they claim up to 23.4× throughput over standard serving on the datacenter side, while making a 35B-parameter frontier model runnable on consumer GPUs, laptops, and even CPU-only environments. If you're building or operating LLM serving infrastructure, this dual-engine architecture is worth watching.
What it is
VIDRAFT (비드래프트), a Seoul-based deep-tech company and resident of the Seoul AI Hub, has announced a paired engine strategy for LLM inference infrastructure built around two proprietary components:
VKAE (VIDRAFT Kernel Acceleration Engine): A kernel-level inference acceleration engine targeting high-throughput datacenter deployments. It ships as a Docker container integrating model weights and an optimized serving environment, exposes an OpenAI-compatible API, and publishes results on a public leaderboard for reproducibility.
VKUE (VIDRAFT Kernel Ubiquitous Engine): A lightweight runtime designed to make frontier-class models deployable far beyond the data center — on consumer GPUs, laptops, and CPU-only machines. It enables on-premise deployment in air-gapped or network-isolated environments (hospitals, finance, defense, manufacturing).
The flagship model running on this stack is Ourbox-35B-JGOS, a 34.7B-parameter model architected so that only ~3B parameters activate per token at inference time — a sparse-activation design that makes aggressive hardware democratization practical without catastrophic quality loss.
How it works
The two engines address opposite ends of the infrastructure spectrum and are designed to be deployed together as a complementary pair.
VKAE operates at the kernel level, rewriting the low-level compute graph of model execution to extract significantly higher throughput from the same physical GPU. Rather than relying on standard inference serving stacks, it applies custom kernel optimizations that reduce overhead per forward pass, allowing a single GPU to handle far more concurrent requests. The OpenAI-compatible API surface means it can drop into existing toolchains with minimal integration work.
VKUE takes a different approach: instead of squeezing more out of expensive hardware, it makes large models tractable on cheap or scarce hardware. The key enabler is the sparse-activation architecture of Ourbox-35B-JGOS — 34.7B total parameters, but only ~3B active per token. This dramatically reduces peak memory bandwidth and compute requirements during a forward pass, enabling deployment on hardware that would normally be completely unsuitable for a model at this scale.
The combined pitch to AI IDC (AI Internet Data Center) operators is a two-tier architecture: VKAE maximizes revenue-per-GPU in the central datacenter, while VKUE extends AI capability to edge sites and closed networks where data cannot leave the premises.
Benchmarks & results
All figures below are sourced directly from the 전자신문 article:
Throughput across hardware tiers (Ourbox-35B-JGOS with VKUE):
| Environment | Tokens/sec |
|---|---|
| B200 GPU | ~18,057 |
| A10G (single card) | ~126 |
| 8 GB gaming laptop GPU | ~20 |
| CPU-only (no GPU) | ~17 |
VKAE acceleration:
- Up to 23.4× higher throughput vs. standard serving on identical GPU hardware
- On Qwen3.5-35B-A3B: single-stream 601 tokens/sec; multi-request load >10,000 tokens/sec
Combined peak (VKAE + VKUE, B200, 256 concurrent users):
- Sustained >18,000 tokens/sec
Quality benchmarks:
- Ourbox-35B-JGOS: GPQA Diamond 86.4% (maj@8)
- Previously announced Darwin-398B-JGOS: GPQA Diamond 90.9%
These benchmark results are published on a public leaderboard and are claimed to be reproducible via the Docker container distribution.
How to try it
VIDRAFT distributes VKAE via a Docker container that bundles model weights and the optimized serving environment. The API surface is OpenAI-compatible, meaning you can point any OpenAI SDK client at it without modifying application code.
A generic test against any OpenAI-compatible endpoint looks like this:
curl https://<vidraft-endpoint>/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <your-api-key>" \
-d '{
"model": "<model-name>",
"messages": [{"role": "user", "content": "Hello"}]
}'
Note: Specific endpoint URLs, model name strings, and API keys are not published in the source article. Contact VIDRAFT directly or check their public leaderboard for current access details. Hugging Face or GitHub distribution channels were not mentioned in the source.
FAQ
Q: How does a 35B-parameter model actually run on a CPU-only machine without being unusably slow?
A: Ourbox-35B-JGOS uses a sparse-activation architecture: despite having 34.7B total parameters, only approximately 3B parameters are activated per token during inference. This dramatically cuts memory bandwidth and FLOPs per forward pass, which is what makes ~17 tokens/sec on a CPU feasible — slow for high-traffic serving, but workable for local or air-gapped deployments.
Q: Can I integrate VKAE into my existing LLM serving pipeline?
A: The OpenAI-compatible API means the integration surface is minimal — any client already talking to OpenAI or a compatible server can switch targets. The Docker container packaging means deployment is self-contained. Specific integration docs are not detailed in the source; reach out to VIDRAFT for enterprise onboarding.
Q: Is VKUE only for edge/offline use, or can it run in a datacenter too?
A: VKUE is positioned primarily for resource-constrained or network-isolated environments. For datacenter deployments with capable GPUs, VKAE is the primary performance lever. The intended pattern is running both: VKAE at the center for throughput, VKUE at the edge for coverage.
Originally reported by 전자신문 (2026-07-13) — source article.
Top comments (0)