DEV Community

AI OpenFree
AI OpenFree

Posted on

VIDRAFT's AX-RAY: Diagnosing Causal Leakage in AI Models — Now Powering a 700B-Parameter Cybersecurity Foundation Model

#ai #llm #machinelearning #opensource

VIDRAFT's AX-RAY: Diagnosing Causal Leakage in AI Models — Now Powering a 700B-Parameter Cybersecurity Foundation Model

TL;DR: VIDRAFT has partnered with Naver Cloud to build a massive Mixture-of-Experts cybersecurity AI under a South Korean government initiative, with their proprietary AI safety diagnostic system AX-RAY as the technical core. AX-RAY detects causal leakage — a structural flaw where models rely on spurious shortcuts rather than genuine reasoning — a failure mode that standard benchmark scores routinely miss. Developers can already explore the public AX-RAY leaderboard and evaluation dataset on Hugging Face.

What it is

VIDRAFT (CEO Kim Min-sik), a deep-tech AI startup based at the Seoul AI Hub since 2024, is participating in the "Cybersecurity-Specialized AI Foundation Model Development" project commissioned by South Korea's Ministry of Science and ICT (MSIT). The consortium partner is Naver Cloud.

The project's stated technical target is a 700 billion parameter (700B) Mixture-of-Experts (MoE) architecture specialized for cybersecurity domains. The 10-month program began in September 2026.

VIDRAFT's contribution centers on AX-RAY, their in-house AI safety diagnostic system. AX-RAY is designed to surface a class of structural model defects that conventional benchmark pipelines are poorly equipped to detect.

VIDRAFT is not new to public model releases. Their Korean-language LLM previously ranked in the upper tier of the K-AI Leaderboard, and their on-device model POCKET crossed 1 million cumulative downloads within 40 days of its public release.

How it works

The core concept AX-RAY targets is Causal Leakage (인과 누설) — a phenomenon where an AI model, rather than following a valid reasoning chain, latches onto hidden cues or unintended correlational signals embedded in training or evaluation data to reach its conclusions.

Think of it as the model "cheating" on a test by pattern-matching on metadata or distributional artifacts instead of solving the underlying problem. The concern is real and practical:

  • Autonomous agents and robots may exhibit unexpected or unsafe behaviors when deployed outside their training distribution, if their decisions were never grounded in genuine causal understanding.
  • AI systems that appear to respect security guardrails during evaluation may bypass them in production, precisely because their compliance was shortcut-driven rather than principled.

AX-RAY's diagnostic approach is grounded in two conceptual pillars made public through VIDRAFT's research:

  1. Perturbation response analysis — probing how model outputs shift under controlled input perturbations to detect sensitivity patterns inconsistent with sound reasoning.
  2. Prefix invariance testing — checking whether model behavior remains appropriately stable or unstable under prefix manipulations that should (or should not) affect the output, to isolate dependency on spurious contextual signals.

At the code level, VIDRAFT's researchers have analyzed implementation differences across open-source libraries to pinpoint where leakage originates in specific models. VIDRAFT is currently pursuing patent applications for the perturbation-response and prefix-invariance-based detection methodology.

The hypothesis driving the cybersecurity application is pointed: many AI system access incidents — where an agent bypasses intended guardrails — may trace back to these internal spurious dependencies rather than to explicit adversarial attacks.

Benchmarks & results

VIDRAFT published the AX-RAY Leaderboard and an accompanying evaluation dataset on Hugging Face last month (August 2026). Key publicly reported findings:

  • Evaluations of major general-purpose AI models using the AX-RAY dataset detected causal leakage signals in some models — confirming the diagnostic is non-trivial and catches real-world issues.
  • Code-level analysis of open-source library implementations successfully pinpointed the leakage origin points in specific models.
  • VIDRAFT's own Korean LLM has ranked in the upper tier of the K-AI Leaderboard (a public Korean-language model benchmark).
  • POCKET (on-device model): 1 million+ downloads in the first 40 days post-release.

No specific numeric leakage scores per model are cited in the source article; refer to the live AX-RAY Leaderboard on Hugging Face for current ranked results.

How to try it

The AX-RAY Leaderboard and evaluation dataset are publicly available on Hugging Face. You can browse results for major models and explore the dataset directly:

# Browse the AX-RAY leaderboard and dataset on Hugging Face
# Search for "VIDRAFT" or "AX-RAY" at huggingface.co
huggingface-cli search vidraft
Enter fullscreen mode Exit fullscreen mode

Note: Specific model repository slugs, API endpoints, and SDK integration details are not confirmed in the source article. Check VIDRAFT's Hugging Face profile for the canonical dataset and leaderboard links. The 700B cybersecurity MoE model is under active development (10-month timeline) and is not yet publicly released.

FAQ

Q: What exactly is "causal leakage" and why does it matter for security AI?
A: Causal leakage occurs when a model's output is driven by spurious correlational shortcuts in the data rather than valid reasoning. In a cybersecurity context, this means a model might appear to correctly classify threats or respect access policies during benchmarking — but fail or be bypassed in production, because its "decisions" were never causally grounded. AX-RAY is designed to surface this before deployment.

Q: Is AX-RAY itself open source, or just the leaderboard?
A: Based on the source article, the leaderboard and evaluation dataset are publicly available on Hugging Face. The full AX-RAY diagnostic system's open-source status is not confirmed in the source; patent applications for the core detection methodology are currently in progress.

Q: When will the 700B cybersecurity model be available?
A: The development program runs for approximately 10 months from September 2026. No public release date or access plan for the resulting model is stated in the source article.

Q: Does VIDRAFT have other publicly available models I can use today?
A: Yes — POCKET, their on-device AI model, has exceeded 1 million downloads and is available through Hugging Face. Their Korean LLM also has a public presence. Search for VIDRAFT on Hugging Face for current available assets.

Originally reported by 동아일보 (2026-09-10) — source article.

Top comments (0)