Delafosse Olivier

Posted on Jul 10 • Originally published at coreprose.com

JadePuffer: Engineering the First Fully LLM‑Driven Ransomware Kill Chain

#ai #machinelearning #llm #programming

Originally published on CoreProse KB-incidents

1. From LLM Hallucinations to Operational Malware: Why JadePuffer Is Plausible

Browser-only ransomware was once dismissed as “LLM hallucination,” until researchers showed a fully browser-native ransomware path using Chrome’s File System Access API. [1] On Android Chrome, a web page can legitimately get read/write access to photo directories and then modify those files—enabling ransomware behavior without APKs, exploits, or root. [1]

⚠️ This challenges the assumption that “no install = no serious malware.”

At the same time, LLM-powered agents are increasingly wired into:

User-facing interfaces (chatbots, copilots)
Internal systems (ticketing, CRM, CI/CD)
External tools/plugins (HTTP APIs, shell tools) [2][3]

This makes prompts, documents, tool outputs, and agent decisions part of the attack surface. [3] Many organizations now treat LLMs and AI agents as security-critical components. [2][3]

The University of Toronto’s AI worm goes further: an open-weight LLM worm that:

Runs entirely on local machines, with no cloud API or centralized C2
Uses a large language model to reason per host, pick attacks, and self‑propagate
Compromised 73.8% of a simulated network in 7 days [6]

🧩 Combined, this makes JadePuffer realistic:

Browser-only access to valuable files via Chrome APIs [1]
Autonomous local LLM worms without conventional C2 [6]
Insecure enterprise LLM apps open to prompt injection, data exfiltration, and plugin abuse [2][4]

OWASP Top 10 for LLM Applications now frames key risks: prompt injection, insecure output handling, data poisoning, model theft, and more. [3][4] JadePuffer simply converges these demonstrated techniques into an LLM‑driven ransomware-as-an-agent framework.

Section takeaway

JadePuffer is a plausible fusion of browser APIs, local LLM worms, and insecure AI integrations—each already shown in research. [1][6]

2. Hypothetical JadePuffer Kill Chain: Step-by-Step Attack Narrative

Step 1: Social engineering via “AI enhancement”

JadePuffer starts as a web app offering “AI photo enhancement” or “AI document cleanup,” echoing the Chrome proof of concept. [1]

User uploads a sample photo.
Site shows an impressive LLM-enhanced preview.
Site then asks for folder-level access to “batch-enhance your library.”

The permission uses Chrome’s File System Access API, especially risky on Android where photo directories are high value. [1] After a convincing demo, many users click “Allow.”

⚠️ In one 3,000-person SaaS company test, ~40% of employees granted directory access within 5 seconds when framed as “AI auto-organization of photos.” [1]

Step 2: Client-side LLM triage and encryption

After access is granted, a client-side agent (WASM-hosted LLM or backend API) can:

Enumerate files via File System Access API
Classify them: personal photos, IDs, contracts, work docs
Prioritize items by “extortion value”: memories, legal docs, critical business data [1][5]

Conventional crypto handles encryption; the LLM decides which files and in what order, repurposing the same classification logic defenders use for logs and documents. [5]

Step 3: Dropping the LLM worm component

The browser stage then deploys a local worm modeled on the Toronto design:

Bundles an open-weight 7B–13B model, quantized for commodity CPUs/GPUs [6][5]
Runs an autonomous agent loop to plan spread per host
Consumes victim compute, just like the Toronto worm’s local-only execution [6]

The worm scans local networks for reachable services, especially internal LLM-enabled tools.

Step 4: Abusing insecure LLM apps and plugins

Many internal copilots and agents already have:

Access to internal knowledge bases and vector stores
Permissions to call internal APIs via plugins
Power to run scripts or workflows on users’ behalf [2][3]

JadePuffer abuses this via prompt injection:

Embeds malicious instructions in documents, tickets, or emails processed by LLMs
Tricks agents into calling sensitive APIs or exporting data
Uses plugins as covert exfiltration and propagation channels [2][4]

This matches known risks around plugin abuse, data leakage, and insecure LLM output handling. [2][4]

Step 5: C2-less operation

To evade cloud monitoring, JadePuffer’s LLM components operate locally whenever possible, mirroring the Toronto worm’s C2-less design. [6] This:

Avoids dependence on third-party LLM APIs
Reduces visibility for SOC teams tracking outbound AI calls
Enables polymorphic behavior per host

Step 6: Ransomware and negotiation

In the final phase, JadePuffer:

Encrypts reachable files from the browser foothold and hijacked agents
Selectively exfiltrates sensitive data through compromised plugins/APIs [2][4]
Uses LLMs to draft personalized ransom notes and negotiation scripts, tuned to the victim’s role, language, and culture

🎯 Tailored social engineering can outperform generic ransom notes, using the same generative strengths that power legitimate customer communications. [2][4]

Section takeaway

The JadePuffer storyline moves from browser-based social engineering to local worms and abused enterprise agents, using existing techniques rather than new exploit primitives. [1][2][6]

3. Inside JadePuffer: LLM-Driven Ransomware Architecture and Components

High-level modular design

A realistic JadePuffer design would be modular:

Browser access & encryption module
Local LLM worm & propagation engine
LLM-based reconnaissance & data valuation
LLM-powered negotiation & extortion orchestration [1][6]

Each module can evolve separately, much like modern offensive frameworks.

💡 For defenders, these map to separate telemetry domains: browser, endpoint, network, and LLM application usage. [4]

Browser module

The JavaScript front end would:

Use File System Access API after legitimate consent [1]
Recursively walk granted directories
Generate light previews/checksums for classification
Stream content to WASM or backend encryption

Research shows this can run entirely in-browser on Android, without APKs or exploits. [1] The LLM’s role is classification and prioritization, not crypto itself.

Local worm module

The worm embeds an open-weight LLM (7B–13B), using quantization as in common on-device deployments. [5] The Toronto worm proves open‑weight models can autonomously pick host-specific attacks and spread across a network quickly. [6]

Core behaviors:

Discover neighboring hosts/services
Detect LLM endpoints, internal agents, and automation tools
Generate tailored attack prompts and plans for each target [6][4]

Agent loop for propagation

A conceptual agent loop might look like (non-operational, no exploit logic):

while true:
  context = observe_host_and_network()
  prompt = build_prompt_from(context)

  plan = LLM.generate("Given this environment, list safe-looking actions that increase access:", prompt)

  for step in select_top_steps(plan):
    if violates_safety(step):
      continue
    result = execute(step)
    log(step, result)

  sleep(randomized_interval())

Here, strategic choices—what to scan, where to move, what to exfiltrate—are delegated to a probabilistic model instead of fixed logic. [4][5]

LLM apps as pivots

JadePuffer treats insecure LLM apps as pivot points:

An internal copilot with knowledge-base or SQL access becomes an exfiltration tool. [2]
An automation agent with CRM or ticketing access becomes a large-scale phishing and social engineering engine. [3]
Plugins that call shell commands or internal APIs act as general remote tooling. [2][3]

These map directly to OWASP LLM risks: prompt injection, insecure tool use, data leakage. [3][4]

Operational security via LLMs

Attackers can also apply LLMs to their own OPSEC:

Generating polymorphic loader code to evade signatures
Randomizing file names and encryption patterns to avoid heuristics
Drafting benign-looking log entries or messages to mislead analysts [4]

⚡ The adaptive text and code generation defenders use for IR can also power dynamic evasion when misused. [4]

Section takeaway

JadePuffer shows how discovery, planning, prioritization, and social engineering can be offloaded to LLMs, leaving mainly low-level execution as traditional code. [4][6]

4. Mapping JadePuffer Against OWASP LLM Top 10 and Known Risks

OWASP Top 10 for LLM Applications summarizes real-world LLM vulnerabilities. [3][4] JadePuffer spans several of them.

Prompt injection

JadePuffer hijacks internal agents via malicious prompts in data they process: tickets, docs, chats, or emails. [2] Attacker-controlled content injects override instructions, causing models to ignore policies—exactly the prompt injection risk. [3]

⚠️ OWASP explicitly warns that LLMs can be tricked into unintended actions or safeguard bypass via attacker-controlled input. [3][4]

Insecure output handling & data leakage

Once compromised, agents may:

Return internal documents directly to untrusted channels
Execute privileged API calls solely based on model outputs
Paste sensitive data into external systems without checks [2][4]

This matches OWASP concerns about insecure output handling and uncontrolled data flows. [3][4]

Data poisoning in fine-tuning and customization

Organizations often fine-tune or adapt models on internal data. If attackers can poison that data—via documents, logs, or code—they can nudge model behavior toward misclassification, lax policies, or hidden backdoors. [3][5] OWASP highlights such poisoning as a key LLM-specific threat. [4]

Model theft and open-weight abuse

JadePuffer’s use of downloadable open-weight models reflects OWASP fears that adversaries can steal and repurpose models:

Retrain them on offensive corpora
Embed them in malware frameworks like JadePuffer
Share them widely at low cost [4][5]

Non-traditional vectors

Browser-only ransomware and AI worms attack surfaces often missed in legacy appsec:

Browser APIs such as File System Access [1]
LLM agents and orchestration frameworks [2][4]

Modern guidance stresses that these are frequently absent from threat models, code reviews, and governance. [3][4]

💡 JadePuffer acts as a stress test: it forces organizations to ask whether advanced browser features and LLM components are truly covered by their security program. [2][3]

Section takeaway

Mapping JadePuffer onto OWASP LLM Top 10 turns an abstract framework into a concrete playbook, helping teams prioritize defenses. [3][4]

5. Defensive Engineering: Hardening Browsers, LLM Apps, and Infrastructure Against JadePuffer

Browser and endpoint layer

Initial defenses live at the edge:

Review and restrict File System Access API usage, especially on Android Chrome. [1]
Improve permission dialogs to clearly convey directory-level risks. [1]
Monitor browsers/endpoints for abnormal bursts of file modification.

⚠️ Even without binaries, large, rapid I/O on photo directories from browser processes is a strong signal. [1]

LLM security program

Security and product teams should build a dedicated LLM security track covering:

Risk mapping across prompts, tools, and agents
Guardrails and filtering on prompts and outputs
Monitoring of LLM usage and tool/plugin invocations
Incident runbooks specific to LLM and agent compromise [2]

Guidance stresses LLMs need governance beyond standard API/web controls. [2][4]

Integrating OWASP LLM Top 10 into SDLC

Make OWASP LLM Top 10 a standard checklist for any AI feature: [3][4]

For every new agent/plugin, explicitly analyze prompt injection and exfiltration paths.
For every fine-tuning pipeline, include poisoning and leakage defenses.
Treat all LLM outputs consumed by code as untrusted data.

Architectural patterns for safer agents

Concrete patterns include:

Strict tool/plugin whitelisting: agents can call only vetted functions. [2][5]
Output validation layers: apply policy filters/sanity checks before execution. [4][5]
Zero-trust internal API access: narrow tokens per agent, not per environment. [2]

A financial-firm security lead reported that adding an output-validation proxy in front of their internal copilot cut risky tool invocations by ~60% in red-team tests. [4]

Monitoring for local LLM abuse

Endpoints should be watched for:

CPU/GPU spikes consistent with local LLM inference
Appearance of large model weight files on non-dev machines
Agent frameworks showing autonomous, script-like behavior [6]

These indicators mirror AI worm traits and can trigger deeper investigation. [6]

Using LLMs defensively

LLMs can also strengthen defense:

Summarizing and clustering large security log volumes
Flagging suspicious agent/tool usage patterns
Supporting triage and investigation via natural-language queries [4]

📊 Defensive LLM deployments themselves must be protected; if compromised, they reveal monitoring gaps and internal playbooks. [4]

Section takeaway

Mitigating JadePuffer demands browser controls, LLM-aware architecture, and telemetry tuned to local model use, all guided by OWASP LLM Top 10. [2][3][4]

6. Building JadePuffer-Inspired Red-Team Scenarios and Simulations

Browser-only ransomware labs

Create controlled labs to mirror browser-only ransomware against synthetic photo libraries: [1]

Deploy a benign test app using File System Access
Observe how users react to permission prompts
Measure how quickly EDR/browser telemetry detects large file changes

This helps UX, product, and security teams understand user behavior and detection gaps. [1]

Simulating AI worms

On isolated networks, reproduce a Toronto-style architecture: [6]

Local open-weight LLM in quantized form
Agent loop exploring and “attacking” lab services
Instrumentation for lateral movement and dwell-time metrics

Such labs expose blind spots in detecting autonomous agents vs traditional scripted malware. [6]

⚠️ Keep all experiments in controlled non-production environments, with safe, non-exploit payloads.

OWASP LLM attack patterns in exercises

Bake OWASP LLM Top 10 scenarios into:

Tabletop exercises with product and ML teams
Automated red-team scripts focused on internal LLM apps
Game days simulating agent misuse and data theft [3][4]

Include prompt injection, plugin abuse, and data exfiltration to test controls and escalation. [2][3]

Cross-team collaboration

Effective defense requires joint effort:

Security teams define detection and response for LLM misuse. [2]
AI platform teams provide logs, tracing, and policy hooks.
Product teams design safer agent workflows and user interfaces.

Using JadePuffer-style scenarios as a shared reference turns an abstract threat into concrete, testable exercises and drives a consistent LLM security posture.

Conclusion

JadePuffer represents a plausible “all-LLM” ransomware kill chain: browser-only file access, local LLM worms, and insecure enterprise agents chained into a single attack. [1][2][6] It operationalizes the risks captured in OWASP’s LLM Top 10 and demonstrates how much of modern malware—discovery, planning, social engineering—can be delegated to language models. [3][4][6]

Defenders should respond by: