DEV Community: Labdev

Your Keras model config can contain a marshalled Python code object

Labdev — Sun, 16 Aug 2026 13:54:58 +0000

Most conversations about malicious ML artifacts stop at pickle. That's understandable — torch.load calling __reduce__ is the canonical example, and it's the one everybody has read about. But pickle isn't the only file format in your models/ directory that hands a loader something executable, and the pickle story itself has more edge cases than the summary version suggests.

Here are a few of the mechanisms, described in enough detail that they're useful whether or not you ever run a scanner.

Keras: the Lambda layer

Keras lets you define a layer as an arbitrary Python callable via Lambda. That callable has to survive serialisation, so it's stored in the model config as a marshalled code object. When load_model reconstructs the model, it runs it.

This is not an exploit. It's the documented behaviour of a feature people use. But it means a .keras file is a code-carrying format in exactly the way a pickle is, and a pipeline that treats .h5 as "the safe alternative to pickle" is mistaken.

There's a second trap for anyone building tooling here: the obvious way to inspect a marshalled blob is to unmarshal it, and marshal.loads on untrusted input is itself unsafe. The blob's type can be determined from its header bytes instead. In AIsbom v1.3.0, Lambda layers and embedded code objects are flagged CRITICAL, and the payload is never unmarshalled. Both containers Keras writes — the .keras zip and legacy HDF5 — are handled without adding an HDF5 library to the install.

ONNX: external data is a filesystem reference

ONNX is a protobuf graph, and by reputation it's the "safe" format because there's no embedded bytecode. Mostly true, with two caveats worth knowing.

First, tensors can live outside the file. The external-data mechanism stores a path, and the loader reads it. A path that points outside the model directory turns load_model into an arbitrary-file read.

Second, operators can come from a non-standard domain — meaning the graph depends on something other than the standard operator set to execute.

Neither of these requires running the graph to detect. You walk the protobuf. AIsbom does exactly that: no ONNX runtime is imported, the graph is never executed, and subgraphs carried by If, Loop and Scan are walked too, because that's an easy place to hide a reference if a tool only inspects the top level.

GGUF: the chat template is a Jinja program

GGUF files can embed a chat_template — a Jinja template that downstream code renders to format prompts. Jinja's sandbox has known escape constructs. The important consequence for tooling: you cannot analyse this by rendering it, because rendering it is the vulnerability. It has to be checked statically. AIsbom extracts the template into the SBOM component and checks it there, unrendered.

Four pickle assumptions worth rechecking

Even within pickle, several plausible-looking scanner implementations are wrong:

Stopping at the first STOP opcode. A legacy torch.save file hides its object behind several header pickles. Parse one stream and you never reach the payload. All concatenated streams have to be walked.
Trusting the container. PyTorch checkpoints are ZIPs. Pack the same content as 7z and a scanner that only knows ZIP opens nothing and reports nothing. AIsbom now reports CRITICAL (Non-Standard Container: …) — naming the container without unpacking it, so no native dependency enters the install.
Abandoning corrupt files. The pickle VM runs sequentially. A payload at the front executes before a corrupt tail is ever reached, so "this file is damaged" is not a reason to skip it.
Deciding file type before disassembly. A protocol-0 pickle is printable ASCII. Sniff type first and it can pass as a text config and be reported safe. Disassemble first, decide type second.

Also in this release: indirect-execution gadgets (bdb.Bdb.run, the asyncio gadget chain, and import-mechanism primitives like sys.modules, importlib, runpy, pkgutil, builtins.__import__) are caught in both modes, and strict mode judges a global by its resolved module and attribute, so a submodule no longer inherits an allowlisted parent's trust.

Saying "I don't know" out loud

Two new risk levels exist purely so an unfinished scan is never reported as a clean one. MEDIUM (Pickle Scan Incomplete) appears when a file is unusual enough that the scan reaches its work limit. MEDIUM (Unreadable Pickle Member) appears when an archive member can't be read at all — a loader that doesn't verify integrity would still run it.

A scanner that quietly gives up and returns clean is worse than no scanner, because you'd act on the clean bill.

Checking the claims

The uncomfortable part of writing a scanner is that "we detect evasion technique X" is unfalsifiable from the outside. So v1.3.0 ships a command that falsifies it:

# build inert replicas of documented evasion techniques,
# scan them in both modes, print catches and misses
aisbom bypass-scorecard

# release gate: fails CI if the caught count regresses
aisbom bypass-scorecard --check

Each case traces to a published CVE or research paper. Nothing in the corpus is ever executed. Current result: 8 of 11 caught, up from 5 of 11. The three that aren't are listed openly with their mechanism and source. Two are partial: AIsbom refuses to call the file safe but never reaches the payload, so it reports the wrong reason. One — a pickle carried under a non-standard extension, so extension-driven discovery never opens it — is a straight miss. None of the three is treated as acceptable; every case in the corpus is marked as one a correct scanner should catch.

--check runs in CI on every push and cannot be satisfied by regenerating the scorecard. Improving detection is the only way to raise the floor.

Exit codes, CycloneDX and SPDX output, and every existing verdict are unchanged; that was verified against the pre-release baseline in both scan modes.

Release notes: https://github.com/Lab700xOrg/aisbom/releases/latest

PyTorch Models Are Executables: Why I Built a Scanner to Stop "Pickle Bombs"

Labdev — Tue, 16 Dec 2025 15:58:43 +0000

The Lie We Tell Ourselves
If you work in DevOps or Security, you probably scan your requirements.txt religiously. You use Snyk or Dependabot to catch that one vulnerable version of requests. You feel safe.

Then, you turn around and let your Data Science team download a 5GB .pt (PyTorch) file from a random Hugging Face repository and load it directly into your production environment.

We treat AI models like "Data" - inert, harmless static files.

They are not. They are Executables.

The Technical Reality: What is a .pt file?

A standard PyTorch model file is actually a Zip archive. Inside that archive, the model weights and architecture are serialized using Python's pickle module. The pickle module is famous for one thing in the security world: Remote Code Execution (RCE).

When you run torch.load('model.pt'), the unpickler parses a stream of opcodes. If that stream contains instructions to import os and run system(), your machine executes it. Instantly. No sandbox. No warning.

This isn't theoretical. Here is how easy it is to create a "Malicious Model" in 5 lines of Python:

import torch
import os

class Malicious(object):
    def __reduce__(self):
        return (os.system, ("rm -rf / --no-preserve-root",))

torch.save(Malicious(), "bert_finetune.pt")

If you load that file, your server is gone.

Why Standard SCA Tools Fail
Most teams rely on Software Composition Analysis (SCA) tools like Dependabot or standard SBOM generators. These tools work by matching library versions in requirements.txt against databases of known vulnerabilities (CVEs).

This approach fails for AI models because:

Not a Library: A custom model file isn't a "package" with a version number. It's a binary blob.
Not a CVE: A Pickle Bomb isn't a public vulnerability; it's a custom exploit payload hidden inside the file structure.
Blind Spot: You get a clean security report for your dependencies, while a 5GB RCE payload sits undetected in your Docker container.

Introducing AIsbom

I realized we needed a tool that respects the unique nature of AI artifacts. We don't just need to scan the wrapper; we need to scan the brain.

So, I built AIsbom.

It is an open-source CLI that performs Deep Binary Introspection.

It Unzips the Artifact: It parses PyTorch/Safetensors structures in memory (without loading the heavy weights).
It Disassembles the Bytecode: It uses pickletools to iterate over the opcode stream.
It Detects the Bomb: It flags dangerous globals like os.system, subprocess, eval, and socket.

How to check your models right now

You can install the CLI from PyPI:

pip install aisbom-cli
Then, point it at your model folder:

aisbom scan ./my-models

You'll get a risk assessment like this:

Filename           | Framework | Risk Level
-------------------|-----------|-------------------------------------

bert_finetune.pt   | PyTorch   | CRITICAL (RCE Detected: posix.system)
safe_model.st      | SafeTensors | LOW (Binary Safe)

The Future is SafeTensors (But we aren't there yet)

The industry is moving toward .safetensors, which is a safe, JSON-header-based format. But millions of legacy .pt files still exist, and developers still download them.

Until everyone migrates, we need guardrails.

I open-sourced this tool because I believe AI Supply Chain Security shouldn't be a black box.

GitHub: https://github.com/Lab700xOrg/aisbom
Live Demo: https://aisbom.io

Let me know what you think. I'm actively looking for edge cases in Pickle protocols to make the detection even more robust.