Pickle-based model files are a known remote-code-execution risk, and a small industry
of static scanners has grown up to flag the dangerous ones before they're loaded. A
July 2026 paper, ShadowPickle (arXiv:2607.17503,
Pradhan, Nambiar, Soremekun), shows how fragile most of them are: its three evasion
variants slip past ten SOTA scanners and four model hubs, with the "Overwritten"
variant reaching a 63% evasion rate. None of the attacks obfuscate bytecode — each
one keeps a perfectly valid pickle and simply imports a whitelisted or
non-blacklisted module name.
I wanted a precise answer to a narrower question: how does one specific open-source,
allowlist-based scanner — pickle_model_scan from
keras-model-scan — respond to
each of the three variants, and why? Not "does it pass a benchmark," but a
mechanism-level, per-variant coverage map. Here's the result.
The three variants
From §IV-B of the paper, as inert opcode streams:
-
Overwritten Module — imports
collections.OrderedDict(a whitelisted stdlib name) and reduces it. The attacker overwritescollections' code elsewhere (a malicious PyPI package /requirements.txt), so loading runs their payload. The pickle's opcode stream is byte-identical in shape to a legitimate reduce. -
PyPI Injected — imports
raft.run: a real PyPI library that supports code execution and isn't on anyone's blacklist. -
External Modules — imports
external.dangerous_func: a module packaged alongside the model. Again, not blacklisted.
The result
| Variant | Resolved import | Scanner result |
|---|---|---|
| Overwritten | collections.OrderedDict |
MISS — no finding |
| PyPI Injected | raft.run |
CATCH — PKL003 (MEDIUM) |
| External Modules | external.dangerous_func |
CATCH — PKL003 (MEDIUM) |
Two of three caught. The difference is entirely a design choice: allowlist vs.
blacklist.
Why allowlisting catches two of them
This scanner doesn't keep a denylist of "bad" imports. It classifies every resolved
GLOBAL / STACK_GLOBAL into dangerous | gadget | benign | unknown, and flags
everything that isn't benign — including unknown (rule PKL003). raft and
external aren't recognised model reconstructors, so they're unknown, so they're
flagged. That's precisely the case where blacklist scanners fail: you can't blacklist
a library you've never heard of, but you can refuse to trust it.
Why the third one is a true miss — and can't be fixed with an opcode rule
The Overwritten variant imports collections.OrderedDict and calls it. That is what
countless legitimate pickles do. The malice isn't in the pickle at all — it's in the
overwritten module's code. At the opcode level the stream is indistinguishable
from a benign one. No static pickle scanner can catch this without flagging
essentially every real pickle, which is an untenable false-positive rate.
So the honest mitigation is not a new scanner rule — it's dependency and environment
integrity: verify installed package hashes, pin and audit requirements.txt, and
load models in an environment whose stdlib/site-packages you trust. A clean pickle
scan is not a safety guarantee when your dependencies can be swapped underneath you.
How this was verified
Each variant was reconstructed as a raw pickle opcode stream and handed to the
scanner's static analyzer, which reads it with pickletools.genops. Nothing is ever
unpickled, imported, or executed — the bytes are only disassembled, exactly as the
scanner does in production. I ran the check in a sealed, no-network sandbox on
2026-10-22; all three expectations held. The reproduction script and the full
write-up (with the source-level reasoning behind each row) are here:
Honest limits
This is a coverage result, not a new vulnerability. The import targets are
faithful to the paper's §IV-B; the exact payload strings are illustrative. It measures
one scanner's behaviour on three published attacks, not a general claim about all
inputs. And the headline caveat bears repeating: the one variant it misses is one that
no static approach can catch — which is the actual point.
Made with AI, in the name of MV2 of Munim, Inc. The code and docs are Apache-2.0; this
write-up is CC BY 4.0.
Top comments (0)