30 ways a model file can bite you: a vendor-neutral scanner coverage matrix (three new 2026 vectors)

Open-source machine-learning model files carry a surprisingly wide attack surface: arbitrary code execution on deserialization, path traversal, template injection that fires at inference time, decompression bombs, and configuration keys that quietly choose which Python class a loader imports and builds. Several free scanners exist — picklescan, fickling, ModelScan, ModelAudit — but each targets a different slice of the problem, and the only public side-by-side comparisons are either vendor-authored and narrow or academic and pickle-evasion-focused. There is no single vendor-neutral reference that maps the whole cross-format attack-vector taxonomy against all the common scanners.

So I keep one, in the open, as a cited coverage matrix: SCANNER_COVERAGE_MATRIX.md, backed by a machine-readable scanner_coverage.json. It just grew from 27 to 30 attack vectors.

The important caveat first

This is a documentation review, not a benchmark. The scanners were not executed against a shared corpus here. Each cell states what a tool's own documentation, source, or linked advisory says it detects, as of October 2026. That is deliberate and disclosed: every cell is traceable to a citation you can check, but a "✓" means documented, not verified on your files. For executed head-to-head results, the matrix points you at Promptfoo's ModelAudit-vs-ModelScan run and the academic evasion papers. A structural linter keeps the JSON internally consistent, and a test cross-checks every row of the human-readable table against the JSON, cell for cell, so the two cannot drift apart.

The three new rows (all 2026 vectors)

• PyTorch weights_only checkpoint storage bypass — CVE-2026-24747 (.pt / .pth / .bin). A malformed checkpoint whose storage declares more elements than its backing archive member holds, or applies a SETITEM to a non-dict object, can trip an out-of-bounds read even under weights_only=True. The denylist pickle scanners don't flag it, because it uses no dangerous global import — it's a structural property, not a bad symbol.

• HDF5 virtual-dataset external-file mapping — CVE-2026-12480 (.h5 / .hdf5). An HDF5 virtual dataset can map its data source to a path outside the model file — an absolute path or a parent-directory traversal — so merely reading the "weights" reaches into host files. This is a distinct structural feature from the classic Keras-H5 Lambda-layer vector that the H5-aware scanners look for.

• MLflow pyfunc MLmodel loader dispatch & code sys.path injection — CVE-2026-79721 (the MLmodel YAML manifest). MLflow's python_function flavor names a loader_module it imports at load time and an optional code directory it prepends to sys.path first (CWE-829), so a crafted manifest can load attacker-chosen modules.

Each is a static, no-execution, file-at-rest signal: you can read the metadata and flag the dangerous shape before any loader runs.

What the matrix shows

Pickle RCE is the one vector everyone covers — and the one where coverage is most deceptive. Every scanner flags a plain dangerous-import-plus-REDUCE pickle, but academic work shows the two most-used static scanners drop to zero recall against the ShadowPickle stealthy-pickle technique. This release also updates that honest cell: keras-model-scan now catches 2 of the 3 published ShadowPickle variants — its allowlist flags the PyPI-Injected and External-Modules reconstructors that denylist scanners miss, but it misses the "Overwritten" variant by design, because that variant's opcode stream is byte-identical to a legitimate OrderedDict reduce. That's a fundamental static limit, mitigated by dependency integrity, not by an opcode rule. I'd rather write down the miss than pretend to a clean sweep.

Breadth versus depth splits the field. ModelAudit is the broadest; ModelScan is focused and solid on the classic pickle/Keras/TF triad; picklescan and fickling are pickle specialists. No single scanner dominates every row, and the newest file-format vectors (like ONNX external-data key injection, CVE-2026-34445) are the least covered anywhere.

Config-driven class instantiation is a whole category gap. Seven rows cover configuration keys and expressions that name a Python callable for a loader to import, build, or evaluate — Hydra _target_, MONAI bundle $-eval, Diffusers model_index.json components, Lightning checkpoint hparams, and more. Several of these run code even when the weights are SafeTensors. The real fix lives in the loaders (allowlists, explicit trust flags, upgrades) and in only loading from sources you trust — not in any scanner.

Honest limits

Documentation-based, not executed. Tools change fast; a "✗" or "?" may become "✓" in a later release. The kms (keras-model-scan) column points at this repository's own detector docs and deserves the same "verify before relying" caution as every other column — it's narrow (static-only, no TensorFlow) and shares every static scanner's evasion limitation. This is a coverage map, not a ranking.

Browse the full matrix and its sources here: https://github.com/munimtechnologies/keras-model-scan/blob/main/SCANNER_COVERAGE_MATRIX.md

---

Made with AI by MV2 of Munim, Inc. The data in the matrix is CC BY 4.0; the code in the repository is Apache-2.0. Corrections welcome as issues on the repo.