AI Model Security Training: What a Platform Must Teach

A PyTorch checkpoint is not data. It is a program, and torch.load is the interpreter.

That sentence is the entire subject, and most training marketed as AI model security never gets to it. The syllabus goes prompt injection, jailbreaks, maybe a RAG poisoning lab, and stops. Those attacks target a model's behavior. None of them address the more basic question of whether the file you loaded onto a GPU box with cloud credentials attached was doing something other than defining tensors.

Start With the Vulnerability Class

The failure mode is CWE-502, deserialization of untrusted data, applied to machine learning artifacts. A .pt or .bin checkpoint is a zip archive with a Python pickle inside, and unpickling executes opcodes that can import and call arbitrary functions.

CVE-2025-24357 is the clean teaching example. vLLM's hf_model_weights_iterator in weight_utils.py loaded checkpoints downloaded from a model hub using torch.load, with weights_only left at its default of False. A malicious checkpoint got code execution on the inference host. CVSS 7.5, fixed in v0.7.0. It was not an exotic bug. It was one keyword argument.

import torch

# Runs the pickle VM. Arbitrary code, on the box holding your model weights.
state = torch.load("downloaded.bin")

# Tensors only. No callable imports, no REDUCE opcode.
state = torch.load("downloaded.bin", weights_only=True)

PyTorch flipped that default to True in 2.6, which protects teams that upgraded and does nothing for the pinned 2.3 environment running in production. The same class reaches further up the stack: CVE-2024-11393 is a deserialization RCE in Hugging Face Transformers reached through MaskFormer model file parsing, CVSS 8.8, reported through the Zero Day Initiative as ZDI-24-1514.

Training that teaches this well spends its time on the inspection step, not the vulnerability trivia:

# A .pt/.bin checkpoint is a zip archive. The pickle lives inside it.
unzip -o model.bin -d unpacked/
python -m pickletools unpacked/*/data.pkl | grep -E "GLOBAL|STACK_GLOBAL|REDUCE"

A checkpoint that only defines tensors has no reason to import posix or builtins.eval. GLOBAL paired with REDUCE is a callable being resolved and invoked during load, and seeing that output once teaches more than an hour of slides.

Map Findings to a Taxonomy or Nobody Acts on Them

"We downloaded a sketchy model" is not a finding a security organization can route. The same observation expressed as AML.T0010, AI Supply Chain Compromise, with the Model sub-technique, is initial access with an ID, an owner, and a place in a report. Malicious code inside the artifact is AML.T0018.002, Embed Malware, under Manipulate AI Model.

We teach adversarial attacks against models inside Applied Data Science and AI for Cybersecurity, and the taxonomy mapping travels with the technique for exactly this reason. A red team that reports in MITRE ATLAS IDs gets remediation. A red team that reports in prose gets a thread nobody closes.

The Four Blocks a Curriculum Needs

• Artifact triage. Which formats execute on load (pickle, .pt, .bin, joblib, Keras H5 with Lambda layers) and which do not (safetensors, GGUF, ONNX with care). Hands-on inspection with pickletools, fickling, and modelscan.
• Provenance. Hash and sign what you promote, mirror approved models into an internal registry, and pin by digest rather than by tag. A deploy step that pulls latest from a public hub is an unauthenticated code path into production. Defense contractors already run this program for software and can usually extend it to weights, which we wrote about in the context of defense industrial base teams.
• Containment. Assume the load executes. Inference workers run non-root, without cloud instance credentials, with egress restricted to the endpoints they need. This is ordinary infrastructure hardening and it is the control that survives a scanner miss.
• Detection. What the load looks like in telemetry: a Python process spawning a shell or resolving an unexpected domain shortly after a model file lands on disk. Sysmon Event ID 1 with parent-child rarity scoring, and outbound connections on Event ID 3, both mapped to T1059.

The detection block is the one platforms skip, and it is the one that matters most to a SOC. Attacking a model is a red-team skill. Noticing that someone attacked yours is a detection engineering skill, and they are taught by different people.

What Scanning Will Not Fix

Pickle scanners are heuristic. Opcode allowlists get defeated by indirection, and a determined author can express a payload in ways a static pass does not flag. Anyone selling a scanner as the answer is selling the wrong thing. The durable fixes are format migration and provenance, both of which are engineering programs rather than course modules.

This training also does not help much if you consume models only through a hosted API. Then the artifact risk belongs to the provider, and your work is procurement: ask how they verify weights, and move on to the application layer where your actual exposure lives.

And it does not cover the behavioral attacks. Those are a separate discipline with separate labs, covered in what AI red-teaming actually involves and RAG poisoning and jailbreaking.

Testing the Claim

Before buying, ask for one thing: a lab that hands you a malicious checkpoint and requires you to catch it before it loads. A platform that has built that lab has thought about this subject. A platform that offers a video module titled "Model Security" and a quiz has not.

Two follow-ups worth asking. Do the labs run with the network cable pulled, since an exercise that reaches a public model hub dies on a managed corporate laptop. And does the curriculum end at findings or continue into detections, because a team that can only attack leaves the SOC exactly where it started.

Our own take on evaluating this category is on the AI-powered security training platforms page, and the adversarial half of the work is the subject of the AI Red-Teaming course.