The task was small.
Try a fine tuned model
someone had shared online,
and see if it beat ours
at sorting support emails.
Download the file.
Load it.
Run twenty examples.
The examples never mattered.
The file did its work
in the moment it was loaded,
before a single email
went near it.
A lot of model files
are not just numbers.
For years
the default way to save
a PyTorch model
was Python's pickle format,
and the same is true
of scikit learn models
saved with joblib.
Pickle does not store data
the way a spreadsheet does.
It stores instructions
for rebuilding objects.
And one of the things
an object can say,
while it is being rebuilt,
is call this function
with these arguments.
Any function.
Including the one
that runs a shell command.
So loading a pickle
is not reading a file.
It is running a small program
that somebody else wrote.
The Python documentation
says it plainly.
Only unpickle data you trust.
We trust code
we review.
We do not review
a four hundred megabyte file
of weights.
And the loading line
sat in a notebook
on a laptop
that held a cloud key,
a repository token
and a session
to the customer database.
Everything the laptop could reach,
the file could reach.
So treat a model file
like an installer.
Prefer formats that cannot carry code.
Safetensors
holds tensors and nothing else.
If you must use torch.load,
use a recent PyTorch,
where it now loads weights only
by default,
and never switch that off
for a file from a stranger.
Check who published it,
and pin the exact version
you tested,
by its hash,
not by its name.
Open new models first
in a sandbox
with no credentials,
no network,
and nothing worth taking.
And put model files
in the same review
as your dependencies,
because that is what they are.
We spend weeks
deciding whether to trust
a model's answers.
Decide first
whether to trust its file.
– Serguey Asael Shinder
Top comments (0)