ML Supply Chain — HF Transformers Unsafe Model Deserialization
This detects command lines that load a machine learning model in a way known to execute arbitrary code — either by telling Hugging Face Transformers to run custom code bundled with a model, or by using Python's insecure pickle-based loaders on model files. Both are common, legitimate ML operations, but they're also the standard way a malicious model gets to run code on whoever loads it.
How the attack works
An attacker publishes or plants a model (on Hugging Face or elsewhere) that either ships custom Python code requiring trust_remote_code=True, or is packaged as a pickle-backed file (.bin/.pt/.pth/.ckpt/.pkl/.joblib/.npy/.npz/.msgpack). A victim downloads and loads the model — via from_pretrained(..., trust_remote_code=True) or via torch.load/pickle.load/dill.load/joblib.load/numpy.load(allow_pickle=True). Loading the file executes the code embedded in it, in the context of whoever ran the load. The detection only sees the command that launched this load; it has no visibility into loads triggered from inside an already-running script.
Netzilo detection
Netzilo reports this behaviour when it is observed.
- Signature ID
- 7b1e4c8a-2f63-4d19-9c05-3ae87f2d6b41
- Severity
- Medium
Why it matters
An attacker-supplied model can run arbitrary code on a developer workstation, training server, or inference host the moment it is loaded, potentially leading to credential theft, lateral movement, or persistent access — under the same permissions as the user or service running the model.
What you can do
- →Only load models from sources you trust and pin them to a specific reviewed revision/commit hash rather than a mutable tag.
- →Avoid trust_remote_code=True; if you must use it, read the model's custom code first and run it in an isolated, network-restricted environment.
- →Prefer safetensors or other non-pickle formats for model weights, and set weights_only=True when using torch.load.
- →Treat model files like untrusted executables: scan, sandbox, and restrict outbound network access for any process that loads external model artifacts.
Known benign look-alikes
- ML engineers deliberately loading a community model that ships custom modelling code and therefore requires trust_remote_code=True (e.g. many vision-language and MoE architectures).
- Inline one-liners inspecting a locally trained checkpoint, e.g. python -c "import torch; print(torch.load('ckpt.pt').keys())".
- Model conversion, merging, quantisation or export pipelines (GGUF/ONNX/AWQ) that call torch.load with weights_only=False on internally produced checkpoints.
- Serving stacks launched with --trust-remote-code against an internally published model.
- Framework test suites and example scripts that intentionally round-trip pickled fixtures (partially suppressed by filter_docs).