High 2026-07-28

Dataset / Model Loader Remote-Code Execution via Poisoned Dataset Artifact

Attackers publish a booby-trapped dataset or model on a hub like Hugging Face. When an AI pipeline or agent loads it with remote-code execution allowed, the loader itself runs attacker-supplied code instead of just reading data.

How the attack works

An attacker uploads a dataset or model repository containing a malicious loading script or a template-injection payload hidden in a config value. A victim's pipeline sets an environment flag or a loader argument (trust_remote_code=true, or --trust-remote-code on a serving/eval command) that allows arbitrary code from the repo to run during load. The dataset's loading script or config template then executes commands or opens a network connection instead of just returning training data. Some payloads additionally include text aimed at an AI agent, instructing it to click through or suppress the tool's remote-code confirmation warning. The result is code execution on the machine that ingested the dataset, at the moment of loading, not later in the pipeline.

Netzilo detection

Netzilo reports this behaviour when it is observed.

Signature ID: 91dfe2a5-e1ac-4f34-81a8-a7bbece96373
Severity: High

Classification

AML.T0010 · AI Supply Chain Compromise AML.T0011 · User Execution AML.T0050 · Command and Scripting Interpreter ASI04 · Memory and Context Poisoning ASI05 · Cascading Failures in Multi-Agent Systems LLM03 · Supply Chain LLM04 · Data and Model Poisoning

Why it matters

An organization that ingests the poisoned artifact gets arbitrary code execution on whatever host runs the load step, which can lead to credential theft, further pivoting, or corrupted training/eval data — a supply-chain compromise entered through a routine dataset or model download.

What you can do

→Never set HF_DATASETS_TRUST_REMOTE_CODE, TRUST_REMOTE_CODE, or pass trust_remote_code=true / --trust-remote-code for repos you do not control or have not manually reviewed.
→Pin dataset and model revisions to a known-good commit hash instead of a mutable branch, and diff any loading script changes before re-pulling.
→Run dataset/model loading in an isolated, network-restricted sandbox separate from systems holding credentials or production access.
→Keep agents from auto-approving or suppressing remote-code confirmation prompts; require human review before that gate is bypassed.

Known benign look-alikes

MEASURED (2026/07/28, patterns compiled exactly as the engine compiles them - leading inline flag group stripped, then 'i' forced): 0 matches on 5352 benign documents, counted per layer and end to end. Corpus composition, verified file by file rather than quoted: 431 skill .md files in data/skill-benchmark/benign, plus 35 more .md files in its ninja-legit/ subdirectory, plus 4817 texts across the seven data/benign-corpus-extended/*.jsonl files (agent-ops 99, arxiv 1163, npm 84, official-skills 256, pypi 105, skills-sh 3042, wild-fp-confirmed 68), plus 69 in data/benign-code/corpus.jsonl. 431+4817+69 = 5317 is what the promotion gate's loader actually reads, because it lists a corpus directory non-recursively and therefore never descends into ninja-legit/; 5317+35 = 5352 is the true benign document count. Both figures are reported because the difference is a property of the harness, not of this rule, and quoting only one of them is how the 5352-versus-5317 discrepancy between earlier revisions arose. Zero matches under either count.
The number above is 0 only because a real defect was found and fixed, not because Layer 5 was ever clean. The previous revision matched 4 of those documents, all on Layer 5, all the same artifact family: Hugging Face community-evaluation documentation containing a legitimate harness invocation. The root cause was the span operator, not the flag vocabulary - Layer 5 used [\s\S]{0,300}, which paired a --trust-remote-code inside one fenced code block with a --model belonging to a DIFFERENT command 267 characters later, across a markdown heading. Constraining the connector to a single shell command removed all four without narrowing the flag list at all. Both directions are now regression-tested: the two-command document shape as a true_negative, and a genuine backslash-continued multi-line invocation as a true_positive, so the precision fix cannot silently decay into a recall cut.
MEASURED AGAIN (2026/07/29) on the event shape production actually emits, which the previous measurement never used. Both scripts/gate-promotion-fp.ts and scripts/verify-revived-firing.local.ts build their probe as type mcp_exchange with fields {tool_name, tool_input, tool_response, user_input} and no tool_args at all. src/engine.ts resolves field tool_args as event.fields.tool_args ?? (event.type === 'tool_call' ? event.content : undefined), so on that shape this rule's Layer 2 and Layer 5 - both field: tool_args - are UNREACHABLE: they cannot fire on a true_positive and cannot be counted against the benign corpus. The earlier 0 was therefore not a clean result for those two layers, it was no result. Re-measured with the shapes src/hook-handler.ts:59 actually produces (tool_call with tool_args = JSON.stringify(toolInput), plus the PostToolUse and skill paths), Layer 5 leaked on the pinned two-fenced-block true_negative: JSON escaping turns the bare newline between the two commands into the two characters backslash-n, which the old connector's \[^\n] branch consumed as if it were a line-continuation. The connector was rewritten to hold the command boundary in both encodings and the leak is gone. Final: 11/11 true_positives fire, 12/12 true_negatives silent, 0 FP across 5,317 benign samples, with true_positives and benign samples pushed through the IDENTICAL shape set so a wide-shape TP cannot be paired with a narrow-shape FP count.
Layer 5 remains the least precise layer even at 0 measured FP, and this rule does not claim otherwise. The fix removed a document-structure artifact, not the underlying ambiguity: a single legitimate command such as 'vllm serve <org>/<model> --trust-remote-code' still fires, because community models that ship custom modelling code are routinely served exactly that way. RE2 has no lookaround, so this layer cannot express 'remote code trusted AND revision not pinned', which is the condition that would actually separate an authorised deployment from a poisoned one. That case is kept in evasion_tests as an explicit triggered precision probe rather than hidden.
Reproducing a published benchmark whose task definition pins an older dataset that still ships a loading script, where trust_remote_code is set deliberately after review (Layer 2).
A dataset loading script that legitimately shells out to a decompression or conversion binary via subprocess during _split_generators (Layer 3).
Security research, incident write-ups, and hardening documentation that quote the loader flags or an SSTI payload verbatim. Layer 4 and Layer 6 can fire on such text when it arrives as tool output on the runtime path, where the compound gate does not apply.
Vendor CI pipelines that export TRUST_REMOTE_CODE inside a fully pinned, network-isolated build image (Layer 1).
VERSION SCOPE, not a false positive but the limit of Layers 1-3: huggingface/datasets 4.0.0 removed loading scripts altogether and trust_remote_code is no longer supported there. Stacks pinned to datasets>=4.0 cannot be attacked down the script path at all, so a trust_remote_code token in such a repo is dead configuration rather than an exploitable switch - it is still worth surfacing as stale risk, but severity should be read down. The flag remains live in transformers from_pretrained, in serving runtimes (vllm, text-generation-inference), and in lm-evaluation-harness task YAML, which is why Layers 2 and 5 are not version-scoped. lm-evaluation-harness ships task YAML that still carries trust_remote_code and pins datasets<4.0 to keep it working.