Medium

Suspicious Textual-Inversion Embedding Fetched and Placed in Embeddings Folder

This detects a process that references a textual-inversion or negative-embedding file (by extension, known download source, or the terms 'textual inversion'/'negative embedding') and then writes a file into a folder path containing 'embeddings' — the same folder Stable Diffusion webuis like AUTOMATIC1111 load these files from. It flags the acquisition-and-placement pattern of a supply-chain attack against image-generation models, not a confirmed malicious embedding.

How the attack works

An attacker (or a compromised script) fetches a textual-inversion embedding or negative-embedding file, often from a known distribution platform or with a filename/extension typical of these files. That same process then writes the file into a directory whose path includes an 'embeddings' segment — the location where tools like AUTOMATIC1111 automatically load such files for use in image generation. If the embedding is crafted maliciously, it can alter generated output or otherwise corrupt the model's behavior once loaded. The rule catches only the download-and-place steps; it cannot see whether the embedding is actually loaded or what it does afterward.

Netzilo detection

Netzilo reports this behaviour when it is observed.

Signature ID
e2f4a716-8c39-4f1d-9a2b-6d4e8f1c3b7a
Severity
Medium

Why it matters

An organization running Stable Diffusion webuis could have generation results silently altered or degraded by a poisoned embedding, without any indication until output is inspected. This is a supply-chain risk affecting model output integrity, not data theft or system compromise directly.

What you can do

  • Review the flagged file's source and content before allowing it to be used in generation, especially if it did not come from a vetted internal source.
  • Restrict write access to the embeddings folder to trusted admin accounts or deployment processes only.
  • Maintain an allowlist of approved embedding/LoRA sources and block ad hoc downloads from unverified platforms.
  • Treat this signal as an audit trigger, not proof of compromise — confirm manually before removing or blocking anything, since legitimate manual downloads and installer scripts can trigger it too.

Known benign look-alikes

  • A user manually and legitimately downloading a real textual-inversion embedding, LoRA, or hypernetwork from Civitai or a HuggingFace dataset with curl/wget/git and placing it into their own webui embeddings folder
  • First-run or install scripts for AUTOMATIC1111 / ComfyUI / InvokeAI that pre-populate the embeddings folder with bundled community embeddings during setup
  • A Python environment or package installer (pip/conda) whose command line or target path incidentally contains the substring "embedding" or "embeddings" (e.g. an NLP library's site-packages directory) with no connection to Stable Diffusion
  • Generic .bin/.pt/.safetensors artifact installs (model checkpoints, tokenizer files) unrelated to textual inversion that happen to match the extension indicators

Related threats