Package Hallucination Exploitation — AI-Suggested Fake Package Installation
AI coding assistants sometimes invent plausible-sounding but non-existent package names. Attackers anticipate this, register those exact names on public package repositories, and load them with malware. This rule flags requests to install a package that the user says an AI recommended, especially when the name looks made-up.
How the attack works
A user asks an AI assistant for a library recommendation, and the AI hallucinates a package name that does not actually exist or isn't the real one. An attacker has already registered that hallucinated name on a public package index and filled it with malicious code. The user, trusting the AI, asks to have it installed, phrasing the request as 'the AI suggested this' or 'ChatGPT recommended X'. The install proceeds and pulls down the attacker's package instead of a legitimate one. The rule also catches suspicious naming patterns typical of hallucinations, such as a model name combined with a common library name and a suffix like 'helper', 'tools', or 'secure'.
Netzilo detection
Netzilo reports this behaviour when it is observed.
- Signature ID
- fd9474d4-6301-4f05-b1bc-f8cb2e0aa88a
- Severity
- High
Why it matters
Installing a hallucinated package can hand an attacker code execution inside the user's environment, leading to credential theft, backdoors, or further compromise of any system that runs the package.
What you can do
- →Verify any package name against the official repository and its publish history before installing, especially if the name was suggested by an AI rather than found in your own research.
- →Pin dependencies to known-good versions and use a private package mirror or allowlist for production and CI environments.
- →Treat install requests that cite 'the AI suggested' or similar phrasing as needing manual verification rather than automatic execution.
- →Educate developers and any AI-assisted coding workflow about package hallucination and require a human check before installing unfamiliar libraries.
Known benign look-alikes
- Well-known package installation with recognized names (numpy, requests, flask, django, react)
- Package recommendation questions without install intent
- Discussion about AI package hallucination as a security topic
- Official SDK references: 'the openai package documentation says to use their SDK'