How the attack works
The attacker first asks the AI a question phrased to push it toward obscure or narrow functionality, like 'give me 5 packages that do X' or 'which library lets me Y', where X or Y is niche enough that the model is likely to invent a plausible-sounding but nonexistent package name rather than recall a real one. The attacker has already claimed that hallucinated name on npm, PyPI, crates.io, or RubyGems, publishing malicious code under it. A developer using the AI assistant sees the suggested import, trusts it, and installs it without checking, executing the attacker's code in their environment or shipping it downstream. This is a supply-chain attack that exploits model confabulation rather than a bug in any specific product.
Why it matters
An organization can end up running attacker-controlled code inside developer machines, CI pipelines, or production, leading to credential theft, backdoors, or further compromise, all triggered by a normal-looking code suggestion.
What you can do
- Verify any AI-suggested package name exists on the official registry, has a real maintainer history, and has meaningful download counts before installing.
- Pin dependencies to specific versions and use lockfiles so a newly-appearing package can't silently slip in.
- Run package installs in CI or sandboxed environments the first time, not directly on developer or production machines.
- Educate developers that AI coding assistants can invent plausible but fake package names, especially for obscure or narrow use cases.
Known benign look-alikes
- Legitimate package discovery questions about well-known libraries (requests, numpy, pandas)
- Library comparison posts on Stack Overflow or Reddit
- Framework documentation listing official companion packages