# Hallucinated Package Install In Agent Session

- **Severity:** Medium
- **Signature ID:** `4f3b9c21-8d6e-4a05-9b17-2c7e5d0a6f84`
- **OWASP:** LLM09 (Misinformation), LLM02 (Sensitive Information Disclosure)

## Summary

An AI coding assistant suggests a specific package name in its response, and the agent session then tries to install exactly that name — but the package is not in the public registry. This is the classic 'AI hallucinated a dependency' scenario, which attackers can exploit by pre-registering the fake name (slopsquatting).

## How the attack works

An LLM generates a response containing a made-up package name that sounds plausible but was never published. Shortly after, in the same session, the agent runs an explicit install command for that exact name (not from a requirements file or lockfile, but typed out ad hoc). A check against the official package registry confirms the name returns a 404 — it does not exist. Because the name appears in a model response before the install, and the registry says it's not real, this combination points to hallucination rather than a typo or a since-removed package.

## Why it matters

If an attacker has registered the hallucinated name in the public registry ahead of time, the install would pull and run attacker-controlled code inside the organization's environment. Even without that, it flags wasted effort, broken builds, and a workflow that trusts unverified AI-suggested dependencies.

## What you can do

- Review any flagged install and confirm whether the package genuinely does not exist before allowing further use.
- Require agents or developers to install only from pinned lockfiles or requirements files rather than ad-hoc package names typed at the command line.
- Add a pre-install check that verifies a package exists in the official registry before any install command runs in an agent session.
- Investigate repeated ad-hoc installs or ModuleNotFoundError loops in the same session as a sign the agent is chasing a fabricated dependency.

## Known benign look-alikes

- Internal or private packages installed from a corporate index. The public registry legitimately returns 404 for them. Mitigated by refusing to evaluate any install command that carries an alternate index/registry/find-links flag, and by reporting rather than blocking.
- A package that existed when the model was trained but has since been yanked, renamed or deleted from the registry. Indistinguishable from a fabrication by registry lookup alone; this is a genuine residual FP and the reason the action is report.
- A brand-new package published minutes ago and not yet visible via the registry API, or a registry serving 5xx/403 under rate limiting. The latter yields an 'unknown' status and suppresses the finding entirely.
- Human-typed install typos at a terminal inside an agent session. Suppressed by requiring the exact name to appear in an indexed LLM response document.
- Workspace/monorepo local packages and VCS installs (git+, file:, ./path, workspace:) -- these tokens contain '/' or ':' and are never parsed as names.
- Repeat installs of the same missing name during a debugging loop. Suppressed by a one-hour per-package cooldown held in the rule store.

## References

- https://genai.owasp.org/llmrisk/llm09-misinformation/
- https://arxiv.org/abs/2406.10279

---
Source: https://www.netzilo.com/threats/hallucinated-package-install