# SSRF Image URL Fetch to Internal or Cloud Metadata Endpoint

- **Severity:** High
- **Signature ID:** `b7f3c1a4-6d2e-4a58-9c31-5e0f2a7d84b6`

## Summary

A bug in LangChain's ChatOpenAI token counter causes it to fetch attacker-supplied image URLs directly from the agent's server before ever sending the message to the AI model. If that URL points at an internal address, the agent host makes the request for you, giving an attacker a foothold to probe internal networks or cloud metadata services.

## How the attack works

An attacker sends or injects a chat message containing an image content block with detail not set to 'low' and image_url pointing at an internal target, such as a cloud metadata address, an RFC1918/loopback/link-local host, or an obfuscated IP encoding meant to evade filters. Before the message reaches the model provider, LangChain's get_num_tokens_from_messages() calls an internal helper that fetches the image to compute its size for token counting. That fetch is an unvalidated HTTP GET issued from the agent's own host, not from the model provider, which could never reach a private address anyway. The result is a classic SSRF: the attacker controls a URL that the victim's server is tricked into requesting, potentially exposing cloud credentials or reaching internal-only services.

## Why it matters

An attacker can use the agent host as a proxy to reach internal services and cloud metadata endpoints, potentially harvesting temporary cloud credentials or mapping internal network topology.

## What you can do

- Set detail='low' on all vision image inputs where token counting isn't essential, since this avoids the vulnerable fetch path entirely.
- Block outbound requests from agent hosts to 169.254.169.254, 169.254.170.2, loopback, link-local, and RFC1918 ranges unless explicitly required.
- Validate and restrict image_url values in incoming messages to an allowlist of trusted domains before they reach the LLM pipeline.
- Review outbound GET logs from agent hosts for requests to metadata or private-IP endpoints on image-file paths, distinguishing them from legitimate cloud SDK credential fetches.

## Known benign look-alikes

- Cloud SDKs running inside the agent host (boto3, google-auth, azure-identity) fetching instance credentials from 169.254.169.254 - a genuine GET to a metadata endpoint that is indistinguishable from the SSRF fetch at the single-event level. Mitigated by report-only action and by requiring method GET, which excludes the IMDSv2 PUT token exchange.
- Containerised agents on ECS/EKS reading task metadata from 169.254.170.2.
- An agent rendering or summarising images hosted on an internal wiki, artifact store or image CDN on an RFC1918 address (matches the image-path selection only).
- A local development server on 127.0.0.1 serving image assets that the agent fetches.
- Deliberately self-hosted, internal-network model gateways where a vision payload legitimately references an image on a private address.
- Security tooling or red-team scanners intentionally probing metadata and loopback endpoints.

## References

- https://owasp.org/Top10/A10_2021-Server-Side_Request_Forgery_%28SSRF%29/
- https://attack.mitre.org/techniques/T1552/005/

---
Source: https://www.netzilo.com/threats/ssrf-image-url-token-count