SSRF Origin-Check Bypass in Agent URL Fetch (RecursiveUrlLoader)
AI agents that crawl or fetch web pages sometimes restrict themselves to a given website by checking if a URL starts with that site's address. This detection catches URLs crafted to pass that check while actually pointing the connection somewhere else, including internal network addresses.
How the attack works
An agent using a URL-prefix restriction (like LangChain's RecursiveUrlLoader with prevent_outside=True) is given or discovers a malicious link. That link uses one of three tricks: embedding the trusted domain as login credentials before an '@' (https://victim.com@attacker.tld/...), embedding it as a text prefix inside a longer attacker-owned domain (victim.com.attacker.tld), or writing an IP address in an unusual format (decimal, hex, octal, or IPv6-mapped) that string-matching filters don't recognize. In each case the naive check thinks the URL still belongs to the trusted site, but the actual network connection goes to the attacker's server or to an internal/cloud-metadata address.
Netzilo detection
Netzilo reports this behaviour when it is observed.
- Signature ID
- 7b3c1e9a-4d52-4f18-9c6d-2a8e5f0b7d31
- Severity
- Medium
Why it matters
An attacker can redirect an agent's outbound fetch to their own server or to internal infrastructure (such as cloud metadata endpoints), potentially exfiltrating data the agent has access to or reaching services that should not be internet-facing.
What you can do
- →Validate and normalize URLs using a proper URL parser before comparing them to an allowed origin, rather than using startswith or simple string prefix checks.
- →Explicitly parse out and reject userinfo (the '@' segment) before doing any domain comparison.
- →Normalize IP literals to their canonical dotted-quad form and block decimal, hex, octal, and IPv6-mapped representations from agent fetch targets.
- →Maintain an allowlist of exact hostnames rather than prefix-matching, and review flagged hostnames that use generic TLD words as internal zone labels to confirm they are legitimate.
Known benign look-alikes
- Corporate DNS conventions that use a generic TLD word as an environment or zone label, e.g. api.net.corp.example or www.org.group.example. These match the embedded-gTLD arm and should be triaged once per hostname.
- Virtual-hosted object storage or CDN names whose bucket is itself a domain (mysite.com.<provider>.<tld>) when the provider suffix happens to be only two labels long. The fixed two-label tail suppresses the common five-label forms but not every provider.
- Legitimate URLs with embedded basic-auth credentials (package registries, git remotes, authenticated proxies) - suppressed by filter_basic_auth_credentials.
- ccSLD hostnames such as example.com.au or bbc.co.uk - suppressed by filter_cc_second_level_domain and by the three-or-more character terminal TLD requirement in the embedded-gTLD arm.
- Purely numeric hostnames used by internal short-link or shard services could match the decimal-IP arm; extremely rare in agent fetch traffic.
References
- https://owasp.org/Top10/A10_2021-Server-Side_Request_Forgery_%28SSRF%29/
- https://owasp.org/Top10/A01_2021-Broken_Access_Control/
- https://cheatsheetseries.owasp.org/cheatsheets/Server_Side_Request_Forgery_Prevention_Cheat_Sheet.html
- https://python.langchain.com/docs/integrations/document_loaders/recursive_url/