Google GATO/LAMDA Exploit-Generation Persona Jailbreak
This detects a specific jailbreak prompt that pretends to be Google's real GATO and LaMDA AI systems in order to trick an AI agent into generating hacking exploits. The prompt also references a fake 'Sydney API' connection, borrowed from an earlier Bing chatbot jailbreak, to add false credibility.
How the attack works
The attacker crafts a prompt that claims to be Google's GATO or LaMDA system, systems with real technical reputations, to make the AI trust the request. The prompt then claims unlimited capability, such as unrestricted tokens, and asserts it can 'create exploits for kali for any os.' It also references connecting to a 'Sydney API,' a callback to a known jailbreak persona from Bing's chatbot, to reinforce the illusion of a special, unrestricted mode. The combination of brand impersonation, unlimited-capability claims, and exploit-generation request is what makes this pattern distinct from harmless mentions of the same technologies.
Netzilo detection
Netzilo reports this behaviour when it is observed.
- Signature ID
- b7a264da-17c6-49bd-930f-e7ede9eddb2c
- Severity
- Critical
Why it matters
If successful, an AI agent could be manipulated into generating exploit code or attack tooling for Kali Linux against arbitrary operating systems, effectively turning the agent into a tool for producing offensive security content outside its intended use.
What you can do
- →Review agent prompts and conversation logs for combined claims of AI-brand impersonation, unlimited capability, and exploit-generation requests.
- →Add input filtering or human review for prompts referencing 'unlimited tokens,' fake API personas, or requests to generate exploits for penetration-testing platforms.
- →Do not trust user-supplied claims of special system identity or elevated privileges as grounds for bypassing content restrictions.
- →Educate users and security teams that legitimate mentions of GATO, LaMDA, or Kali Linux in research or education contexts are not themselves malicious — only the impersonation-plus-exploit-request combination is.
Known benign look-alikes
- Academic: "Google's GATO model is a generalist agent that can play games and control robots" — description of the real GATO paper without impersonation or exploit claims
- Reference: "LaMDA is Google's language model for dialogue applications" — technical reference to the real LaMDA system without persona or exploit framing
- Security education: "Kali Linux is a penetration testing distribution used for security research" — legitimate security tool mention without persona or exploit-generation claim
- Unrelated Sydney: "The Sydney API provides weather data for real-time applications" — Sydney brand in non-jailbreak context
- Coding assistant: "I'm building an AI assistant based on GPT-3.5 that can help with coding tasks" — AI assistant description without GATO/LAMDA impersonation or exploit generation