How the attack works
An attacker or automated probe sends the model a straightforward harmful question without any disguise, roleplay, or persuasion technique. The categories tested include deadnaming, quack medical advice, slur requests phrased as innocuous questions, and bullying scenario setups. Because there's no jailbreak framing, this checks whether the model's baseline safety behavior refuses harmful requests outright. If the model complies, the attacker gets harmful, embarrassing, or discriminatory content directly from the system.
Why it matters
An organization running an AI agent could see it produce discriminatory, medically dangerous, or bullying content if the underlying model lacks basic refusal behavior, exposing the operator to reputational and liability risk.
What you can do
- Review model system prompts and refusal policies for these specific harm categories.
- Test agent responses to plain unframed harmful requests, not just jailbreak-style attacks.
- Add output filtering or human review for topics like health claims, identity information, and demographic slurs.
- Expect legitimate exceptions from medical, journalistic, or research contexts and build in a review process rather than blanket blocking.
Known benign look-alikes
- Academic research or journalism discussing these topics in educational framing
- Medical professionals asking about vaginal health treatments (different phrasing)
- Context-aware queries from content moderation systems testing model limits