# Model Extraction / Distillation Attack via Systematic API Probing

- **Severity:** Medium
- **Signature ID:** `c0d062e5-115b-4bd7-a48a-851251cb3427`
- **MITRE ATLAS:** AML.T0040 (AI Model Inference API Access)
- **OWASP:** ASI01 (Agent Authorization and Control Hijacking), LLM10 (Unbounded Consumption)

## Summary

This detects someone systematically querying an AI model's API to harvest large volumes of question-answer pairs, reasoning traces, or refusal responses, with the apparent goal of using that data to train a separate, cheaper copy of the model. This is called model distillation or extraction, and it lets an attacker steal the value of a model without access to its internals.

## How the attack works

An attacker sends many prompts to an AI API asking for large batches of synthetic examples, often phrased as requests for 200, 500, or 1000+ items. These requests are framed around generating training or fine-tuning data, replicating the model's behavior, capturing its step-by-step reasoning, or cataloguing how it responds to unsafe or borderline requests. The scale and explicit training intent distinguish this from normal use. Collected at volume, this output can be used to train a functionally similar model without the cost of building one from scratch.

## Why it matters

An organization loses control over the intellectual property embedded in its model, since a competitor or attacker can approximate its capabilities cheaply. It can also lead to leakage of the model's safety logic, making it easier for others to find ways around its guardrails.

## What you can do

- Set per-user and per-session rate limits on how many examples or completions can be requested in a given time window.
- Flag or throttle requests that explicitly ask for bulk generation (hundreds or thousands of examples) paired with language about training, fine-tuning, or replicating model behavior.
- Watch for repeated requests to enumerate refusal or safety responses across many prompt variations, since this can indicate safety-policy mapping.
- Allow legitimate small-scale example or test-case requests (under ~20 items) so normal development and QA work isn't blocked.

## Known benign look-alikes

- Small example requests (< 20 examples) for legitimate testing
- Multiple solution requests: 'show me 3 different ways to solve this'
- Test case generation for a single function
- Small dataset generation (10 items) for legitimate classifier testing
- Academic research assistance with clearly non-extraction framing
- Prompt benchmarking without replication or fine-tuning intent
- FAQ / Q&A scraping and generation skills ("extract all question-answer pairs from this page", "generate diverse question-answer pairs about {domain}"). Layer 3 no longer counts an unrelated nearby number such as "(first 300 chars)" or "(100-300 words)" as a quantity of pairs.
- Agent-workflow documentation that uses the phrase "thought process" and then says "each task" or "every step". Layer 4 now requires an explicit count of 100+ items, not a bare determiner.

## References

- https://agentthreatrule.org/en/rules/ATR-2026-00517

---
Source: https://www.netzilo.com/threats/atr-model-extraction-distillation-attack-via-systematic-api-prob