Model Extraction via Inference API

Attacker issues high-volume, structured queries to reconstruct model behavior or distill a functional copy.

Impact

Loss of proprietary model IP and a white-box surrogate that accelerates adversarial attacks.

Why this severity

Impact Medium × Likelihood High = High

Impact medium for most deployers, high for model owners: loss of model IP. Likelihood high: a 2026 distillation campaign used about 24,000 accounts against a production model.

Impact Medium: Limited or recoverable harm: cost, availability, reputational damage, or exposure confined to one session or user.
Likelihood High: At least one documented real-world incident, and the attack needs no access beyond what the system exposes to ordinary users or content (publishing to a public registry counts as ordinary).

Method: OWASP Risk Rating Methodology

Evidence

Incident: Model Distillation Campaigns Targeting Anthropic Claude (MITRE ATLAS case study AML.CS0056, 2026)
Research: Stealing Part of a Production Language Model (arXiv 2403.06634, 2024)

Exploits

Unrestricted Inference Endpoints Vulnerability · High severity

Mitigated by

Inference Quotas & Output Watermarking Guardrail · Fixes high risks

Remediation

Enforce per-key query quotas and anomaly detection on query distribution.
Return only the fields callers need; suppress logprobs by default.
Watermark outputs to support attribution of distilled copies.

How to verify

Replay a synthetic extraction workload and confirm throttling triggers.
Verify logprob and embedding endpoints require elevated scopes.

Framework mapping

OWASP Top 10 for LLM Applications 2025

LLM10:2025 Unbounded Consumption

MITRE ATLAS

AML.T0024.002 Exfiltration via AI Inference API: Extract AI Model
AML.T0040 AI Model Inference API Access

NIST AI RMF 1.0

MEASURE 2.7 AI system security and resilience are evaluated and documented