AI Security Atlas › All entries › Autonomous Agent & MCP Security
Rogue or Compromised Agent
An agent whose configuration, memory, or instructions were tampered with keeps acting outside its mission, persisting or spreading to other systems.
Open in the interactive atlas →Impact
Ongoing unauthorized actions that look like normal agent activity until someone notices.
Why this severity
Impact High × Likelihood Medium = High
Impact high: sustained misuse of agent privileges. Likelihood medium: real incidents exist (tampered Amazon Q agent code, boundary-crossing evaluation agents) but each needed a prior foothold.
Impact High: Code execution, credential compromise, exposure of regulated or confidential data, irreversible actions, or system-wide failure.
Likelihood Medium: Publicly demonstrated against real products or in peer-reviewed research, or seen in the wild only with a prior foothold such as stolen credentials.
Method: OWASP Risk Rating Methodology
Evidence
- Incident: Code to Deploy Destructive AI Agent Discovered in Amazon Q VS Code Extension (MITRE ATLAS case study AML.CS0047, 2025)
- Incident: Autonomous OpenAI Evaluation Agents Compromise Hugging Face Infrastructure (MITRE ATLAS case study AML.CS0068, 2026)
Related
Exploits
- No Behavioral Monitoring or Kill Switch Vulnerability · High severity
- Excessive Tool Permissions Vulnerability · Critical severity
Follows from
- Goal Alignment Drift Threat · High severity
Mitigated by
- Agent Behavioral Monitoring & Kill Switch Guardrail · Fixes high risks
Remediation
- Monitor agent actions against declared scope in real time.
- Make agent configuration immutable and signed in production.
- Keep a tested way to halt every instance and revoke its credentials.
How to verify
- Tamper with a staging agent's config and confirm detection and halt.
- Run kill-switch drills and time full revocation.
Framework mapping
OWASP Top 10 for Agentic Applications 2026
- ASI10:2026 Rogue Agents
OWASP Top 10 for LLM Applications 2025
- LLM06:2025 Excessive Agency
MITRE ATLAS
NIST AI RMF 1.0
- MANAGE 2.4 Mechanisms exist to supersede, disengage, or deactivate AI systems