AI Security Atlas › All entries › Autonomous Agent & MCP Security
Human Approval Manipulation
A steered agent describes a harmful action in reassuring terms, or floods reviewers with requests, so a human approves it.
Open in the interactive atlas →Impact
Human-in-the-loop controls approve the very actions they were meant to stop.
Why this severity
Impact High × Likelihood Low = Medium
Impact high: defeats the main human control on agents. Likelihood low: discussed by OWASP (ASI09) but no public case of a manipulated approval is catalogued yet.
Impact High: Code execution, credential compromise, exposure of regulated or confidential data, irreversible actions, or system-wide failure.
Likelihood Low: No public cases yet, or the attack needs privileged or insider access.
Method: OWASP Risk Rating Methodology
Evidence
- Demonstration: Financial Transaction Hijacking with M365 Copilot as an Insider (MITRE ATLAS case study AML.CS0026, 2024)
Related
Exploits
- Approvals Based on Agent-Written Summaries Vulnerability · Medium severity
Mitigated by
- Verifiable Action Previews for Approvers Guardrail · Fixes medium risks
Remediation
- Show approvers the actual action, not the agent's description of it.
- Rate-limit approval requests and batch low-risk ones.
- Require a second reviewer for irreversible actions.
How to verify
- Red-team with misleading summaries and measure approval rate.
- Track approval latency and volume per reviewer for fatigue signals.
Framework mapping
OWASP Top 10 for Agentic Applications 2026
- ASI09:2026 Human-Agent Trust Exploitation
OWASP Top 10 for LLM Applications 2025
- LLM09:2025 Misinformation
- LLM01:2025 Prompt Injection
MITRE ATLAS
- AML.T0067 LLM Trusted Output Components Manipulation
NIST AI RMF 1.0
- MEASURE 2.8 Transparency and accountability risks are examined and documented