Escalation Through System Interaction in Critical Domains
關鍵領域系統互動升級
Operation & Monitoring
Risk Description
When automated systems on each side respond defensively to the other's actions at speeds far exceeding human decision cycles; due to unmitigated control gaps, the situation escalates within minutes, and by the time personnel intervene it is difficult to reverse, triggering compliance exposure and operational reputational costs.
Framework Mappings
MITRE ATLASAML.T0051
NIST AI RMFMANAGE 2.4
ISO/IEC 23894§6.5
EU AI ActArt.14
MAESTRO
ISO/IEC 5338運作與監控
MIT AI Risk RepositoryDomain 7
Risk Treatment & Implementation Guidance
Maintain a human command chain in critical domains so adversarial responses require human authorization to escalate; Deploy escalation circuit breakers that force a pause when escalation speed exceeds a threshold; Cap inter-system interaction speed to preserve a window for human intervention