S0167RS6-T19-S0167-Z · Full risk code

Automation Bias Causing Anomalies to Be Overlooked

自動化偏誤導致忽略異常

Operation & Monitoring
Risk Description

When operators come to rely on the system's high accuracy and gradually stop reviewing independently; due to unmitigated control gaps, when the system misjudges a rare situation, clear anomaly signals are overlooked, and the problem is discovered only after consequences materialize, triggering compliance exposure and operational reputational costs.

Framework Mappings

NIST AI 600-1Human-AI Configuration
NIST AI RMFMEASURE 2.5
ISO/IEC 42005影響評估流程
ISO/IEC 5338運作與監控
MIT AI Risk RepositoryDomain 5

Risk Treatment & Implementation Guidance

Maintain operators' independent review ability through periodic training; Inject known anomalies deliberately to test whether operators still detect and report; Design anomaly-salience mechanisms prompting extra scrutiny in rare situations