Over-Reliance on Risk Assessment Tools
風險評估工具的過度依賴
Operation & Monitoring
Risk Description
When a decision maker uses the system's score as the primary basis without adequately weighing favorable specific evidence in the individual case; due to unmitigated control gaps, the tool has higher error rates for a particular group, so those cases continue to receive adverse outcomes, triggering compliance exposure and operational reputational costs.
Framework Mappings
EU AI ActArt.14
NIST AI 600-1Human-AI Configuration
NIST AI RMFMEASURE 2.11
ISO/IEC 42005影響評估流程
ISO/IEC 5338運作與監控
MIT AI Risk RepositoryDomain 5
Risk Treatment & Implementation Guidance
Mandate that risk-assessment systems are reference only, requiring case-specific evidence in decisions; Calibrate and debias where error is higher for particular groups and disclose applicability limits; Periodically audit bias and monitor per-group outcome disparities