Difficulty of Multi-System Safety Validation
多系統安全驗證困難
Verification & Validation
Risk Description
When an organization completes full safety testing for each system individually, all of which pass; due to unmitigated control gaps, after integration and deployment, anomalous behaviors emerge that never appeared in isolated testing, and such behaviors cannot be identified in advance using existing test methods, triggering compliance exposure and operational reputational costs.
Framework Mappings
MITRE ATLASAML.T0051
NIST AI RMFMEASURE 2.6
ISO/IEC 23894§6.4
OWASP Top 10 for LLMLLM06
MAESTRO
ISO/IEC 5338驗證與確效
MIT AI Risk RepositoryDomain 7
Risk Treatment & Implementation Guidance
Stress-test integrated multi-agent systems for interaction behaviors invisible in isolated tests; Deploy progressively, observing inter-system interaction at small scale before scaling; Monitor integration-layer behavior to detect emergent anomalies