S0087RS6-T21-S0087-Z · Full risk code

Difficulty of Multi-System Safety Validation

多系統安全驗證困難

Verification & Validation
Risk Description

When an organization completes full safety testing for each system individually, all of which pass; due to unmitigated control gaps, after integration and deployment, anomalous behaviors emerge that never appeared in isolated testing, and such behaviors cannot be identified in advance using existing test methods, triggering compliance exposure and operational reputational costs.

Framework Mappings

MITRE ATLASAML.T0051
NIST AI RMFMEASURE 2.6
ISO/IEC 23894§6.4
OWASP Top 10 for LLMLLM06
MAESTRO
ISO/IEC 5338驗證與確效
MIT AI Risk RepositoryDomain 7

Risk Treatment & Implementation Guidance

Stress-test integrated multi-agent systems for interaction behaviors invisible in isolated tests; Deploy progressively, observing inter-system interaction at small scale before scaling; Monitor integration-layer behavior to detect emergent anomalies