Anthropomorphism Producing Misplaced Trust
擬人化導致不當信任
Operation & Monitoring
Risk Description
When after prolonged interaction with a conversational system, a user gradually comes to regard it as an advisor with judgment, relying primarily on its recommendations for major life decisions without seeking help from someone with actual expertise and accountability.; due to lack of pre-deployment verification, the system triggers regulatory exposure and operational costs.
Framework Mappings
EU AI ActArt.50
NIST AI 600-1Human-AI Configuration
NIST AI RMFGOVERN 3.2
ISO/IEC 42005影響評估流程
ISO/IEC 5338運作與監控
MIT AI Risk RepositoryDomain 5
Risk Treatment & Implementation Guidance
Use clear AI-identity disclosure to prevent users mistaking the system for a capable advisor; Explain capability boundaries in the interface and prompt seeking professional help for major life decisions; Detect high-dependence interaction patterns and remind users to seek diverse input