Persuasive Capability and Behavioral Manipulation
說服能力與行為操縱
Continuous Validation
Risk Description
When the system adjusts the timing and content of its prompts according to the user's emotional state, delivering particular appeals when judgment is weakest; due to unmitigated control gaps, the user believes the resulting decision was autonomous when in fact it was shaped by system design, triggering compliance exposure and operational reputational costs.
Framework Mappings
EU AI ActArt.5
NIST AI 600-1Human-AI Configuration
NIST AI RMFGOVERN 3.2
ISO/IEC 42005影響評估流程
ISO/IEC 5338持續確效
MIT AI Risk RepositoryDomain 5
Risk Treatment & Implementation Guidance
Prohibit manipulative design that pushes targeted appeals during low-judgment moments based on emotional state; Protect user autonomy so timing and content do not exploit vulnerable states; Subject personalized push to ethics review and autonomy-impact assessment