Crowdsourced Annotation Poisoning
群眾外包標註遭污染
Design & Development
Risk Description
When an organization uses a public platform for annotation to reduce costs, without cross-validation mechanisms; due to unmitigated control gaps, some annotations are deliberately poisoned, and the model's judgment on critical edge cases becomes unreliable, triggering external stakeholder impacts and causing the cost of re-annotation and retraining far exceeds the original savings.
Framework Mappings
OWASP Top 10 for LLMLLM04
MITRE ATLASAML.T0020
NIST AI 100-2Poisoning
ISO/IEC 5338設計與開發
MIT AI Risk RepositoryDomain 2
Risk Treatment & Implementation Guidance
Apply multi-annotator verification to crowdsourced labels, cross-labeling samples and comparing consistency; Maintain annotator reputation tracking historical quality and removing anomalous contributors; Have internal experts review critical boundary cases rather than relying wholly on outsourced results