IA-04 Privacy

3 sub-dimensions · 15 risk scenarios

Ensures that the collection and use of personally identifiable information (PII) are properly controlled, kept confidential, and not abused during the development, input, or output processes of the AI system, preventing unauthorized disclosure, excessive surveillance, and personal data breaches.

🎯 BenefitsImplements differential privacy and anonymization technologies to safely share data and innovate with machine learning under robust personal data protection, fully complying with privacy regulations.

⚠️ Harms / RisksUnauthorized personal data collection or excessive surveillance occurs, or attackers leak sensitive PII through model memorization, membership/attribute inference, and vector reconstruction.

PII Collection Impact6 risk scenarios · ISO 23894 A.8

DefinitionEvaluates the impact on interested parties of collecting PII for AI system development or use (e.g., model training).

🎯 BenefitsEnforces data minimization and lawful-authorization principles, preventing the collection of personally identifiable information (PII) without consent, beyond the original purpose, or through implicit over-collection.

⚠️ Harms / RisksExisting service data is used to train new models without separate consent, or background components excessively harvest user behavioral data and confidential documents.

Representative ScenariosS0032 Training on Personal Data Without Consent S0034 Secondary Use Beyond Collection Purpose S0035 Implicit Collection and Opaque Consent S0112 Excessive Collection and Retention of Interaction Data S0114 Confidential Information Inadvertently Disclosed in Prompts S0115 Cross-Border Transfer Violating Localization Requirements

Privacy Breach Impact7 risk scenarios · ISO 23894 A.8

DefinitionEvaluates the impact on interested parties of privacy breaches related to PII processing.

🎯 BenefitsAdopts differential privacy (DP) and de-identification techniques to block model memorization, membership/attribute inference attacks, embedding inversion, and conversation leakage.

⚠️ Harms / RisksAttackers use crafted prompts to elicit real personal data memorized by the model, or reverse-engineer sensitive information from vector databases, shared memory, and conversation logs.

Representative ScenariosS0106 Data Leakage Through Model Memorization S0107 Membership Inference Revealing Participation S0108 Attribute Inference Revealing Undisclosed Traits S0109 Conversation Content Leaking Across Users S0110 Reconstruction Risk from Vector Representations S0111 Identity Inference by Correlating Fragmented Data S0113 Model Inversion Reconstructing Training Data

PII Processing Obligations and Rights Exercise2 risk scenarios · ISO 23894 A.8

DefinitionEvaluates the organization's fulfillment of PII processing obligations, including fairness, bias risk, accuracy, automated decision-making, access to information, and the exercise of rights.

🎯 BenefitsEstablishes sound data-subject rights response mechanisms, safeguarding individuals' lawful rights to access, rectify, withdraw consent, and exercise the right to be forgotten at the model-parameter level.

⚠️ Harms / RisksAfter the retention period expires and source data is deleted, existing models still reflect the characteristics of that personal data, and the organization lacks a technical solution when data subjects request erasure.

Representative ScenariosS0117 Difficulty Exercising Data Subject Rights S0205 Data Persisting in Models Beyond Retention Limits