Content moderation systems
Incidents implicated systems
Incidente 4508 Reportes
Kenyan Data Annotators Reported Psychological Trauma from Labeling Graphic Text for ChatGPT Safety Systems
2021-11-01
Sama data annotators in Nairobi, Kenya, reported lasting psychological trauma after labeling graphic sexual and violent text for OpenAI's content-moderation classifiers used in ChatGPT. Workers alleged low pay and inadequate psychological support; Sama disputed aspects of those accounts. The contract ended early, and former workers later petitioned Kenya's Parliament for stronger protections.
MásIncidente 9556 Reportes
Global Cybercrime Network Storm-2139 Allegedly Exploits AI to Generate Deepfake Content
2024-12-19
A global cybercrime network, Storm-2139, allegedly exploited stolen credentials and developed custom tools to bypass AI safety guardrails. They reportedly generated harmful deepfake content, including nonconsensual intimate images of celebrities, and their software is reported to have disabled content moderation, hijacked AI access, and resold illicit services. Microsoft disrupted the operation and filed a lawsuit in December 2024, later identifying key members of the network in February 2025.
MásIncidente 3112 Reportes
YouTube Auto-Moderation Reportedly Banned Women of Sex Tech Conference
2020-05-02
YouTube’s automated content moderation tool erroneously removed The Women of Sex Tech conference's live-streamed event and banned the conference from the platform, despite not violating the platform’s sexual content policies.
MásIncidente 10552 Reportes
FBI Reports AI Use by Threat Actors in Broader Cyber Context Including Infrastructure Intrusions
2025-04-29
FBI Deputy Assistant Director Cynthia Kaiser stated that adversarial actors, particularly those affiliated with China and organized cybercriminal groups, are increasingly integrating AI tools across the cyberattack lifecycle, with documented use cases reportedly including purported AI-generated spear phishing, business identity fabrication, internal network mapping, and deepfake-enabled fraud. The tools are allegedly already assisting intrusions targeting U.S. infrastructure.
Más