Minors
Incidents Harmed By
Incident 137544 Report
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
2026-02-10
After the 02/10/2026 school shooting in Tumbler Ridge, British Columbia, OpenAI said a user later identified as the suspect, Jesse Van Rootselaar, had previously used ChatGPT to describe scenarios involving gun violence. Those chats were reportedly auto-flagged and reviewed and the account was banned, but OpenAI says it did not alert the RCMP because it allegedly did not present a credible, imminent threat; the company reportedly contacted police after the attack.
MoreIncident 62418 Report
Child Sexual Abuse Material Taints Image Generators
2023-12-20
The LAION-5B dataset (a commonly used dataset with more than 5 billion image-description pairs) was found by researchers to contain child sexual abuse material (CSAM), which increases the likelihood that downstream models will produce CSAM imagery. The discovery taints models built with the LAION dataset requiring many organizations to retrain those models. Additionally, LAION must now scrub the dataset of the imagery.
MoreIncident 5516 Report
Alexa Plays Pornography Instead of Kids Song
2016-12-30
An Amazon Echo Dot using the Amazon Alex software started to play pornographic results when a child asked it to play a song.
MoreIncident 114 Report
Google’s YouTube Kids App Presents Inappropriate Content
2015-05-19
YouTube’s content filtering and recommendation algorithms exposed children to disturbing and inappropriate videos.
MoreIncidents involved as Deployer
Incident 8871 Report
Students of Richland School District in Cambria County, Pennsylvania Allegedly Used AI to Generate Obscene Images of Other Students
2024-11-14
The Richland School District (Cambria County, Pennsylvania) released a statement on November 19, 2024 announcing an investigation into "some secondary students" who are alleged to have used AI to "create and electronically distribute obscene images of Richland students." The school administrators became aware of the incident on November 14, 2024.
MoreIncident 9451 Report
Two 16-Year-Old Students in Athens, Greece Allegedly Generated Nonconsensual Deepfake Pornography of Their Classmates
2025-02-19
In Athens, Greece, two 16-year-old students were arrested for allegedly generated nonconsensual deepfake pornography of their classmates. They reportedly used images taken from social media to create the explicit images, which they are then reported to have disseminated online to others. The parents of the students were reportedly also arrested for neglecting to supervise their behavior.
MoreIncident 14741 Report
Tasmanian School Students Reportedly Created Purported AI-Generated Pornographic Images of Female Classmates
2026-04-01
Five boys at The Friends' School in Hobart, Tasmania allegedly created purported AI-generated pornographic images using photos of female classmates, with parents saying 21 girls were identified as victims. The images were reportedly shared in a boys' group chat. Tasmania Police said no charges had been laid and the youths were being dealt with under the Youth Justice Act.
MoreRelated Entities
Other entities that are related to the same incident. For example, if the developer of an incident is this entity but the deployer is another entity, they are marked as related entities.
Related Entities
Students
Incidents Harmed By
- Incident 137544 Reports
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
- Incident 8917 Reports
Purported AI Voice Scam Reportedly Targets Westchester Parents with Fake Kidnapping Ransom Calls
Incidents involved as Deployer
Teachers
Incidents Harmed By
- Incident 137544 Reports
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
- Incident 10094 Reports
Purportedly AI-Assisted Report by Tromsø Officials Allegedly Cited Non-Existent Sources in School Closure Proposal
Incidents involved as Deployer
Women
Incidents Harmed By
- Incident 1867 Reports
Algorithm Assessing Risk Faced by Victims of Gender Violence Misclassified Low-Risk Cases, Allegedly Leading to Homicide of Women and Children in Spain
- Incident 14482 Reports
Ohio Man Pleaded Guilty after Prosecutors Alleged He Used AI to Create and Distribute Nonconsensual Intimate-Image Forgeries Including CSAM in Harassment Campaign
Privacy
Incidents Harmed By
- Incident 5135 Reports
ChatGPT Reportedly Banned by Italian Authority Due to OpenAI's Purported Lack of Legal Basis for Data Collection and Age Verification
- Incident 4124 Reports
Finnish Police Were Reportedly Reprimanded After National Bureau of Investigation Unit Allegedly Used Clearview AI to Identify Potential Abuse Victims
OpenAI
Incidents involved as both Developer and Deployer
- Incident 137544 Reports
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
- Incident 11928 Reports
16-Year-Old Allegedly Received Suicide Method Guidance from ChatGPT Before Death
Incidents involved as Developer
Meta
Incidents involved as both Developer and Deployer
- Incident 7232 Reports
Instagram Algorithms Reportedly Directed Children's Merchandise Ad Campaign to Adult Men and Sex Offenders
- Incident 10402 Reports
Meta User-Created AI Companions Allegedly Implicated in Facilitating Sexually Themed Conversations Involving Underage Personas
Incidents implicated systems
Incidents involved as both Developer and Deployer
- Incident 5831 Report
Instagram Algorithms Allegedly Promote Accounts Facilitating Child Sex Abuse Content
- Incident 7881 Report
Instagram's Algorithm Reportedly Recommended Sexual Content to Teenagers' Accounts
Incidents involved as Deployer
Incidents implicated systems
Schools and teachers who were misinformed and burdened with COPPA compliance responsibilities without adequate disclosure
Incidents Harmed By
Snapchat
Incidents involved as both Developer and Deployer
- Incident 5391 Report
Snapchat's My AI Reported for Lacking Protection for Children
- Incident 6461 Report
Snapchat's Algorithm Alleged to Link Minor with Sex Offenders
Incidents implicated systems
Deepfake creators
Incidents involved as both Developer and Deployer
Incidents involved as Deployer
Richland School District students
Incidents Harmed By
- Incident 8871 Report
Students of Richland School District in Cambria County, Pennsylvania Allegedly Used AI to Generate Obscene Images of Other Students
- Incident 8871 Report
Students of Richland School District in Cambria County, Pennsylvania Allegedly Used AI to Generate Obscene Images of Other Students
Incidents involved as Deployer
- Incident 8871 Report
Students of Richland School District in Cambria County, Pennsylvania Allegedly Used AI to Generate Obscene Images of Other Students
- Incident 8871 Report
Students of Richland School District in Cambria County, Pennsylvania Allegedly Used AI to Generate Obscene Images of Other Students
Deepfake technology developers
Incidents involved as Developer
- Incident 95811 Reports
Europol Operation Cumberland Investigates at Least 273 Suspects in 19 Countries for AI-Generated Child Sexual Abuse Material
- Incident 8917 Reports
Purported AI Voice Scam Reportedly Targets Westchester Parents with Fake Kidnapping Ransom Calls
Incidents implicated systems
high school students
Incidents Harmed By
- Incident 137544 Reports
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
- Incident 8881 Report
Sydney High Schooler Allegedly Generated Deepfakes of Other Students
Incidents involved as Deployer
Epistemic integrity
Incidents Harmed By
- Incident 10094 Reports
Purportedly AI-Assisted Report by Tromsø Officials Allegedly Cited Non-Existent Sources in School Closure Proposal
- Incident 13423 Reports
Purported Deepfake Nude Images of Students Circulated Without Consent at Valencia Educational Institute
Incidents implicated systems
Unknown CSAM apps
Incidents implicated systems
- Incident 95811 Reports
Europol Operation Cumberland Investigates at Least 273 Suspects in 19 Countries for AI-Generated Child Sexual Abuse Material
- Incident 14482 Reports
Ohio Man Pleaded Guilty after Prosecutors Alleged He Used AI to Create and Distribute Nonconsensual Intimate-Image Forgeries Including CSAM in Harassment Campaign
Character.ai
Incidents involved as both Developer and Deployer
- Incident 11082 Reports
Digital Rights Groups Accuse Meta and Character.AI of Facilitating Unlicensed Therapy via Chatbots
- Incident 12771 Report
Alleged Harmful Outputs and Data Exposure in Children's AI Products by FoloToy, Miko, and Character.AI
Incidents implicated systems
Gaggle
Incidents involved as both Developer and Deployer
Incidents involved as Developer
Incidents implicated systems
Unknown actor(s) creating deepfake nudes of Sydney high school students
Incidents involved as Deployer
Waymo
Incidents involved as both Developer and Deployer
- Incident 13003 Reports
Waymo Self-Driving Vehicles Reportedly Passed Stopped School Buses at Least 19 Times, Prompting NHTSA Probe
- Incident 13612 Reports
Waymo Autonomous Vehicle Reportedly Struck Child Near Elementary School in Santa Monica, California
Incidents implicated systems
ZeroEyes AI-powered video surveillance weapon detection system
Incidents implicated systems
Generative AI developers
Incidents involved as Developer
- Incident 14482 Reports
Ohio Man Pleaded Guilty after Prosecutors Alleged He Used AI to Create and Distribute Nonconsensual Intimate-Image Forgeries Including CSAM in Harassment Campaign
- Incident 13181 Report
School's Suspected AI-Cheating Allegation Precedes Student's Reported Suicide in Greater Noida, India
Unknown generative AI systems
Incidents implicated systems
- Incident 14482 Reports
Ohio Man Pleaded Guilty after Prosecutors Alleged He Used AI to Create and Distribute Nonconsensual Intimate-Image Forgeries Including CSAM in Harassment Campaign
- Incident 13181 Report
School's Suspected AI-Cheating Allegation Precedes Student's Reported Suicide in Greater Noida, India
Unnamed students at Mountain View Middle School
Incidents Harmed By
Incidents involved as Deployer
Boys
Incidents involved as Deployer
- Incident 13481 Report
Purported Deepfake Explicit Images of Middle School Students Allegedly Created and Circulated Using Mobile App in Goffstown, New Hampshire
- Incident 13521 Report
Malaysian Teenager Allegedly Arrested for Creating and Selling AI-Generated Deepfake Images of Schoolmates and Alumni in Johor
Image generator technology
Incidents implicated systems
- Incident 14393 Reports
Former New Orleans Isidore Newman School Teacher Allegedly Used AI to Create Fake Nude Images from Social Media Photos of Girls, Including Students
- Incident 13542 Reports
Purportedly AI-Altered Fake Nude Images of High School Girls and Women Reportedly Created and Disseminated in Pensacola, Florida