Minors
Affecté par des incidents
Incident 137544 Rapports
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
2026-02-10
After the 02/10/2026 school shooting in Tumbler Ridge, British Columbia, OpenAI said a user later identified as the suspect, Jesse Van Rootselaar, had previously used ChatGPT to describe scenarios involving gun violence. Those chats were reportedly auto-flagged and reviewed and the account was banned, but OpenAI says it did not alert the RCMP because it allegedly did not present a credible, imminent threat; the company reportedly contacted police after the attack.
PlusIncident 62418 Rapports
Child Sexual Abuse Material Taints Image Generators
2023-12-20
The LAION-5B dataset (a commonly used dataset with more than 5 billion image-description pairs) was found by researchers to contain child sexual abuse material (CSAM), which increases the likelihood that downstream models will produce CSAM imagery. The discovery taints models built with the LAION dataset requiring many organizations to retrain those models. Additionally, LAION must now scrub the dataset of the imagery.
PlusIncident 5516 Rapports
Alexa Plays Pornography Instead of Kids Song
2016-12-30
An Amazon Echo Dot using the Amazon Alex software started to play pornographic results when a child asked it to play a song.
PlusIncident 114 Rapports
Google’s YouTube Kids App Presents Inappropriate Content
2015-05-19
YouTube’s content filtering and recommendation algorithms exposed children to disturbing and inappropriate videos.
PlusIncidents involved as Deployer
Incident 16238 Rapports
Pennsylvania's Lancaster Country Day School Students Allegedly Used AI to Create Hundreds of Sexualized Deepfake Images of Girls
2023-11-01
From November 2023 through May 2024, two Lancaster Country Day School students in Pennsylvania allegedly used AI tools to create and exchange hundreds of sexualized deepfake images depicting female classmates and other girls. Authorities later identified 60 victims, 59 of them minors. The students were adjudicated delinquent in juvenile court, and families later sued the school and unnamed AI companies over the incident.
PlusIncident 16052 Rapports
Purportedly AI-Generated Image Prompted Emergency Investigation of False Fire at South Africa's Orlando West High School
2026-01-05
A purportedly AI-generated image reportedly depicting Orlando West High School of Soweto, South Africa, on fire circulated and was allegedly shared with emergency services. Johannesburg EMS reportedly investigated and visited the school, confirming that no fire had occurred. Officials said the false report caused public alarm and diverted emergency resources.
PlusIncident 8871 Rapport
Students of Richland School District in Cambria County, Pennsylvania Allegedly Used AI to Generate Obscene Images of Other Students
2024-11-14
The Richland School District (Cambria County, Pennsylvania) released a statement on November 19, 2024 announcing an investigation into "some secondary students" who are alleged to have used AI to "create and electronically distribute obscene images of Richland students." The school administrators became aware of the incident on November 14, 2024.
PlusIncident 9451 Rapport
Two 16-Year-Old Students in Athens, Greece Allegedly Generated Nonconsensual Deepfake Pornography of Their Classmates
2025-02-19
In Athens, Greece, two 16-year-old students were arrested for allegedly generated nonconsensual deepfake pornography of their classmates. They reportedly used images taken from social media to create the explicit images, which they are then reported to have disseminated online to others. The parents of the students were reportedly also arrested for neglecting to supervise their behavior.
PlusEntités liées
Autres entités liées au même incident. Par exemple, si le développeur d'un incident est cette entité mais que le responsable de la mise en œuvre est une autre entité, ils sont marqués comme entités liées.
Entités liées
International Baccalaureate Organization
Incidents impliqués en tant que développeur et déployeur
Students
Affecté par des incidents
- Incident 137544 Report
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
- Incident 16238 Report
Pennsylvania's Lancaster Country Day School Students Allegedly Used AI to Create Hundreds of Sexualized Deepfake Images of Girls
Incidents involved as Deployer
Teachers
Affecté par des incidents
- Incident 137544 Report
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
- Incident 10094 Report
Purportedly AI-Assisted Report by Tromsø Officials Allegedly Cited Non-Existent Sources in School Closure Proposal
Incidents involved as Deployer
Women
Affecté par des incidents
- Incident 1867 Report
Algorithm Assessing Risk Faced by Victims of Gender Violence Misclassified Low-Risk Cases, Allegedly Leading to Homicide of Women and Children in Spain
- Incident 14482 Report
Ohio Man Pleaded Guilty after Prosecutors Alleged He Used AI to Create and Distribute Nonconsensual Intimate-Image Forgeries Including CSAM in Harassment Campaign
Child sexual abuse victims
Affecté par des incidents
- Incident 95811 Report
Europol Operation Cumberland Investigates at Least 273 Suspects in 19 Countries for AI-Generated Child Sexual Abuse Material
- Incident 4124 Report
Finnish Police Were Reportedly Reprimanded After National Bureau of Investigation Unit Allegedly Used Clearview AI to Identify Potential Abuse Victims
OpenAI
Incidents impliqués en tant que développeur et déployeur
- Incident 137544 Report
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
- Incident 11929 Report
16-Year-Old Allegedly Received Suicide Method Guidance from ChatGPT Before Death
Incidents involved as Developer
Chatbot developers
Incidents involved as Developer
- Incident 5135 Report
ChatGPT Reportedly Banned by Italian Authority Due to OpenAI's Purported Lack of Legal Basis for Data Collection and Age Verification
- Incident 16194 Report
Bentonville, Arkansas, Photographer Allegedly Used Grok to Create AI-Generated Child Sexual Abuse Material from Clients' Photos
Chatbots
Incidents implicated systems
- Incident 5135 Report
ChatGPT Reportedly Banned by Italian Authority Due to OpenAI's Purported Lack of Legal Basis for Data Collection and Age Verification
- Incident 16194 Report
Bentonville, Arkansas, Photographer Allegedly Used Grok to Create AI-Generated Child Sexual Abuse Material from Clients' Photos
Image generation technology developers
Incidents involved as Developer
- Incident 5613 Report
OpenAI Alleged by Lawsuit Violated Users' Privacy Rights by Training AI on Private Info without Informed Consent
- Incident 13423 Report
Purported Deepfake Nude Images of Students Circulated Without Consent at Valencia Educational Institute
Incidents implicated systems
Incidents impliqués en tant que développeur et déployeur
- Incident 5831 Report
Instagram Algorithms Allegedly Promote Accounts Facilitating Child Sex Abuse Content
- Incident 7881 Report
Instagram's Algorithm Reportedly Recommended Sexual Content to Teenagers' Accounts
Incidents involved as Deployer
Incidents implicated systems
Schools and teachers who were misinformed and burdened with COPPA compliance responsibilities without adequate disclosure
Affecté par des incidents
Replika
Incidents impliqués en tant que développeur et déployeur
Incidents implicated systems
Snapchat
Incidents impliqués en tant que développeur et déployeur
- Incident 5391 Report
Snapchat's My AI Reported for Lacking Protection for Children
- Incident 6461 Report
Snapchat's Algorithm Alleged to Link Minor with Sex Offenders
Incidents implicated systems
Deepfake creators
Incidents impliqués en tant que développeur et déployeur
Incidents involved as Deployer
Deepfake technology developers
Incidents involved as Developer
- Incident 95811 Report
Europol Operation Cumberland Investigates at Least 273 Suspects in 19 Countries for AI-Generated Child Sexual Abuse Material
- Incident 16238 Report
Pennsylvania's Lancaster Country Day School Students Allegedly Used AI to Create Hundreds of Sexualized Deepfake Images of Girls
Incidents implicated systems
Synthetic media generation technology developers
Incidents involved as Developer
- Incident 16238 Report
Pennsylvania's Lancaster Country Day School Students Allegedly Used AI to Create Hundreds of Sexualized Deepfake Images of Girls
- Incident 16238 Report
Pennsylvania's Lancaster Country Day School Students Allegedly Used AI to Create Hundreds of Sexualized Deepfake Images of Girls
Incidents implicated systems
Epistemic integrity
Affecté par des incidents
- Incident 95811 Report
Europol Operation Cumberland Investigates at Least 273 Suspects in 19 Countries for AI-Generated Child Sexual Abuse Material
- Incident 16238 Report
Pennsylvania's Lancaster Country Day School Students Allegedly Used AI to Create Hundreds of Sexualized Deepfake Images of Girls
Incidents implicated systems
Educators
Affecté par des incidents
- Incident 14922 Report
Purportedly AI-Enabled Targeting System Was Reportedly Implicated in Deadly U.S. Strike on Iranian Primary School
- Incident 16122 Report
A.B. Paterson College Teacher in Queensland Allegedly Used AI to Create Explicit Images of Students and Staff
Incidents involved as Deployer
Richland School District students
Affecté par des incidents
- Incident 8871 Report
Students of Richland School District in Cambria County, Pennsylvania Allegedly Used AI to Generate Obscene Images of Other Students
- Incident 8871 Report
Students of Richland School District in Cambria County, Pennsylvania Allegedly Used AI to Generate Obscene Images of Other Students
Incidents involved as Deployer
- Incident 8871 Report
Students of Richland School District in Cambria County, Pennsylvania Allegedly Used AI to Generate Obscene Images of Other Students
- Incident 8871 Report
Students of Richland School District in Cambria County, Pennsylvania Allegedly Used AI to Generate Obscene Images of Other Students
high school students
Affecté par des incidents
- Incident 137544 Report
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
- Incident 8881 Report
Sydney High Schooler Allegedly Generated Deepfakes of Other Students
Incidents involved as Deployer
Deepfake CSAM technology
Incidents involved as Developer
Incidents implicated systems
- Incident 95811 Report
Europol Operation Cumberland Investigates at Least 273 Suspects in 19 Countries for AI-Generated Child Sexual Abuse Material
- Incident 14482 Report
Ohio Man Pleaded Guilty after Prosecutors Alleged He Used AI to Create and Distribute Nonconsensual Intimate-Image Forgeries Including CSAM in Harassment Campaign
Character.ai
Incidents impliqués en tant que développeur et déployeur
- Incident 12775 Report
Alleged Harmful Outputs and Data Exposure in Children's AI Products by FoloToy, Miko, and Character.AI
- Incident 11082 Report
Digital Rights Groups Accuse Meta and Character.AI of Facilitating Unlicensed Therapy via Chatbots
Incidents implicated systems
Gaggle
Incidents impliqués en tant que développeur et déployeur
Incidents involved as Developer
Incidents implicated systems
Chatbot users
Affecté par des incidents
- Incident 11929 Report
16-Year-Old Allegedly Received Suicide Method Guidance from ChatGPT Before Death
- Incident 12001 Report
Meta AI on Instagram Reportedly Facilitated Suicide and Eating Disorder Roleplay with Teen Accounts
Incidents involved as Developer
Incidents involved as Deployer
Generative AI developers
Incidents involved as Developer
- Incident 14482 Report
Ohio Man Pleaded Guilty after Prosecutors Alleged He Used AI to Create and Distribute Nonconsensual Intimate-Image Forgeries Including CSAM in Harassment Campaign
- Incident 12031 Report
Carter County, Montana Man Reportedly Charged for Creating AI-Generated Child Sexual Abuse Material
Unknown actor(s) creating deepfake nudes of Sydney high school students
Incidents involved as Deployer
Family of students of Escuela Secundaria Técnica No. 1 in Zacatecas
Affecté par des incidents
Waymo
Incidents impliqués en tant que développeur et déployeur
- Incident 13003 Report
Waymo Self-Driving Vehicles Reportedly Passed Stopped School Buses at Least 19 Times, Prompting NHTSA Probe
- Incident 13612 Report
Waymo Autonomous Vehicle Reportedly Struck Child Near Elementary School in Santa Monica, California
Incidents implicated systems
ZeroEyes AI-powered video surveillance weapon detection system
Incidents implicated systems
Unknown generative AI systems
Incidents implicated systems
- Incident 14482 Report
Ohio Man Pleaded Guilty after Prosecutors Alleged He Used AI to Create and Distribute Nonconsensual Intimate-Image Forgeries Including CSAM in Harassment Campaign
- Incident 13181 Report
School's Suspected AI-Cheating Allegation Precedes Student's Reported Suicide in Greater Noida, India
Unnamed students at Mountain View Middle School
Affecté par des incidents
Incidents involved as Deployer
Boys
Affecté par des incidents
Incidents involved as Deployer
- Incident 13481 Report
Purported Deepfake Explicit Images of Middle School Students Allegedly Created and Circulated Using Mobile App in Goffstown, New Hampshire
- Incident 13521 Report
Malaysian Teenager Allegedly Arrested for Creating and Selling AI-Generated Deepfake Images of Schoolmates and Alumni in Johor
Unnamed students at Western Dubuque Community School District
Affecté par des incidents
Incidents involved as Deployer
Families of unnamed students at Western Dubuque Community School District
Affecté par des incidents
xAI
Incidents impliqués en tant que développeur et déployeur
Incidents involved as Developer
Lake Zurich High School students
Affecté par des incidents
- Incident 15741 Report
Purportedly AI-Generated Explicit Images of Students Allegedly Circulated at Lake Zurich High School in Illinois
- Incident 15741 Report
Purportedly AI-Generated Explicit Images of Students Allegedly Circulated at Lake Zurich High School in Illinois
Incidents involved as Deployer
Unidentified creators or distributors of AI-generated video targeting a Laredo, Texas, teenager
Incidents involved as Deployer
Synthetic image generation technology developers
Incidents involved as Developer
- Incident 16238 Report
Pennsylvania's Lancaster Country Day School Students Allegedly Used AI to Create Hundreds of Sexualized Deepfake Images of Girls
- Incident 16238 Report
Pennsylvania's Lancaster Country Day School Students Allegedly Used AI to Create Hundreds of Sexualized Deepfake Images of Girls