AI-enabled decision support systems
Incidents implicated systems
Incident 137544 Report
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
2026-02-10
After the 02/10/2026 school shooting in Tumbler Ridge, British Columbia, OpenAI said a user later identified as the suspect, Jesse Van Rootselaar, had previously used ChatGPT to describe scenarios involving gun violence. Those chats were reportedly auto-flagged and reviewed and the account was banned, but OpenAI says it did not alert the RCMP because it allegedly did not present a credible, imminent threat; the company reportedly contacted police after the attack.
MoreIncident 3734 Report
Amazon’s Experimental Hiring Tool Allegedly Displayed Gender Bias in Candidate Rankings
2016-08-10
Between 2014 and 2017, Amazon reportedly developed an AI-powered recruiting tool to score job applicants, trained on a decade of resumes purportedly drawn largely from men. Media reports say the system learned to favor male candidates, penalizing terms like "women's" and graduates from certain all-women's colleges. Efforts to remove these biases reportedly did not guarantee fairness, and the project was ultimately abandoned. Amazon reportedly states recruiters never solely relied on the tool.
MoreIncident 4021 Report
COMPAS Algorithm Reportedly Performs Poorly in Crime Recidivism Prediction
2016-05-23
Correctional Offender Management Profiling for Alternative Sanctions (COMPAS), a recidivism risk-assessment algorithmic tool used in the judicial system to assess likelihood of defendants' recidivism, is found to be less accurate than random untrained human evaluators.
MoreIncident 60821 Report
UnitedHealth Accused of Deploying Allegedly Flawed AI to Deny Medical Coverage
2023-02-28
UnitedHealthcare allegedly used a faulty AI algorithm with a 90% error rate to override doctors' recommendations and deny health coverage. This AI, developed by NaviHealth, reportedly led to premature discharge from care facilities and substantial out-of-pocket expenses for patients, according to a lawsuit filed in the District Court for Minnesota.
MoreRelated Entities
Other entities that are related to the same incident. For example, if the developer of an incident is this entity but the deployer is another entity, they are marked as related entities.
Related Entities
Educational communities
Incidents Harmed By
- Incident 137544 Reports
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
- Incident 19512 Reports
Predictive Policing Program by Florida Sheriff’s Office Allegedly Violated Residents’ Rights and Targeted Children of Vulnerable Groups
Incidents involved as Deployer
Associated machine learning models trained on historical Amazon resume data
Incidents implicated systems
St George’s Hospital Medical School automated admissions screening program
Incidents implicated systems
Government of Argentina
Incidents involved as both Developer and Deployer
Incidents involved as Deployer
Minors
Incidents Harmed By
- Incident 137544 Reports
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
- Incident 1867 Reports
Algorithm Assessing Risk Faced by Victims of Gender Violence Misclassified Low-Risk Cases, Allegedly Leading to Homicide of Women and Children in Spain
Law enforcement facial recognition systems
Incidents implicated systems
- Incident 4124 Reports
Finnish Police Were Reportedly Reprimanded After National Bureau of Investigation Unit Allegedly Used Clearview AI to Identify Potential Abuse Victims
- Incident 15452 Reports
Florida Police Reportedly Wrongfully Arrested Commercial Crabber Robert Dillon After Allegedly False Facial Recognition Match
Chicago Police Department
Incidents involved as both Developer and Deployer
Incidents Harmed By
Incidents involved as Deployer
ShotSpotter
Incidents involved as Developer
- Incident 11215 Reports
Police Departments Reported ShotSpotter as Unreliable and Wasteful
- Incident 2559 Reports
Unreliable ShotSpotter Audio Previously Used to Convict Chicago Man in Murder Case
Incidents implicated systems
University of Texas at Austin's Department of Computer Science
Incidents involved as Deployer
Unknown
Incidents Harmed By
- Incident 5172 Reports
Man Arrested For Sock Theft by False Facial Match Despite Alibi
- Incident 5181 Report
New York Detective Reportedly Misused Woody Harrelson's Face to Perform Face Recognition Search
Incidents involved as Developer
Microsoft
Incidents involved as Developer
- Incident 1884 Reports
Argentinian City Government Deployed Teenage-Pregnancy Predictive Algorithm Using Invasive Demographic Data
- Incident 14002 Reports
West Midlands Police Reportedly Relied on Erroneous Copilot-Generated Intelligence in Maccabi Tel Aviv Away-Fan Ban Decision
Incidents involved as Deployer
Intensive Partnerships for Effective Teaching
Incidents involved as both Developer and Deployer
Intensive Partnerships for Effective Teaching teacher-evaluation system
Incidents implicated systems
Law enforcement
Incidents Harmed By
- Incident 4465 Reports
ShotSpotter Failed to Alert Authorities of Mass Shooting in North Carolina
- Incident 11911 Report
NYPD Facial Recognition System Allegedly Produced Erroneous Match That Reportedly Resulted in Wrongful Detention of Trevis Williams
Incidents involved as Deployer
Clearview AI
Incidents involved as Developer
- Incident 4406 Reports
Louisiana Police Wrongfully Arrested Black Man Using False Face Match
- Incident 4124 Reports
Finnish Police Were Reportedly Reprimanded After National Bureau of Investigation Unit Allegedly Used Clearview AI to Identify Potential Abuse Victims
Incidents implicated systems
Chatbot developers
Incidents involved as Developer
- Incident 137544 Reports
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
- Incident 5385 Reports
Texas A&M-Commerce Professor Reportedly Used ChatGPT to Accuse Students of Cheating and Assign Incomplete Grades
Incidents implicated systems
Large language model developers
Incidents involved as Developer
- Incident 137544 Reports
OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
- Incident 5385 Reports
Texas A&M-Commerce Professor Reportedly Used ChatGPT to Accuse Students of Cheating and Assign Incomplete Grades
Incidents implicated systems
Black men who underwent lung function tests between 2010 and 2020 and potentially received inaccurate or delayed diagnoses and medical interventions due to the biased algorithm
Incidents Harmed By
NaviHealth
Incidents involved as both Developer and Deployer
Incidents involved as Developer
Turnitin
Incidents involved as both Developer and Deployer
- Incident 5141 Report
Turnitin's ChatGPT-Detection Tool Falsely Flagged Student Essays as AI-Generated
Incidents involved as Developer
- Incident 5721 Report
University Student Reportedly Received Zero After Turnitin AI Detector Flagged Essay
- Incident 8491 Report
AI Detection Tools Allegedly Misidentify Neurodivergent and ESL Students' Work as AI-Generated in Academic Settings
Incidents implicated systems
lawyers and other experts who were not informed of the tool's limitations
Incidents Harmed By
National security and intelligence stakeholders
Incidents Harmed By
Incidents involved as Deployer
Department of Veterans Affairs (VA)
Incidents involved as both Developer and Deployer
Incidents Harmed By
GPTZero
Incidents involved as Developer
- Incident 8491 Report
AI Detection Tools Allegedly Misidentify Neurodivergent and ESL Students' Work as AI-Generated in Academic Settings
- Incident 16321 Report
Yale Student Suspended and Given F After GPTZero Allegedly Misclassified Exam as AI-Generated
Incidents implicated systems
Wrongfully arrested individuals
Incidents Harmed By
- Incident 15452 Reports
Florida Police Reportedly Wrongfully Arrested Commercial Crabber Robert Dillon After Allegedly False Facial Recognition Match
- Incident 8961 Report
Alleged Misuse of Facial Recognition Technology by Law Enforcement Reportedly Leading to Wrongful Arrests and Violations of Investigative Standards
Real-time facial recognition alert pipeline to New Orleans Police Department
Incidents implicated systems
Automated person-tracking via clothing and physical descriptors
Incidents implicated systems
Unspecified developer of aerial imagery risk analysis system
Incidents involved as Developer
Department of Government Efficiency (DOGE)
Incidents involved as both Developer and Deployer
Incidents involved as Deployer
Gaggle
Incidents involved as both Developer and Deployer
Incidents involved as Developer
Incidents implicated systems
Judicial integrity
Incidents Harmed By
- Incident 14167 Reports
Purported Facial Recognition Error Reportedly Led to Arrest and Monthslong Jailing of Tennessee Woman in North Dakota Fraud Case
- Incident 11911 Report
NYPD Facial Recognition System Allegedly Produced Erroneous Match That Reportedly Resulted in Wrongful Detention of Trevis Williams
Omnilert
Incidents involved as Developer
- Incident 12503 Reports
Alleged False Positive by Omnilert AI Gun Detection System Prompts Police Search at Baltimore County High School
- Incident 12671 Report
Omnilert AI Reportedly Triggered False Gun Alert at Parkville High, Prompting Student Relocation
Incidents implicated systems
ZeroEyes AI-powered video surveillance weapon detection system
Incidents implicated systems
United States Customs and Border Protection
Incidents involved as Deployer
- Incident 13531 Report
ICE Facial Recognition App Mobile Fortify Reportedly Misidentified Woman Twice During Immigration Enforcement in Oregon
- Incident 13621 Report
Border Patrol Agent Allegedly Claimed Facial Recognition Identified Minneapolis ICE Observer and Global Entry Was Reportedly Revoked Three Days Later
United States Department of Homeland Security
Incidents involved as Developer
Incidents involved as Deployer
Unnamed woman detained by United States Immigration and Customs Enforcement
Incidents Harmed By
Mobile Fortify
Incidents implicated systems
- Incident 13531 Report
ICE Facial Recognition App Mobile Fortify Reportedly Misidentified Woman Twice During Immigration Enforcement in Oregon
- Incident 13621 Report
Border Patrol Agent Allegedly Claimed Facial Recognition Identified Minneapolis ICE Observer and Global Entry Was Reportedly Revoked Three Days Later