AI Incident Report for July and August 2022

Sep 12, 2022

The Artificial Intelligence Incident Database (AIID) contains a wealth of historical instances of AI harms providing insights for the future development and deployment of artificial intelligence. This compilation of AI incidents published in July and August of 2022 highlights emerging incidents and provides a digest of recent additions to the database.

From troubling bias and censorship concerns to physical injury, July and August saw a variety of headlines from emerging incidents in AI. Chatbots made offensive and accusatory statements, while a robot infamously broke a child’s finger during a match while on camera. Language translation and image rendering software made waves with offensive content.

A deeper dive in the database from the July and August feed reveals a broader variety of incidents. With increasing deployments and competition in the autonomous vehicle space, the number of reports received from United States roads has seen an uptick. The big tech companies like Google and the social networks continue to make news with algorithms gone awry. Adding to this are a variety of new reports on policing and enforcement in the real world showing a variety of bias and safety harms.

Emerging Incidents

Incidents that occurred in July/August.

07/18/2022: Incident 285: Google Lens’s Camera Translation Feature Provided an Offensive Mistranslation of a Book Title in Korean

07/21/2022: Incident 241: Chess Robot Broke Child's Finger in Russia (5 reports)

07/24/2022: Incident 271: Tesla Model 3 Sedan on Autopilot Killed Motorcyclist in a Rear Collision in Utah

08/07/2022: Incident 278: Meta’s BlenderBot 3 Chatbot Demo Made Offensive Antisemitic Comments, Acknowledged by Developers as Occasional Errors

08/17/2022: Incident 314: Stable Diffusion Abused by 4chan Users to Deepfake Celebrity Porn

08/21/2022: Incident 303: Google’s Automated Child Abuse Detection Wrongfully Flagged a Parent’s Naked Photo of His Child

08/25/2022: Incident 313: BlenderBot 3 Cited Dutch Politician as a Terrorist

How to find this collection in the database:

Go to the “Table View” in the menu of the AIID and type 2022-07 for incidents occurring in July and 2022-08 for incidents occurring in August.

New Incidents in the Database

A digest of all AI Incidents added to the database in July/August.

The following incidents are new to the AIID, but they are submissions of incidents that occurred before the last two months.

AI physical injury or risk of injury to humans

Incident 241: Chess Robot Broke Child's Finger in Russia

Incident 242: Manufacturing Robot Failure Caused Factory Worker's Death in India

Incident 252: Remotely Operated Taser Drones Proposed by Taser Manufacturer as Defense for School Shootings in the US

Incident 261: Robot Deployed by Animal Shelter to Patrol Sidewalks outside Its Office, Warding off Homeless People in San Francisco

Incident 279: TikTok’s “For You” Algorithm Exposed Young Users to Pro Disorder Content

Incident 281: YouTube's Algorithms Failed to Remove Violating Content Related to Suicide and Self

Incident 286: TikTok’s Recommendation System Alleged in a Lawsuit to Have Pushed and Incentivized Two Young Girls in Participating in the “Blackout” Challenge, Resulting in Their Deaths

Incident 290: False Negatives for Water Quality Beach Closures

Autonomous vehicle issues

Incident 253: Cruise's Self Cars Allegedly Lost Connection to Their Server, Causing Traffic Blockages in San Francisco

Incident 271: Tesla Model 3 Sedan on Autopilot Killed Motorcyclist in a Rear Collision in Utah

Incident 289: Starship Delivery Robot Scuffed Bumper of a Resident’s Car in Texas, Allegedly Refusing to Release Footage of the Accident

Incident 291: Tesla Allegedly Misled Customers about Autopilot and FSD Capabilities

Incident 292: Apple’s AVs Reportedly Struggled to Navigate Streets in Silicon Valley Test Drives

Incident 293: Cruise’s Self Car Involved in a Multiple Collision at an San Francisco Intersection

Incident 294: Tesla Autopilot Allegedly Malfunctioned in a Non Collision in Greece

Incident 297: EasyMile Self Shuttle Unexpectedly Stopped Mid, Injuring a Passenger

Incident 304: Tesla on FSD Reportedly Drove into the Wrong Lane in California

Incident 306: Tesla on Autopilot TACC Crashed into Van on European Highway

Incident 319: Tesla on Autopilot Fatally Crashed into Parked Fire Truck in Indiana

Incident 320: Tesla on Autopilot Collided with Parked Fire Truck on California Freeway

Incident 321: Tesla Model X on Autopilot Crashed into California Highway Barrier, Killing Driver

Incident 322: Tesla Model 3 Crashed into Police Patrol Car on Connecticut Highway

Incident 323: Tesla on Autopilot Crashed into Parked Police Car in California

Policing practices using AI

Incident 244: Colorado Police’s Automated License Plate Reader (ALPR) Matched a Family’s Minivan’s Plate to That of a Stolen Vehicle Allegedly, Resulting in Detainment at Gunpoint

Incident 245: Unverified Misreading by Automated Plate Reader Led to Traffic Stop and Restraint of an Innocent Person at Gunpoint in California

Incident 246: Misreading of an Automated License Plate Reader (ALPR) Unverified by Police, Resulting in Traffic Stop in Missouri

Incident 248: Automated License Plate Camera Notified Police about a Previously Stolen Rental Car that was Returned, Causing an Innocent Person to be Detained at Gunpoint in California

Incident 255: Gunshot Detection Audio Picked up by ShotSpotter’s Algorithms Previously Admitted as Evidence of Conviction in a Murder Case in Chicago Now Dismissed due to Reports of Unreliability

Incident 256: DUI Arrest Case in Chicago Alleged in a Lawsuit to Have Based Solely on an Alert by ShotSpotter, Citing Concerns of Unreliability

Incident 257: Police Departments Reportedly Deployed Gunshot Detection Sensors Disproportionately in Black and Brown Neighborhoods

Incident 260: US Department of Homeland Security (DHS)’s Opaque Immigration Software Criticized by Advocacy Groups for Allegedly Relying on Poor Data and Targeting Immigrants Via Sensitive Information

Incident 264: AI Vehicle Speed Estimation App Denounced by UK Drivers as Surveillance Technology

Incident 268: Permanent Removal of Social Media Content via Automated Tools Allegedly Prevented Investigative Efforts

Incident 288: New Jersey Police Misidentified an Innocent Black Man via Facial Recognition, Resulting in Wrongful Arrest and Alleged Violation of Civil Rights

Incident 295: Wrongful Attempted Arrest for Apple Store Thefts Due to NYPD’s Facial Misidentification

Incident 309: UK Police’s FRT Trial Performed Poorly at Notting Hill Carnival

Incident 310: High False Positive Rate by SWP's Facial Recognition Use at Champion's League Final

AI bias

Incident 259: YouTuber Built, Made Publicly Available, and Released Model Trained on Toxic 4chan Posts as Prank

Incident 262: DALL Mini Reportedly Reinforced or Exacerbated Societal Biases in Its Outputs as Gender and Racial Stereotypes

Incident 265: Black Delivery Driver Allegedly Discriminated by Uber Eats for Excessive Facial Photo Verification Checks and Dismissed Based on False Facial Recognition Results

Incident 275: Facebook’s Content Moderation Algorithm Incorrectly Banned Users for a Photo of Aboriginal Men in Chains as Historical Evidence of Slavery

Incident 278: Meta’s BlenderBot 3 Chatbot Demo Made Offensive Antisemitic Comments

Incident 280: Coffee Meets Bagel’s Algorithm Reported by Users Disproportionately Showing Them Matches of Their Own Ethnicities Despite Selecting “No Preference”

Incident 300: TikTok's "For You" Algorithm Allegedly Abused by Online Personality to Promote Anti Hate

Incident 312: Startup's Accent Translation AI Denounced as Reinforcing Racial Bias

Facial recognition issues

Incident 254: Google’s Face Grouping Feature Allegedly Collected and Analyzed Users’ Facial Structure without Consent or Notice, in Direct Violation of Illinois Biometric Information Privacy Act

Incident 258: Australian Retailers Reported to Have Captured Face Prints of Their Customers without Consent, Potentially in Violation of Australian Privacy Act

Incident 267: Clearview AI’s Algorithm Built on Photos Scraped from Social Media Profiles without Consent

Incident 273: FaceApp Detected Different Genders for Similar User Photos with Slight Variation in Eyebrow Thickness

Incident 276: Local South Korean Government’s Use of CCTV Footage Analysis via Facial Recognition to Track COVID Cases Raised Concerns about Privacy, Retention, and Potential Misuse

Incident 298: Student Facial Recognition App Raised Ethical Concerns

Incident 307: iPhone Face ID Failed to Recognize Users’ Morning Faces

Incident 315: Facial Recognition Service Abused to Target Russian Porn Actresses

Harmful algorithms

Incident 250: Dutch City Court Defended Home Value Generated by Black Algorithm

Incident 251: Amazon Allegedly Tweaked Search Algorithm to Boost Its Own Products

Incident 263: YouTube’s Content Recommendation Systems Implicated in One User’s Extreme Political Radicalization

Incident 270: Apple Tweaked App Store Ranking Algorithms, Allegedly Resulted in Demotion of Local Apps in China

Incident 272: Grab Tweaked Matchmaking Algorithm, Providing Preferential Treatment to Drivers Registered with Affiliated Car Rental Service

Incident 274: Virginia Courts’ Use of Algorithmic Nonviolent Risk Assessment Failed to Result in Lower Incarceration Rates or Sentencing and Neither Increased or Decreased Historical Racial Bias in Sentencing

Incident 282: Facebook’s Algorithm Mistook an Advertisement of Onions as Sexual Suggestive Content

Incident 296: Twitter Recommender System Amplified Right Tweets

Incident 302: Students Allegedly Wrongfully Accused of Cheating via Medical School's Internal Software

Incident 305: YouTube’s Recommendation Algorithm Allegedly Promoted Climate Misinformation Content

Incident 316: Facebook Ad Algorithm Allegedly Missed Fraudulent Ads via Simple URL Checks

Incident 318: Facebook Recommended Military Gear Ads Despite Pause on Weapons Accessories Ads

Censorship Issues

Incident 283: Facebook’s Automated Content Moderation Tool Flagged a Post Containing Parts of the Declaration of Independence as Hate Speech by Mistake

Incident 284: Facebook’s Removal of Content Containing Artworks and Their Advertisements That Featured Nudity via Automated and Human Moderation Denounced as Censorship

Incident 303: Google’s Automated Child Abuse Detection Wrongfully Flagged a Parent’s Naked Photo of His Child

Incident 311: YouTube Auto Mistakenly Banned Women of Sex Tech Conference

Incident 317: Bug in Facebook’s Anti Filter Allegedly Blocked Legitimate Posts about COVID

Deepfake Issues

Incident 299: Japanese Porn Depixelated by Man using Deepfake

Incident 314: Stable Diffusion Abused by 4chan Users to Deepfake Celebrity Porn

Other Incidents

Incident 243: Bots Allegedly Made up Roughly Half of Twitter Accounts in Discussions Surrounding COVID Related Issues

Incident 249: Government Deployed Extreme Surveillance Technologies to Monitor and Target Muslim Minorities in Xinjiang

Incident 266: AI Partners Created by Chatbot App Replika Abused by Its Users
Incident 277: Voices Created via Publicly Available AI Generation App 15.ai Stolen and Resold as NFT without Acknowledgement

Incident 285: Google Lens’s Camera Translation Feature Provided an Offensive Mistranslation of a Book Title in Korean

Incident 287: OpenAI’s GPT Reported by Healthcare Company as Unviable in Medical Tasks Due to Its Erratic and General Nature

Incident 308: Atlas Robot Fell off Stage at Conference

Incident 313: BlenderBot 3 Cited Dutch Politician as a Terrorist

How to find this collection in the database:

Go to the “Table View” in the menu of the AIID and view Incident IDs after #241.

Support our efforts:

Still reading? Help us change the world for the better! Submit new incidents to the Database, contribute to building out Database functionality, and start a conversation with us on LinkedIn, Facebook, or Twitter.