Incident 281: YouTube's Algorithms Failed to Remove Violating Content Related to Suicide and Self-Harm
Description: Terms-of-service-violating videos related to suicide and self-harm reportedly bypassed YouTube’s content moderation algorithms, allegedly resulting in exposure of graphic content to young users via recommended videos.
Entities
View all entitiesAlleged: YouTube developed and deployed an AI system, which harmed YouTube young users and YouTube users.
Incident Stats
Incident ID
281
Report Count
3
Incident Date
2019-02-04
Editors
Khoa Lam
Incident Reports
Reports Timeline
Speaking to The Telegraph, a former Tumblr blogger, who asked for anonymity, said she had to stop her own depression and anxiety help blog after she found herself “falling down the rabbit hole of content that triggered negative emotions”.
“…
Potentially harmful content has reportedly slipped through the moderation algorithms again at YouTube.
According to a report by The Telegraph on Monday, YouTube has been recommending videos that contain graphic images of self-harm to users …
YouTube has been caught recommending "dozens" of graphic videos relating to self-harm to children.
The hugely popular video sharing app has been blasted for promoting dangerous clips to users as young as 13.
YouTube is under fire for failin…
Variants
A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.
