Incident 133: Online Trolls Allegedly Abused TikTok’s Automated Content Reporting System to Discriminate against Marginalized Creators

Description: TikTok's automated content reporting system was allegedly abused by online trolls to intentionally misreport content created by users of marginalized groups.


New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History
Alleged: TikTok developed and deployed an AI system, which harmed TikTok content creators of marginalized groups.

Incident Stats

Incident ID
Report Count
Incident Date
Sean McGregor, Khoa Lam

GMF Taxonomy Classifications

Taxonomy Details

Known AI Goal

Automated Content Curation

Potential AI Technology

Visual Object Detection, Diverse Data

Potential AI Technical Failure

Lack of Transparency, Misconfigured Threshold, Misconfigured Aggregation, Incomplete Data Attribute Capture

TikTok Deleted My Account Because I’m a Latina Trans Woman · 2020

My name is Rosalynne (Rose) Montoya, I am a Latina, bisexual, transgender woman. I am a social media content creator and before Monday December 14th, 2020, I had grown my audience to 300K+ followers on my TikTok account. (@RosalynneMontoya)…


A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.