Incident 133: Online Trolls Allegedly Abused TikTok’s Automated Content Reporting System to Discriminate against Marginalized Creators

Description: TikTok's automated content reporting system was allegedly abused by online trolls to intentionally misreport content created by users of marginalized groups.
Alleged: TikTok developed and deployed an AI system, which harmed TikTok content creators of marginalized groups.

Suggested citation format

Hall, Patrick. (2020-12-15) Incident Number 133. in McGregor, S. (ed.) Artificial Intelligence Incident Database. Responsible AI Collaborative.

Incident Stats

Incident ID
Report Count
Incident Date
Sean McGregor, Khoa Lam


New ReportNew ReportDiscoverDiscover

Incidents Reports

My name is Rosalynne (Rose) Montoya, I am a Latina, bisexual, transgender woman. I am a social media content creator and before Monday December 14th, 2020, I had grown my audience to 300K+ followers on my TikTok account. (@RosalynneMontoya).

One of my videos about injecting my estrogen was reported. Shortly after I posted it, I received a notification that my account was permanently deleted and banned for violating the community guidelines. This was wrong. I’m always sure to follow the community guidelines of the platform I’m posting on. Myself, alongside many other creators, especially BIPOC, LGBTQPIA+, and those living with disabilities, are being targeted by trolls who are intentionally falsely reporting our content with the goal to delete our videos from the app.

TikTok’s algorithm allows these discriminatory actions to censor our content.I have appealed the decision to delete my account, but received the notification that they have reviewed my appeal and “it was determined that my account violates community guidelines and cannot be restored.” I am eagerly waiting TikTok’s email for a full explanation.

I use my social media accounts to educate people. I share my story, my transition, my surgeries, etc. I also invite my sister who is Deaf to help me teach people ASL.

I recently started a series on TikTok about trans history after receiving a hate comment stating that being trans is just a new trendy thing, when in fact we have always existed. I often respond to hate comments with humor and education. I strive to live by the words: kill them with kindness. Working as an educator can be difficult, but it’s important and fulfilling as well. I receive thousands of messages from people thanking me for the work I do, for inspiring them, and even for changing their minds.

This last month, as my following has exponentially grown, I’ve noticed more and more hate comments and trolls. Having been a content creator for a long time, I’m used to blocking hate comments and even death threats. However, the community guidelines violations on TikTok work differently from the other platforms I use. The guidelines in place are general and vague. There is little transparency on why TikTok removes or does not remove content. They also do not give reasons on why an appeal was denied. The community guidelines should be in place to protect the community. Creators shouldn’t have to put up with discrimination, hateful comments, or death threats. The algorithm should work to protect marginalized people, not discriminate against us.

From what I’ve gathered, if enough people report a video on TikTok (even if it’s a false report) the video is removed automatically, regardless of whether the video actually violated the guidelines or not. This can be appealed and if approved by a computer program, the content will be available again.

The problem, though, is that this leaves a flag on the account reported. After enough violations (even if appealed and deemed to be appropriate) an account can be deleted. The current algorithm is flawed and is allowing many people who are breaking the community guidelines to slip through the cracks, and many of us who follow them are being deleted and censored due to hateful trolls with an agenda to have our content removed from the app.

In the last month I have had many videos reported and removed due to false claims of violating TikTok’s community guidelines. The majority of my appeals were approved and most of my content was allowed back on TikTok. At the same time, I was also warned multiple times while live-streaming that my content was ‘vulgar’, even though I wasn’t breaking any of the community guidelines.

After being continuously reported, my appeals to go live were declined. I was banned from live more than 4 times for being a trans Latina woman while speaking about beauty tips, music, and my transition while wearing tank tops or bralettes. Each time I was reported the timespan of the ban was increased.

In the few weeks before my account was deleted, I had to consistently block the same person over and over ( ). His goal was to use my content for his own gain while spreading discrimination against my community. He would screen record my content (including a video with my 17 year Deaf old sister) then use it to spread hate on his page. He has reposted at least 7 of my videos (without tagging me) followed by his crude commentary.

These were brought to my attention thanks to my followers who sent them to me. I reported these videos, but TikTok’s algorithm concluded that they do not violate their community guidelines. After blocking him, he made another account and spammed my videos. Every time I blocked him, he came back again and eventually his friends joined in as well. From having so many wrongful reports of violating community guidelines, I was notified that my bio had been deleted. I appealed this and it was immediately restored. I had no idea these trolls held the power to have my entire account deleted from the app. These creators laughed when my account was deleted and are already spamming my new account.

After my appeal was denied, I went live on my new TikTok account, one of the trolls entered the live and reported me. I was banned from livestreaming for 24hours for “serious pornography.” I was wearing a tanktop.

Many fellow creators have noticed that they are also being targeted for being BIPOC, LGBTQPIA+, living with a disability, etc. Our content is being wrongfully reported by discriminatory haters. As a result, TikTok’s algorithm allows our content to be censored, banned, and deleted. We are just as deserving to use the app as these other people. Internet trolls should not hold more power than content creators.

I would like to ask TikTok to sit down with myself and other creators to discuss their community guidelines, and how they can improve the app to be more inclusive. I want my account back and the funds I accrued through the TikTok Creator Fund, but I also want to see real change to the algorithm and transparency with how their community guidelines function.

Real people—not just computer programs— should be involved before accounts are deleted. The community guidelines should not be able to be used as a tool to discriminate and target marginalized people.

I applaud other social media platforms that have created advisory boards where content creators can hold the platform accountable. I want to create content on platforms that protect the community, but I don’t wish to create content on platforms that censor my content simply for being trans and Latina.

Rose’s ‘TikTok Deleted My Profile Because I’m A Latinx Trans Woman’ YouTube video:

Sign her petition here:

TikTok Deleted My Account Because I’m a Latina Trans Woman