Incident 132: TikTok’s Content Moderation Allegedly Failed to Adequately Take down Videos Promoting Eating Disorders
Suggested citation format
TikTok said it banned six accounts reported to it for posting content promoting eating habits that are likely to lead to health problems in its latest effort to crackdown on harmful content.
The app is rife with dangerous material including 'pro-ana' or pro-anorexia and 'pro-mia' or pro-bulimia content which has plagued other social media networks such as Tumblr in the past.
Its algorithm means if users engage with harmful content then TikTok will continue to show them it, and even if they don't engage with it, it may appear on their For You Page regardless.
The app has 800m active users, 41% of whom are aged between 16 and 24, according to Datereportal and Globalwebindex.
TikTok said that it had banned six accounts for posting content promoting eating habits that are likely to lead to health problems in its latest effort to crack down on harmful content.
The ban, earlier in December, came after a Guardian investigation found pro-anorexia material was still searchable on the platform.
The app has struggled with dangerous material including 'pro-ana' or pro-anorexia and 'pro-mia' or pro-bulimia content which plagued other social media networks such as Tumblr in the past.
The platform banned content labeled #proana and #anorexia which had 2.1m and 446,000 views respectively in the summer, according to Mashable UK. Users who searched for them were redirected to a support page titled "Need Help?"
However, if they are entered as words rather than hashtags then the content remains accessible and users have also been able to bypass the ban using common misspellings of popular search terms.
Tom Quinn, Director of External Affairs at eating disorder charity, Beat told Insider: "It is vital that TikTok continue to update their safety procedures in order to reduce the ability of their users to post or find harmful content. This should include identifying where misspellings of common hashtags are used in order to circumvent the rules."
Danae Mercer, a journalist and body image influencer based in the UAE began talking about body confidence for various reasons including struggling with an eating disorder when she was 19.
She told Insider: "Social media, like the pro-ana communities on Tumblr and websites, discussing celebrity weight loss really escalated my illness so I'm trying to help make social media a safer space, the kind I needed when I was sick.
"Those communities massively fuel eating disorders, helping them become a cycle where the illness is encouraged, tips are shared, and being sicker is applauded."
How TikTok works
TikTok has 800m active users worldwide, 41% of whom are aged between 16 and 24, according to Datereportal and Globalwebindex.
TikTok is unique from other networks because of its For You Page (FYP) algorithm, the inner workings of which the Chinese owned app recently revealed in an effort to be more transparent.
When a video is uploaded, it is shown to a small group of users regardless of whether they follow the account it came from or not.
If they engage with the video, either by liking, sharing, or even watching it all, then it will be presented to an even larger group.
If this larger group reacts well, then it is shown to an even larger group and the process continues until the clip eventually goes viral.
Therefore, if users engage with harmful content then TikTok will continue to show them it, trapping them in a vicious cycle, and even if they don't engage with it, it may appear on their FYP regardless.
In a statement, TikTok said users searching for eating disorder-related content will be redirected to the National Eating Disorder Association (NEDA) helpline and that it would prohibit ads promoting fasting apps and weight loss gimmicks to under 18s.
Last year, Pinterest also tackled this issue by training its algorithm to recognize content that promotes self-harm with the company reporting that those pins were down by 88%, Wired reported.
When users specifically search for the remaining Pins, Pinterest instead recommends users a series of wellbeing exercises.
Did our AI mess up? Flag the unrelated incidents