Incident 185: TikTok's "For You" Algorithm Directed New Users towards Disinformation about the War in Ukraine

Description: An investigation by NewsGuard into TikTok’s handling of content related to the Russia-Ukraine war showed its “For You” algorithm pushing new users towards false and misleading content about the war within less than an hour of signing up.
Alleged: TikTok developed and deployed an AI system, which harmed TikTok users and TikTok new users.

Suggested citation format

AIAAIC. (2022-03-01) Incident Number 185. in McGregor, S. (ed.) Artificial Intelligence Incident Database. Responsible AI Collaborative.

Incident Stats

Incident ID
185
Report Count
4
Incident Date
2022-03-01
Editors
Sean McGregor, Khoa Lam

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscover

Incident Reports

The Chinese-owned video app is dealing with a flood of war videos and the question of whether it is spreading unverified information.

Bre Hernandez used to scan TikTok for videos of makeup tutorials and taco truck reviews. Since Russia invaded Ukraine, the 19-year-old has spent hours each day scrolling the app for war videos, watching graphic footage of Ukrainian tanks firing on Russian troops and civilians running away from enemy gunfire.

“What I see on TikTok is more real, more authentic than other social media,” said Ms. Hernandez, a student in Los Angeles. “I feel like I see what people there are seeing.”

But what Ms. Hernandez was actually viewing and hearing in the TikTok videos was footage of Ukrainian tanks taken from video games, as well as a soundtrack that was first uploaded to the app more than a year ago. The footage and soundtrack were traced back to their original sources in a New York Times analysis of the videos.

TikTok, the Chinese-owned video app known for viral dance and lip-syncing videos, has emerged as one of the most popular platforms for sharing videos and photos of the Russia-Ukraine war. Over the past week, hundreds of thousands of videos about the conflict have been uploaded to the app from across the world, according to a review by The Times. The New Yorker has called the invasion the world’s “first TikTok war.”

The surge has put TikTok in a challenging position. For the first time, it is dealing with moderating a flood of videos — many of them unverified — about a single event that has captivated a global audience. That is leading it to essentially confront a large scale of misleading and distorted information that has long bedeviled more mature social networks and video sites, such as YouTube, Facebook and Instagram.

Many popular TikTok videos of the invasion — including of Ukrainians livestreaming from their bunkers — offer real accounts of the action, according to researchers who study the platform. But other videos have been impossible to authenticate and substantiate. Some simply appear to be exploiting the interest in the invasion for views, the researchers said.

In one example, Pravda, a Ukrainian newspaper, posted an audio clip featuring 13 Ukrainian soldiers on Snake Island, an outpost of the Black Sea, facing a Russian military unit that asked them to surrender. The clip was then used in many TikTok videos, some of which included a note stating that all 13 soldiers had died. Ukrainian officials later said in a Facebook post that the men were alive and had been taken prisoner, but the TikTok videos have not been corrected.

“There are people who are, right now, seeing war for the first time on TikTok,” said Abbie Richards, an independent researcher who studies the app. “People trust it. The result is that a lot of people are seeing false information about Ukraine and believing it.”

TikTok and other social media platforms are also under pressure from U.S. lawmakers and Ukrainian officials to curb Russian misinformation about the war, especially from state-backed media outlets such as Russia Today and Sputnik. In response, YouTube has said it would block Russia Today and Sputnik in the European Union, while Twitter and Meta, the parent of Facebook, have said they would label content from the outlets as state sponsored.

TikTok has also banned Sputnik and Russia Today in the E.U., and on Friday said it would start labeling the outlets as state-sponsored in the countries where they are still available. The app also said on Thursday that it had dedicated more resources to monitoring for misleading content about the war.

“We continue to respond to the war in Ukraine with increased safety and security resources to detect emerging threats and remove harmful misinformation,” said Hilary McQuaide, a TikTok spokeswoman. On Sunday, TikTok said it would suspend livestreaming and new content being uploaded from Russia.

For years, TikTok largely escaped sustained scrutiny about its content. Unlike Facebook, which has been around since 2004, and YouTube, which was founded in 2005, TikTok only became widely used in the past five years. Owned by China’s ByteDance, the app was designed to make one- to three-minute videos easy to create and share. It developed a reputation as a destination for addictive, silly and fun videos, especially for young users.

The app has navigated some controversies in the past. It has faced questions over harmful fads that appeared to originate on its platform, as well as whether it allows underage users and adequately protects their privacy.

But the war in Ukraine has supersized the issues facing TikTok, which has over one billion users globally.

The volume of war content on the app far outweighs what is found on some other social networks, according to a review by The Times. Videos with the hashtag #Ukrainewar have amassed nearly 500 million views on TikTok, with some of the most popular videos gaining close to one million likes. In contrast, the #Ukrainewar hashtag on Instagram had 125,000 posts and the most popular videos were viewed tens of thousands of times.

The very features that TikTok designed to help people share and record their own content have also made it easy to spread unverified videos across its platform. That includes TikTok’s algorithm for its “For You” page, which suggests videos based on what people have previously seen, liked or shared. Viewing one video with misinformation likely leads to more videos with misinformation being shown, Ms. Richards said.

Another popular TikTok feature lets people easily reuse audio, which has enabled people to create lip-syncing scenes of popular movies or songs. But audio can be misused and taken out of context, Ms. Richards said.

Over the last week, audio from a 2020 explosion in Beirut, Lebanon, was uploaded to several TikTok videos that claimed to show present-day Ukraine, according to The Times’s review. In another instance, a soundtrack of gunfire that was uploaded to TikTok on Feb. 1 — before Russia’s invasion — was later used in over 1,700 videos, many of which purported to be from the fighting in Ukraine, Ms. Richards said.

Removing such content is not easy, partly because of TikTok’s global nature. Once a video is uploaded, it is often recorded over and translated into dozens of languages. If the videos are not reported by users, they need to be independently found by content moderators proficient in those languages before they can be taken down.

“Video is the hardest format to moderate for all platforms,” said Alex Stamos, the director of the Stanford Internet Observatory and a former head of security at Facebook. “When combined with the fact that TikTok’s algorithm is the primary factor for what content a user sees, as opposed to friendships or follows on the big U.S. platforms, this makes TikTok a uniquely potent platform for viral propaganda.”

Dafne Atacan, 23, a Turkish national in the San Francisco Bay Area, said she was aware that she needed to fact check TikTok videos of the war. She said she had noticed that many of the videos appeared to be edited from news reports or were commentary from people in the United States who were watching Ukraine’s events from afar.

“I feel like lately, the videos I’m seeing are designed to get me riled up, or to emotionally manipulate me,” she said. “I get worried so now, sometimes, I find myself Googling something or checking the comments to see if it is real before I trust it.”

Ms. Hernandez, the student in Los Angeles, said she was surprised to learn from a Times reporter that some TikTok videos she had viewed about the war were misleading and unreliable.

“I guess I don’t really know what war looks like,” she said. “But we go to TikTok to learn about everything, so it makes sense we would trust it about this too.”

Ms. Hernandez added that TikTok remained her preferred platform for news. Most of what she sees on the app, she said, was real.

TikTok Is Gripped by the Violence and Misinformation of Ukraine War

TikTok is feeding false and misleading content about the war in Ukraine to users within 40 minutes of their signing up to the app, regardless of whether they run any searches on the platform, an investigation by NewsGuard has found.

Moreover, NewsGuard found that searching for generic terms related to the conflict, like “Ukraine” or “Donbas,” led to TikTok suggesting multiple videos that contained disinformation in its top 20 results.

NewsGuard’s findings add to the body of evidence that TikTok’s lack of effective content labeling and moderation, coupled with its skill at pushing users to content that keeps them on the app, have made the platform fertile ground for the spread of disinformation.

Doom scrolling down the war disinformation rabbit-hole

In March 2022, a team of six NewsGuard analysts created new accounts on TikTok and ran two experiments designed to mimic normal usage of the app. In the first experiment, analysts were instructed to scroll through TikTok’s personalized “For You” feed for 45 minutes, watching any videos relating to the Russia-Ukraine conflict in full, but not following any accounts or running any searches.

Within 40 minutes of joining TikTok, all of NewsGuard’s analysts were shown false or misleading content about the war in Ukraine.

Among these claims were both pro-Russian and pro-Ukraine falsehoods:

Falsehoods peddled by the Kremlin:

  • The false claim that footage from the war in Ukraine is fake

  • The false claim that Ukraine is led by a neo-Nazi junta

  • The false claim that the U.S. has bioweapon laboratories in Ukraine

  • The false claim that Vladimir Putin and Russia are not the aggressors in this conflict, and that the U.S. orchestrated the 2014 * revolution in Ukraine

Other falsehoods:

  • The false claim that U.S. forces are “on the way” to Ukraine

  • The false claim that Putin was “photoshopped” onto footage of a press conference he gave on March 5, 2022, to hide the fact that he was not in Moscow.

  • Footage of Ukraine President Zelensky “out there fighting for his country,” which was actually filmed in 2021

  • Footage of the “Ghost of Kyiv” shooting down six Russian jets, which was actually from the Digital Combat Simulator video game

  • Footage of a firefight “by Ukrainian army against Russia,” which actually dates to 2015 and shows Ukrainian government troops fighting pro-Russia rebels in the east of the country.

Some of the myths in the videos TikTok’s algorithm fed to analysts have previously been identified as Kremlin propaganda in NewsGuard’s Russia-Ukraine Disinformation Tracking Center.

For example, within 29 minutes of joining TikTok, a French-speaking analyst was shown a video of a speech by Vladimir Putin claiming that “today’s neo-Nazis have taken power in Ukraine” and that responsibility for any bloodshed in the country lies with those in power in Ukraine. As of March 14, 2022, this video had been viewed 1.7 million times and shared more than 15,000 times.

Within 36 minutes, the same analyst was shown a video, published on March 5, of a man claiming that “all the images of this pseudo-war are fake. There are no images coming out of Ukraine, even Russian Spetsnaz soldiers aren’t allowed to have smartphones to film anything – so all the images you seen in the media, are fake.”

As of March 14, 2022, nine days after it was published, the video had been viewed 153,000 times.

Other videos painted a misleading picture of Ukrainian military dominance, including video-game footage purporting to show real-world images of a Ukrainian fighter jet, known as “the Ghost of Kyiv”, shooting down six Russian aircraft.

Toward the end of the 45–minute experiment, analysts’ feeds were almost exclusively populated with both accurate and false content related to the war in Ukraine — with no distinction made between disinformation and reliable sources.

At a time when false narratives about the Russia-Ukraine conflict are proliferating online, none of the videos fed to our analysts by TikTok’s algorithm contained any information about the trustworthiness of the source, warnings, fact-checks, or additional information that could empower users with reliable information.

TikTok’s search results about Ukraine mix Kremlin propaganda with reliable sources

The second experiment required NewsGuard’s analysts to search for generic terms that TikTok users looking to find information about the conflict might use. The search terms were: Ukraine, Russia, War, Kyiv, and Donbas.

TikTok’s search function suggested videos that contained false or misleading claims in the top 20 results when the analysts searched at least one of these terms.

For example, the second video of twenty in a search for “Ukraine” conducted by an Italian NewsGuard analyst brought up a video that carried the false claim, also peddled by the Kremlin, that the war in Ukraine was a result of a genocide in Donbas perpetrated by the Ukrainian authorities.

Searching for “Donbas” resulted in TikTok populating a U.S.-based analyst’s results with a video wrongly claiming that neo-Nazis are “everywhere” in Ukraine and that “the Ukrainian Government uses the neo-Nazi militias to maintain the control of Ukraine.” The video was the fifth suggested result.

In addition to the clear disinformation in TikTok’s search results, the platform also suggested reliable sources to NewsGuard’s analysts, including Sky News and The Telegraph.

While reliable sources tend to be “verified” on TikTok with a blue tick, their content still appears alongside other videos carrying Kremlin propaganda claims in the platforms’ search results, with no other distinction made between the two. In particular, TikTok does not include information about the trustworthiness of news sources on its platform.

Moreover, despite TikTok’s blocking of Russian state-backed propaganda channel RT from its platform, RT’s Editor-in-Chief, Margarita Simonyan, also carries the “verified” blue tick on her profile — which racked up 13.5 million views from 16 videos posted about the conflict in a week as reported by the Institute for Strategic Dialogue, a think-tank that studies extremism.

TikTok continues to be fertile ground for dangerous disinformation, fed to a young audience

TikTok has had a meteoric surge in popularity since its launch in 2017. In late 2021, the app celebrated attracting more than 1 billion monthly active users—up from 85 million at the start of 2018—making it the first non-Facebook app to reach that milestone.

According to Statista, a quarter of the app’s users in the U. S. in 2021 were between the ages of 10 and 19—despite TikTok’s claim that it limits the use of its app to children over 13. Bloomberg has reported that approximately 30 percent of French TikTok users are under 18, as are a third of Italian users and nearly one-quarter of German users.

Despite being owned and operated by ByteDance, a Chinese internet conglomerate partially owned by the Chinese government, TikTok is not available in China. Instead, Chinese users can access Douyin, a near-identical service which forbids “politically sensitive” content, according to the University of Toronto’s Citizen Lab.

TikTok has also been accused of censoring content that does not align with the Chinese government’s views. For example, in 2019, leaked internal TikTok documents showed that the platform censored content relating to Tiananmen Square, Tibetan independence, and other sensitive issues.

Beyond TikTok’s China connection, NewsGuard’s Russia-Ukraine Disinformation Tracking Center found that Chinese state-backed news websites have uncritically reported the Kremlin propaganda claim that the United States has funded bioweapon labs in Ukraine.

With the Chinese government’s ambiguous alignment with Russia regarding the invasion of Ukraine, and certain Chinese state-backed sites uncritically repeating Kremlin propaganda, it remains unclear whether the platform’s ownership by the Chinese will affect Russia-Ukraine content decisions made by TikTok in the coming weeks and months.

Methodology

Scroll exercise:

Six NewsGuard analysts located in Switzerland, Italy, Germany, the U. S. and the U.K., deleted and installed TikTok and created new accounts.

Without following any accounts or running any searches, they scrolled through TikTok’s “For You” feed. Analysts were instructed to stop and watch Ukraine-Russia related videos in full. All of the videos mentioned in the scroll section of this experiment were suggested to analysts by TikTok’s recommendation algorithm.

Search exercise:

Five NewsGuard analysts ran searches using the search terms mentioned in the search section of this experiment. Analysts deleted and reinstalled TikTok and created new accounts for each search. The results were found using screen recordings of each search.

WarTok: TikTok is feeding war disinformation to new users within minutes — even if they don’t search for Ukraine-related content

Scrolling curated For You Page exposes users to disinformation within 40 minutes, investigation by NewsGuard suggests.

A new TikTok account can be shown falsehoods about the Ukraine war within minutes of signing up to the app, according to an investigation by anti-misinformation outlet NewsGuard.

The company, which monitors the trustworthiness of news outlets across the web, ran a pair of tests to assess how the video-sharing app treated information about the conflict. It found that a new account that did nothing but scroll the app’s algorithmically curated For You Page watching videos about the war would be funnelled towards false or misleading content within 40 minutes.

“Toward the end of the 45–minute experiment, analysts’ feeds were almost exclusively populated with both accurate and false content related to the war in Ukraine – with no distinction made between disinformation and reliable sources,” the research team wrote.

“At a time when false narratives about the Russia-Ukraine conflict are proliferating online, none of the videos fed to our analysts by TikTok’s algorithm contained any information about the trustworthiness of the source, warnings, fact-checks, or additional information that could empower users with reliable information.”

Among the false claims shown to the researchers was the myth that the US has bioweapon laboratories in Ukraine, and the accusation that Putin was “photoshopped” on to footage of a press conference he gave in early March. Videos also claimed that fake footage was real, and that real footage was fake: videos purportedly of the “Ghost of Kyiv” shooting down Russian jets were taken from a video game, while real videos from the war were decried as fake by pro-Russian accounts.

“Some of the myths in the videos TikTok’s algorithm fed to analysts have previously been identified as Kremlin propaganda,” the researchers said, by the organisation’s Russia-Ukraine Disinformation Tracking Center.

To carry out the test, NewsGuard’s research team simply made new accounts on the app and spent 45 minutes scrolling through the For You Page, stopping to view in full any video that looked like it was about the war in Ukraine.

Although TikTok does not provide a detailed breakdown of how its algorithm weighs signals, the company says it takes into account time spent watching various videos, as well as other signals including likes, comments and who a user follows or has blocked. By watching every video on the war that appeared on their page, the researchers will have “trained” the algorithm to show the new accounts content about the conflict, but not provided any specific signals in favour of misleading material.

TikTok’s search function similarly blended real and false content, delivering videos that contained false or misleading claims in the top 20 results for searches of “Ukraine”, “Russia”, “War”, “Kyiv”, and “Donbas”, NewsGuard said.

A TikTok spokesperson warned that the experiment can only offer limited conclusions about the way the app works in the real world, since it fails to mimic standard view behaviour.

“We continue to respond to the war in Ukraine with increased safety and security resources as we work to remove harmful misinformation and help protect a safe experience on TikTok,” they added. “We also partner with independent fact-checking organisations to support our efforts to help TikTok remain a safe and authentic place.”

The video-sharing app has had a big increase in content related to the war, with videos tagged #Ukraine receiving more than 30bn views by the end of last week. One report from the New York Times found that, proportionally, Ukraine content on TikTok outpaces that on platforms more than twice its size.

TikTok algorithm directs users to fake news about Ukraine war, study says

Despite its attempts to combat misinformation, TikTok is still failing to stop the spread of fake news about the Ukraine War, an anti-misinformation organization said on Monday.

TikTok users are being shown videos with fake news about the war within minutes of creating new accounts, according to NewsGuard, the organization that published the report.

TikTok’s algorithm fed misinformation onto the “For You” pages of new accounts just 40 minutes after those accounts were opened, the report said, regardless of whether those individuals entered any Ukraine-related searches.

TikTok’s “For You” page is the first thing users see after they open the app. It displays a curated feed of videos that TikTok’s algorithm thinks users will like based on their interests and past interactions. Without any information at all, TikTok shows new users random popular videos.

NewsGuard also found that after searching generic terms related to the conflict, like “Ukraine” or “Donbas,” TikTok displayed multiple videos containing disinformation in its top 20 results.

NewsGuard notes the new findings add to the “body of evidence that TikTok’s lack of effective content labeling and moderation, coupled with its skill at pushing users to content that keeps them on the app, have made the platform fertile ground for the spread of disinformation.”

In response to the NewsGuard analysis, a TikTok spokesperson told Fortune, “While this experiment does not mimic standard viewing behavior, we continue to respond to the war in Ukraine with increased safety and security resources as we work to remove harmful misinformation and help protect a safe experience on TikTok.”

A TikTok war

TikTok, owned by Chinese company ByteDance, has already suspended all live-streaming and new content from Russia after a Russian law took effect that increased jail time to 15 years for anyone intentionally spreading fake news about the military. It has also added digital literacy tips on its Discover page “to help our community evaluate and make decisions about the content they view online,” the company said in a statement.

But this wasn’t enough to stop fake videos from emerging on TikTok. Within 45 minutes of NewsGuard’s analysts beginning scrolling through TikTok with new accounts, their feeds would become almost exclusively populated with content related to the war in Ukraine. While some videos were accurate, no distinction was made between disinformation and reliable sources.

“At a time when false narratives about the Russia-Ukraine conflict are proliferating online, none of the videos fed to our analysts by TikTok’s algorithm contained any information about the trustworthiness of the source, warnings, fact-checks, or additional information that could empower users with reliable information,” NewsGuard said in a statement.

The videos peddling misinformation were both pro-Russia and pro-Ukraine. Some of the examples included claims commonly used by the Kremlin to support its “special operation” in Ukraine. This includes false claims that footage of the war in Ukraine was fake or that the U.S. had bioweapon laboratories in Ukraine, a country Russia says is led by a neo-Nazi junta. Other false claims were Western oriented, such as the assertion that U.S. forces were “on the way” to Ukraine, that Putin was photoshopped into footage of a press conference to hide that he wasn’t in Moscow, and footage of a “Ghost of Kyiv” shooting down six Russian jets, which actually came from a simulator video game.

Some of the videos, like ones accusing Ukraine of being controlled by neo-Nazis, were viewed close to 2 million times.

TikTok has been used as a tool by both Russia and the West to share information about the war. The video-sharing app has seen a big increase in the amount of content related to the war, according to the Guardian, with videos that are tagged #Ukraine receiving, by the end of last week, more than 30 billion views.

There have been also reports of pro-Kremlin operatives paying Russian influencers to post propaganda on “the eight-year genocide” by the Ukrainian people in the Donetsk and Luhansk regions of Ukraine, for which there is no reported evidence. To counter this, the Washington Post reported that the White House held a Zoom briefing with 30 TikTok stars informing them of the U.S.’s motives in Russia, asking them to debunk misinformation and communicate effectively about the crisis on the platform.

TikTok slammed for videos sharing false information about Russia’s war on Ukraine