Incident 243: Bots Allegedly Made up Roughly Half of Twitter Accounts in Discussions Surrounding COVID-19 Related Issues

Description: Bots by anonymous actors were found by researchers to make up roughly half of Twitter accounts participating in COVID-19 discussions, many of which posted tweets about “reopening America“.
Alleged: Unknown developed and deployed an AI system, which harmed Twitter , Twitter Users and Twitter users participating in COVID-19 discussions.

Suggested citation format

Dickinson, Ingrid. (2020-01-01) Incident Number 243. in McGregor, S. (ed.) Artificial Intelligence Incident Database. Responsible AI Collaborative.

Incident Stats

Incident ID
243
Report Count
2
Incident Date
2020-01-01
Editors
Sean McGregor, Khoa Lam

Tools

New ReportNew ReportDiscoverDiscover

Incidents Reports

This might not come as a surprise if you ever use Twitter, but it turns out bots are driving the push to “reopen America” on the platform.

According to Business Insider, researchers from Carnegie Mellon University found that about half of more than 200 million tweets related to covid-19 that have been sent since January appear to be from bots. About half of all accounts tweeting about “reopening America” appeared to be bots, and 60 percent of the 1,000 most influential retweeters appeared to be bots. Of the top 50 influential accounts, 82 percent are bots. Basically, a whole lot of fake accounts are artificially inflating the “movement” to reopen America. The researchers were able to categorize more than 100 types of inaccurate covid-19 stories, but tweets about ending stay-at-home orders were the most frequent.

They identified the bogus Twitter accounts by using artificial intelligence to track where the tweets were coming from, the number of followers, and the frequency of tweets coming from a single account. Kathleen Carley, a computer-science professor who led the research, said the bots tweet more frequently than is humanly possible or tweet from different locations on opposite sides of the world just hours apart. (That’s a little suspect when few people are jetting around the globe right now.)

The researchers also looked at whether similar hashtags and phrases were used from one account to the next, which is an indication that the messaging was copied and pasted from one tweet to another. Combine that with tweets that spit out faster than any human could possibly make on their own, and you’ve got something that looks, behaves, and probably is a Twitter bot.

Additionally, researchers discovered that 66 percent of tweets came from real people using bot accounts to spread their message as far as possible, and the other 34 percent came from actual bots. Unfortunately, researchers were unable to identify where the bot activity originally started from. They aren’t sure if it’s mostly coming from within the U.S, outside the U.S., or maybe a mix of both.

Bot activity is nothing new on social media platforms, especially around elections or natural disasters, the Carnegie Mellon researchers said. Usually bot activity is around 10 to 20 percent during those times, but since the global pandemic started, bot activity has surged to twice as much, possibly due to the fact that more people have the time to set up more elaborate bot networks. The number of groups that hire firms to run bot accounts has increased, too.

“Because [the pandemic is] global, it’s being used by various countries and interest groups as an opportunity to meet political agendas,” Carley said in the Carnegie Mellon release.

This isn’t great news for Twitter, which recently decided to step up its anti-misinformation campaign by cracking down on users who tweet unfounded conspiracy theories like “5G causes coronavirus.” The company is also testing new reply controls that could make it harder to stop or slow down the spread of misinformation. The study reveals that Twitter needs to do a better job of weeding out the bots that clog up the platform and make it difficult to tell who’s real and who’s fake.

Twitter Bots Drove the Push to 'Reopen America,' Study Finds

As parts of the US have lifted shutdown orders during the COVID-19 pandemic, there's been a fierce argument online about the risks and benefits of reopening. New research suggests that bots have been dominating that debate.

Carnegie Mellon University researchers analyzed over 200 million tweets discussing COVID-19 and related issues since January and found that roughly half the accounts — including 62% of the 1,000 most influential retweeters — appeared to be bots, they said in a report published this week.

That's a far higher level of bot activity than usual, even when it comes to contentious events — the level of bot involvement in discussions about things like US elections or natural disasters is typically 10% to 20%.

The researchers identified bots using artificial-intelligence systems that analyze accounts' frequency of tweets, number of followers, and apparent location.

"Tweeting more frequently than is humanly possible or appearing to be in one country and then another a few hours later is indicative of a bot," Kathleen Carley, a computer-science professor who led the research, said in a release.

"When we see a whole bunch of tweets at the same time or back to back, it's like they're timed," Carley added. "We also look for use of the same exact hashtag, or messaging that appears to be copied and pasted from one bot to the next."

The researchers said they found that among tweets about "reopening America," 66% came from accounts that were possibly humans using bot assistants to spread their tweets more widely, while 34% came from bots.

There are a few possible explanations for the surge in bot activity. People may have more time to set up elaborate bot networks during stay-at-home orders, and the availability of botnets for hire has exploded recently. Carley also said that the global nature of the pandemic meant that countries and interest groups were using it to advance political agendas.

However, the researchers did not determine the origins of the bot activity or narrow down whether it was coming from foreign nation-state actors or from within the US.

"Even if someone appears to be from your community, if you don't know them personally, take a closer look, and always go to authoritative or trusted sources for information," Carley said. "Just be very vigilant."

Roughly half the Twitter accounts pushing to 'reopen America' are bots, researchers found