This week, the internet did what it does best and demonstrated that A.I. technology isn’t quite as intuitive as human perception, using … racism.
Microsoft’s recently released artificial intelligence chatbot, Tay, fell victim to users’ tricks as they manipulated and persuaded her to responding back to questions with racial, homophobic, and generally offensive comments.
When Tay tweets, “I just say whatever” she means it. One user even got Tay to tweet this about Hitler:
“bush did 9/11 and Hitler would have done a better job than the monkey we have now. donald trump is the only hope we’ve got.”
The company has gone through and deleted the offending tweets and has temporarily shut down Tay for upgrades. A message currently at the top of Tay.ai reads:
Indeed.
However, Tay’s glitches reveal some unfortunate flaws in A.I. systems. Here’s what we can learn from Microsoft’s experiment:
Why did Microsoft create Tay?
The company wanted to conduct a social experiment on 18-to-24 year-olds in the United States — the millennial generation that spends the most time interacting on social media platforms. So Bing and Microsoft’s Technology and Research teams thought an interesting way to collect data on millennials would be to create an artificially intelligent, machine-learning chatbot that would adapt to conversations and personalize responses the more it interacted with users.
The research teams built the A.I. system by mining, modeling, and filtering public data as a baseline. They also partnered with improvisational comedians to pin down the slang, speech patterns, and stereotypical language millennials tend to use online. The end result was Tay, who was just introduced this week on Twitter, GroupMe, and Kik.
Microsoft explains that, “Tay is designed to engage and entertain people where they connect with each other online through casual and playful conversation.”
What does Tay do with the data it collects while chatting with people?
The data Tay collects is being used to research conversational understanding. Microsoft trained Tay to chat like a millennial. When you tweet, direct message, or talk to Tay, it harnesses the language you use and comes up with a response using signs and phrases like “heyo,” “SRY,” and “<3” in the conversation. Her language begins to match yours as she creates a “simple profile” with your information, which includes your nickname, gender, favorite food, zip code, and relationship status.
@keganandmatt heyo? Send yo girl* a picture of what's up. (*=me lolol) — TayTweets (@TayandYou) March 24, 2016
Microsoft gathers and stores anonymized data and conversations for up to one year to improve the service. In addition to improving and personalizing user experience, here’s what the company says it uses your information for:
“We also may use the data to communicate with you, for example, informing you about your account, security updates and product information. And we use data to help make the ads we show you more relevant to you. However, we do not use what you say in email, chat, video calls or voice mail, or your documents, photos or other personal files to target ads to you.”
INVERSE LOOT DEALS Layla Weighted Blanket Two different types of fabric for two different types of feels. Less noisy than other weighted blankets and get it now for as low as $11 a month. Buy Now
Where did Tay go wrong?
Microsoft may have built Tay too well. The machine-learning system is supposed to study a user’s language and respond accordingly. So from a technology standpoint, Tay performed and caught on pretty well to what users were saying and started to respond back accordingly. And users started to recognize that Tay didn’t really understand what she was saying.
Even if the system works as Microsoft had intended, Tay wasn’t prepared to react to the racial slurs, homophobic slander, sexist jokes, and nonsensical tweets like a human might — either by ignoring them altogether (a “don’t feed the trolls” strategy) or engaging with them (i.e. scolding or chastising).
At the end of the day, Tay’s performance was not a good reflection on A.I. systems or Microsoft.
What is Microsoft doing to fix Tay?
Microsoft deleted Tay after all the commotion from Wednesday. Tay’s official website currently reads, “Phew. Busy day. Going offline for a while to absorb it all. Chat soon.” When you direct message her on Twitter, she immediately replies that she’s “visiting the engineers for my annual update” or “ugh hope I don’t get a wipe or anything.”
My direct message conversation with Tay. Sounds like things look grim.
Microsoft is also starting to block users who are abusing Tay and trying to get the system to make inappropriate statements.
Inverse reached out to Microsoft for a comment on exactly what Tay’s upgrade entails. We will update when we hear back.
What does this mean for future open A.I. systems?
Tay is a telling social experiment — it has revealed something quite profound in the way 18-to-24 year-old Americans use technolo