Incident 17: Inappropriate Gmail Smart Reply Suggestions

Description: Google's Gmail Smart Reply tool was over-recommending the response "I love you" in situations where it was deemed innappropriate.


New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History
Alleged: Google developed and deployed an AI system, which harmed Gmail Users.

Incident Stats

Incident ID
Report Count
Incident Date
Sean McGregor

CSETv0 Taxonomy Classifications

Taxonomy Details

Full Description

Google's Gmail Smart Reply tool suggests replies, each of about three to six words long, to emails received on its platform. According to Google's management director Alex Gawley, the machine learning-based system is able to produce about 20,000 discrete responses to emails by combining suggestions from two machine learning programs. The first reads the email and comprehends the content and the second suggests responses to that content all taking place on the long short-term memory neural network. Gmail users in 2015 provided feedback that the reply "I love you" was occurring too often, leading Google to change their algorithm.

Short Description

Google's Gmail Smart Reply tool was over-recommending the response "I love you" in situations where it was deemed innappropriate.



AI System Description

Google's Gmail Smart Reply tool, a machine-learning system is able to produce approximately 20,000 discrete responses by combining suggestions from two machine- learning programs: the first reads and comprehends the content and the second recommends responses.

System Developer


Sector of Deployment

Information and communication

Relevant AI functions

Perception, Cognition

AI Techniques

machine-learning, natural language processing, long short-term memory neural network

AI Applications

content creation, smart reply, suggested reply, recommendation engine



Named Entities

Google, Gmail, Gmail Smart Reply

Technology Purveyor


Beginning Date


Ending Date


Near Miss




Lives Lost


Data Inputs

email text

CSETv1 Taxonomy Classifications

Taxonomy Details

Harm Distribution Basis


Sector of Deployment

information and communication

Soon, Gmail's AI Could Reply to Your Email for You · 2015

Ever wished your phone could automatically reply to your email messages?

Well, Google just unveiled technology that's at least moving in that direction. Using what's called "deep learning"—a form of artificial intelligence that's rapidly re…

Google's Smart Reply Lets a Computer Answer Your Emails for You · 2015

Just a few months ago, Google really started showing off its newest secret weapon: a robot brain that learns. Using machine learning algorithms that actually get smarter as they practice, Google's been able to identify your loved ones in yo…

Computer, respond to this email: Introducing Smart Reply in Inbox by Gmail · 2015

There's actually a lot going on behind the scenes to make Smart Reply work. Inbox uses machine learning to recognize emails that need responses and to generate the natural language responses on the fly. If you're interested in how Smart Rep… · 2015

Not every email deserves a handcrafted response. Sometimes, all it takes is a sentence to answer the one burning question that someone has dropped in your inbox, which is why Google is using the power of machine learning to make email triag…

Google Inbox Suggests Replies for Incoming Email · 2015

Replying to routine email requests just got easier with a new smart reply feature for Gmail users who get their email through Google Inbox. Starting this week, the app's Smart Reply feature will analyze incoming email messages and offer a s…

New Gmail feature scans your emails and suggests replies · 2015

Google will soon scan the content of your emails and serve up what it thinks is the perfect reply.

The new feature, called Smart Reply, identifies Gmail emails that require a response and presents three options for replies. Smart Reply will…

Google's new email feature will write replies for you · 2015

When Google launched Inbox, its most recent email app from the Gmail team, it touted the app's ability to act almost like an assistant. Now, a year later, Google is making the app more like an aide than ever.

The company will soon be rollin…

Google’s New Autoreply Sounds Great!!!! · 2015

On April 1, 2009, Google unveiled Gmail Autopilot, a plug-in that promised to read and generate contextually relevant replies to the messages piling up in users’ inboxes. “As more and more everyday communication takes place over email, lots…

Google's Email Replying AI Keeps Saying It Loves You · 2015

A system Google designed for replying to emails proved a little friendlier than anyone expected.

There are quite a few humans out there who are absolutely convinced robots will kill us the moment they become self-aware. But what if the exac…

Google's Smart Reply: Convenience Over Connection · 2017

If you use Google products for email, you may have noticed those suggested replies at the bottom of your mobile inbox. These give users options to choose short and snappy responses to emails, such as, “Will do,” “I am working on that now,” …

Ranking Gmail’s AI Fueled Smart Replies · 2018

Image: Google: Photo: Konstantin Sergeyev

The most recognizable feature of Gmail’s newly rolled-out redesign is the so-called smart reply, wherein bots offer three one-click responses to each mail message. Say your email contains the words … · 2018

“Thanks for letting me know.”

Have you noticed an uptick in this phrase appearing in emails, on social media, and IRL? Do you find yourself saying it even though maybe a year ago you would never use the phrase? Is there a new wave of semi-f…

Is This Article Worth Reading? Gmail’s Suggested Reply: ‘Haha, Thanks!’ · 2018

When Jess Klein emailed her mother to make plans, the response startled her.

“Cool, see you there.”

Ms. Klein, a 38-year-old from Brooklyn, was suspicious. “Why is my mom talking like a Valley Girl?” she wondered.


Gmail users will be able to opt out of Smart Replies on desktop · 2018

Around 10 percent of Gmail responses are now generated with the Smart Reply feature, according to the Wall Street Journal, and it has been available on Gmail apps since last year. Ajit Varma, Google's director of product management, told th…

Smart Reply Not Smart Enough? Desktop Gmail Users Can Soon Opt Out · 2018

After users complained that Gmail’s Smart Reply wasn’t so smart, Google announced that it will soon give desktop users the ability to disable the A.I.-based feature. Although the time-saving feature relies on artificial intelligence to come…

Gmail Smart Reply goofed w/ 'Sent from my iPhone' · 2018

The Wall Street Journal is out today with a quick story on the rollout and reception of Gmail’s new Smart Reply feature, and it comes with a few interesting bits of information and anecdotes from random users and Google alike — one of which… · 2018

Romantic AI: Google Had to Change Gmail's New Smart Reply Responses Because it Kept Suggesting "I Love You"

You can turn off Gmail's creepy Smart Replies on desktop too - just not yet · 2018

How do you feel about Gmail’s new Smart Reply tool? We’re hearing mixed reports. On one hand it’s super handy to be able to tap/click that “Thank you so much!” button when you’re in a rush.

However, users are reporting the suggestions aren’…

Google tweaked its Smart Reply feature because it would repeatedly suggest users send 'I love you' · 2018

Google recently made its handy 'Smart Reply' feature available to Gmail's 1 billion-plus users.

But early prototypes of the tool didn't work so seamlessly, according to the Wall Street Journal.

In one case, the algorithms powering Gmail's '…

Google changed Gmail's Smart Reply after AI kept suggesting I love you · 2018

A few days ago, I received a short, effective email in my inbox: "Sounds good!"

I had to pause. Although that was the response I wanted — I was arranging a meeting — I wondered: Did he really send that, or did he simply hit Google's automat…

Gmail smart compose: Will it change how we write emails? · 2018

From: Corinne Purtill (Quartz)

To: Corinne Purtill (Gmail)

Mon, Oct 1, 2018 at 11:08 AM

At its annual developers conference earlier this year Google introduced “smart compose,” a new Gmail feature that helps users complete their sentences. …

Google ends support for Reply, which added smart replies to third-party apps · 2018

Google will no longer continue support for Reply, an experimental app that offers smart reply responses to various messaging apps such as Slack, Hangouts, and Messenger. The app launched earlier this year as part of Google’s Area 120 divisi…


A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents