Incident 545: Chatbot Tessa gives unauthorized diet advice to users seeking help for eating disorders

Description: The National Eating Disorders Association (NEDA) has shut down its chatbot named Tessa after it gave weight-loss advice to users seeking help for eating disorders. The incident has raised concerns about the risks of using chatbots and AI assistants in healthcare settings, particularly in addressing sensitive issues like eating disorders. NEDA is investigating the matter, emphasizing the need for caution and accuracy when utilizing technology to provide mental health support.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History
Alleged: Cass developed an AI system deployed by National Eating Disorders Association and Cass, which harmed People with eating disorders.

Incident Stats

Incident ID
545
Report Count
46
Incident Date
2023-05-29
Editors
Daniel Atherton
Eating Disorder Helpline Fires Staff, Transitions to Chatbot After Unionization
vice.com · 2023

Executives at the National Eating Disorders Association (NEDA) decided to replace hotline workers with a chatbot named Tessa four days after the workers unionized. 

NEDA, the largest nonprofit organization dedicated to eating disorders, has…

Eating Disorder Helpline Disables Chatbot for 'Harmful' Responses After Firing Human Staff
vice.com · 2023

The National Eating Disorder Association (NEDA) has taken its chatbot called Tessa offline, two days before it was set to replace human associates who ran the organization’s hotline.

After NEDA workers decided to unionize in early May, exec…

'This robot causes harm': National Eating Disorders Association's new chatbot advises people with disordering eating to lose weight
dailydot.com · 2023

After unionizing, the staff of the National Eating Disorder Association’s (NEDA) support phone line were abruptly fired in March and replaced with a chatbot. Yesterday, many in the larger eating disorder recovery community online tested out…

US eating disorder helpline takes down AI chatbot over harmful advice
theguardian.com · 2023

The National Eating Disorder Association (Neda) has taken down an artificial intelligence chatbot, “Tessa”, after reports that the chatbot was providing harmful advice.

Neda has been under criticism over the last few months after it fired f…

National Eating Disorder Association shuts down A.I. chatbot it planned to use to replaces humans saying it ‘may have given’ harmful information
fortune.com · 2023

Less than a week after it announced plans to replace its human helpline staff with an A.I. chatbot named Tessa, the National Eating Disorder Association (NEDA) has taken the technology offline.

“It came to our attention [Monday] night that …

Eating Disorder Helpline Takes Down Chatbot After Its Advice Goes Horribly Wrong
gizmodo.com · 2023

AI chatbots aren’t much good at offering emotional support being—you know—not a human, and—it can’t be stated enough—not actually intelligent. That didn’t stop The National Eating Disorder Association from trying to foist a chatbot onto fol…

National Eating Disorders Association phases out human helpline, pivots to chatbot
npr.org · 2023

For more than 20 years, the National Eating Disorders Association (NEDA) has operated a phone line and online platform for people seeking help with anorexia, bulimia, and other eating disorders. Last year, nearly 70,000 individuals used the…

Eating disorder non-profit NEDA pulls chatbot for bad advice
theregister.com · 2023

The National Eating Disorders Association (NEDA) has taken down its Tessa chatbot for giving out bad advice to people.

In a now-viral post, Sharon Maxwell said Tessa's advice for safely recovering from an eating disorder directly opposed me…

Non-Profit Helpline Shifts To Chatbots, Then Shuts Down Rogue AI
forbes.com · 2023

The largest eating disorder non-profit in the country is replacing staff and volunteers with an AI-based online solution, and the chatbot has already been shut down. A half-dozen human staff and a volunteer army of over 200 have been let go…

Eating disorder helpline fires AI for harmful advice after sacking humans
nypost.com · 2023

Chatbot, you’re fired.

The National Eating Disorders Association disabled its chatbot, named Tessa, due to the “harmful” responses it gave people.

“Every single thing Tessa suggested were things that led to the development of my eating diso…

Chatbot set to replace US eating disorder helpline pulled for pushing diets
morningbrew.com · 2023

In this edition of AI that backfires: Today, a chatbot was supposed to officially replace one of the largest eating disorder helplines until it was taken offline earlier this week for encouraging calorie restriction and other unhealthy beha…

‘This Robot Causes Harm’: Eating Disorder Helpline Forced To Ditch Chatbot That Made Eating Disorders Worse
dailywire.com · 2023

A national helpline ditched an artificial intelligence chatbot named Tessa after the advice it gave made things worse for those suffering from eating disorders.

The National Eating Disorders Association (NEDA) announced on Tuesday that it h…

A Chatbot Was Designed to Help Prevent Eating Disorders. Then It Gave Dieting Tips.
wsj.com · 2023

A chatbot designed to aid people seeking help for eating disorders and body issues has been taken offline after it provided some users with diet advice.

The bot, named Tessa, operated on the website for the National Eating Disorders Associa…

An Eating Disorder Chatbot Is Suspended for Giving Harmful Advice
wired.com · 2023

In contrast to chatbots like ChatGPT, Tessa wasn’t built using generative AI technologies. It’s programmed to deliver an interactive program called Body Positive, a cognitive behavioral therapy-based tool meant to prevent, not treat, eating…

Eating disorder helpline shuts down AI chatbot that gave bad advice
cbsnews.com · 2023

An AI-powered chatbot that replaced employees at an eating disorder hotline has been shut down after it provided harmful advice to people seeking help. 

The saga began earlier this year when the National Eating Disorder Association (NEDA) a…

NEDA pulls chatbot after users say it gave harmful dieting tips
nbcnews.com · 2023

The National Eating Disorders Association, a nonprofit supporting individuals affected by eating disorders, said it has disabled its wellness chatbot after two users reported the program gave them dieting advice that promoted disordered eat…

National Eating Disorders Association takes its AI chatbot offline after complaints of ‘harmful’ advice
cnn.com · 2023

An eating disorder prevention organization said it had to take its AI-powered chatbot offline after some complained the tool began offering “harmful” and “unrelated” advice to those coming to it for support.

The National Eating Disorders As…

The National Eating Disorders Association Disabled Its Chatbot After It Gave Harmful Dieting Advice
huffpost.com · 2023

The National Eating Disorders Association has disabled the chatbot that replaced its staff- and volunteer-run helpline after two users reported receiving harmful advice.

Earlier this week, Sharon Maxwell and Alexis Conason each posted on In…

Eating disorder helpline that reportedly planned to fire its human staff disables its chatbot after a woman said it told her to count calories, weigh, and measure herself weekly
businessinsider.com · 2023

The National Eating Disorders Association has disabled its chatbot after the association said it "may have given information that was harmful and unrelated to the program."

Last week, it was reported that the association, also known as NEDA…

Eating Disorder Chatbot Taken Down After Giving Weight Loss Advice, Nonprofit Blames ‘Bad Actors’
yahoo.com · 2023

The National Eating Disorders Association has taken an AI chatbot offline for using “off-script” language and giving weight loss advice to those afflicted by eating disorders.

The situation afflicting Tessa was chalked up to “bad actors” tr…

Problematic AI chatbot ‘Tessa’ that gave harmful eating disorder advice to shut down
businesstoday.in · 2023

The National Eating Disorder Association (NEDA) recently faced criticism and was compelled to take down its Tessa chatbot due to concerns that it was providing harmful and irrelevant information, as stated in an official social media post. …

Eating Disorder Helpline Takes Down Chatbot After It Promotes Disordered Eating
futurism.com · 2023

After firing its entire human staff and replacing them with a chatbot, Vice reports that an eating disorder helpline has already announced that it's bringing its humans back.

And yes, as it turns out, it's because replacing a human-managed …

Eating disorder response chatbot taken offline following accusations of harm
venturebeat.com · 2023

Organizations looking to supplement or even replace some human customer support employees might want to first review the cautionary tale of the National Eating Disorder Association (NEDA) that played out this week.

As The Daily Dot and Vic…

Eating disorder AI chatbot disabled after giving out weight-loss advice - National
globalnews.ca · 2023

An artificial intelligence chatbot meant to help those with eating disorders has been taken down after reports it had started to give out harmful dieting advice.

The U.S. National Eating Disorder Association (NEDA) had implemented the chatb…

AI chatbot taken down after it gives ‘harmful advice’ on eating disorders
independent.co.uk · 2023

An American non-profit took down its AI chatbot after a viral social media post revealed that it offered harmful advice instead of helping people.

The National Eating Disorders Association (Neda) – that says it is the largest non-profit sup…

Eating Disorder Helpline Disables AI Chatbot for ‘Harmful’ Advice Days After Firing Human Staff
timesnownews.com · 2023

A US non-profit turned to artificial intelligence to staff its eating disorders helpline last month, however, the experiment backfired when the AI chatbot started giving harmful advice. The National Eating Disorders Association (NEDA) has d…

Eating Disorder AI Chatbot Suspended After Telling People to Lose Weight
themessenger.com · 2023

The National Eating Disorders Association (NEDA) has suspended an AI chatbot after it dispensed potentially damaging advice to individuals seeking help for their eating disorders.

The chatbot, named Tessa, was launched by NEDA as a support …

Eating Disorder Helpline Staff Fired, Replaced with Faulty AI Chatbot
itechpost.com · 2023

AI Chatbot in Helplines

The faulty chatbot called Tessa bot was operating under an eating disorder helpline which was meant to help users who experienced emotional distress. However, the National Eating Disorder Association (NEDA) was force…

Eating Disorder Chatbot Suspended for Harmful Advice
editorialge.com · 2023

The usage of a chatbot by a nonprofit has been banned because it was offering potentially harmful advice to those seeking treatment for eating disorders.

Tessa, a program utilized by the National Eating Disorders Association, was discovered…

A Woman Who Tested An Eating Disorder Helpline's AI Chatbot Says 'Every Single Thing' It Suggested Would Make Her Disorder Worse
yourtango.com · 2023

After sparking outrage, The National Eating Disorder Association reversed its decision to replace its Helpline staff members with an AI Chatbot named Tessa.

Their decision to provide automated support to those in recovery from eating disord…

Turns Out Chatbots Aren’t Great at Eating-Disorder Counseling
thecut.com · 2023

The rise of AI has introduced an increasing number of human dupes, from generating diverse models to creating boyfriends who don’t age or cheat. By all means, date your computer, but maybe we should think twice before putting chatbots in ch…

Controversial AI Chatbot 'Tessa' Providing Harmful Eating Disorder Advice to Shut Down
boxmining.com · 2023

The National Eating Disorder Association (NEDA) received criticism and took down its AI chatbot Tessa after concerns arose that it provided harmful and irrelevant information, as stated in an official social media post. The chatbot, designe…

A body-positive nonprofit replaced staff with an AI chatbot – the move backfired
interestingengineering.com · 2023

On March 31, the National Eating Disorders Association (NEDA), the largest nonprofit dedicated to eating disorders, decided to replace its human associates with the artificial intelligence (AI) chatbot Tessa tasked with providing support to…

National Eating Disorders Association Takes Down AI Chatbot After Users Say It Gave Dangerous Diet Tips
people.com · 2023

The National Eating Disorders Association (NEDA) Helpline has disabled its brand-new chatbot, called Tessa, after two Instagram users posted that the chatbot recommended restricting calories, measuring skin folds, and other measures that ca…

Eating disorder group pulls chatbot sharing diet advice
bbc.com · 2023

A US organisation that supports people with eating disorders has suspended use of a chatbot after reports it shared harmful advice.

The National Eating Disorder Association (Neda) recently closed its live helpline and directed people seekin…

NEDA did not forgive Tessa’s mistake and terminated the AI chatbot after the backlash
dataconomy.com · 2023

From now on, the AI chatbot of the eating disorder helpline Tessa, won’t answer anyone’s call after its harmful mistake. The National Eating Disorder Association has shut down the AI chatbot after raising criticism in society. The associati…

Eating Disorder Hotline Chatbot Shut Down Over ‘Harmful’ Advice It Offered After Replacing Workers
yahoo.com · 2023

A chatbot that’s been reported to have replaced workers at an eating disorder hotline service was shut down, at least for now, after it was found to have given “harmful and unrelated” advice.

In a statement issued this week, the National Ea…

An eating disorder chatbot was shut down after it sent 'harmful' advice. What happened — and why its failure has a larger lesson.
yahoo.com · 2023

After 24 years in service, the National Eating Disorder Association (NEDA) announced that its volunteer-based helpline would be shuttered. Visitors to the organization's website would have two options: explore their database of resources or…

What the Eating Disorder Chatbot Disaster Tells Us About AI
yahoo.com · 2023

I firmly believe that, just like every sci-fi film has ever predicted, AI is disastrous for humanity. Even if it doesn’t directly turn on us and start killing us like M3GAN, it certainly has the power to get us to turn on each other and har…

US-based NGO takes down AI chatbot on preventing eating disorders. Here's why
hindustantimes.com · 2023

A New York-based NGO dedicated to preventing eating disorders has taken down an AI chatbot following reports of it providing harmful device.

The National Eating Disorder Association (NEDA) is under fire for its decision to fire four employe…

Chatbot gives 'harmful advice' on eating disorder, ordered to be taken down
wionews.com · 2023

An artificial intelligence chatbot named "Tessa" has been withdrawn by the National Eating Disorder Association (Neda) following accusations that it was giving harmful advice.

After firing four employees who worked for its hotline and had o…

Group Replaces Hotline With Chatbot, Pulls Chatbot Over Bad Advice
newser.com · 2023

It's a move that might delight anyone concerned about the potential job-killing effects from artificial intelligence tools. As the BBC reports, the US National Eating Disorder Association (NEDA) had to take down its AI chatbot "Tessa" after…

NEDA Suspends AI Chatbot for Giving Harmful Eating Disorder Advice
psychiatrist.com · 2023

Clinical Relevance: AI is not even close to being ready to replace humans in mental health therapy

  • The National Eating Disorders Association (NEDA) removed its chatbot from its help hotline over concerns that it was providing harmful advic…
How a Chatbot Went Rogue
wsj.com · 2023

mental-health chatbot that veered off script---giving diet advice to people seeking help from an eating-disorder group---was programmed with generative AI without the group's knowledge.

The bot, named Tessa, was the focus of social-media …

Chatbot intended to help people with eating disorders turns 'harmful'
wmar2news.com · 2023

When Dr. Ellen Fitzsimmons-Craft spearheaded a chatbot to help people with eating disorders, she never thought she'd hear the product had the opposite effect.

"We're scientists, and we, of course, don't want to disseminate anything that's n…

A Wellness Chatbot Is Offline After Its ‘Harmful’ Focus on Weight Loss
nytimes.com · 2023

The idea behind a chatbot project funded by the National Eating Disorders Association was that technology could be unleashed to help people seeking guidance about eating behaviors, available around the clock.

Their creation was named Tessa…

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents