Skip to Content
logologo
AI Incident Database
Open TwitterOpen RSS FeedOpen FacebookOpen LinkedInOpen GitHub
Open Menu
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse

Incident 796: Facebook's Content Moderation System Flagged and Removed Emergency Updates as Spam During Wildfires

Description: Facebook's AI moderation system wrongly flagged and removed dozens of posts containing vital emergency information during California's wildfire season, including real-time updates on evacuations and fire tracking. Posts from official sources like Cal Fire and the U.S. Forest Service were marked as spam, potentially endangering lives by restricting access to crucial updates. Despite user complaints, the issue persisted, with Facebook acknowledging the problem only after media inquiry.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History

Entities

View all entities
Alleged: Meta and Facebook developed and deployed an AI system, which harmed California residents , Wildfire evacuees , Emergency responders , Disaster relief workers , Fire safety coordinators , Facebook users and Facebook users in disaster zones.

Incident Stats

Incident ID
796
Report Count
3
Incident Date
2024-06-01
Editors
Daniel Atherton
Applied Taxonomies
MIT

MIT Taxonomy Classifications

Machine-Classified
Taxonomy Details

Risk Subdomain

A further 23 subdomains create an accessible and understandable classification of hazards and harms associated with AI
 

7.3. Lack of capability or robustness

Risk Domain

The Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental harms, and (7) AI system safety, failures & limitations.
 
  1. AI system safety, failures, and limitations

Entity

Which, if any, entity is presented as the main cause of the risk
 

AI

Timing

The stage in the AI lifecycle at which the risk is presented as occurring
 

Post-deployment

Intent

Whether the risk is presented as occurring as an expected or unexpected outcome from pursuing a goal
 

Unintentional

Incident Reports

Reports Timeline

Incident OccurrenceFacebook is blocking emergency warnings as wildfires roar through West+1
Facebook flagged and removed emergency wildfire information as 'spam'
Facebook is blocking emergency warnings as wildfires roar through West

Facebook is blocking emergency warnings as wildfires roar through West

washingtonpost.com

Facebook flagged and removed emergency wildfire information as 'spam'

Facebook flagged and removed emergency wildfire information as 'spam'

mashable.com

Facebook lands in hot water after marking vital wildfire safety warnings as spam: 'It's life-threatening'

Facebook lands in hot water after marking vital wildfire safety warnings as spam: 'It's life-threatening'

thecooldown.com

Facebook is blocking emergency warnings as wildfires roar through West
washingtonpost.com · 2024

Around 2 p.m. Sunday afternoon, Lauri Hutchinson's daughter ran inside yelling, "Fire."

A column of gray smoke was rapidly ballooning from the remote Northern California town of Clearlake, directly across the lake from them. A retired firef…

Facebook flagged and removed emergency wildfire information as 'spam'
mashable.com · 2024

Facebook is reportedly taking down important safety information as wildfire season spreads across the western part of the country.

An investigation by The Washington Post found dozens of instances of the tech giant flagging and removing pos…

Facebook lands in hot water after marking vital wildfire safety warnings as spam: 'It's life-threatening'
thecooldown.com · 2024

Facebook is under heavy scrutiny after flagging emergency updates on extreme weather events posted by volunteer groups, departments, and local organizations as spam.

The Washington Post reported collecting 40 examples of Facebook taking dow…

Variants

A "variant" is an AI incident similar to a known case—it has the same causes, harms, and AI system. Instead of listing it separately, we group it under the first reported incident. Unlike other incidents, variants do not need to have been reported outside the AIID. Learn more from the research paper.
Seen something similar?

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents

Waze Navigates Motorists into Wildfires

Waze Navigates Motorists into Wildfires

Dec 2017 · 22 reports
Uber AV Killed Pedestrian in Arizona

Uber AV Killed Pedestrian in Arizona

Mar 2018 · 25 reports
Security Robot Rolls Over Child in Mall

Security Robot Rolls Over Child in Mall

Jul 2016 · 27 reports
Previous IncidentNext Incident

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents

Waze Navigates Motorists into Wildfires

Waze Navigates Motorists into Wildfires

Dec 2017 · 22 reports
Uber AV Killed Pedestrian in Arizona

Uber AV Killed Pedestrian in Arizona

Mar 2018 · 25 reports
Security Robot Rolls Over Child in Mall

Security Robot Rolls Over Child in Mall

Jul 2016 · 27 reports

Research

  • Defining an “AI Incident”
  • Defining an “AI Incident Response”
  • Database Roadmap
  • Related Work
  • Download Complete Database

Project and Community

  • About
  • Contact and Follow
  • Apps and Summaries
  • Editor’s Guide

Incidents

  • All Incidents in List Form
  • Flagged Incidents
  • Submission Queue
  • Classifications View
  • Taxonomies

2024 - AI Incident Database

  • Terms of use
  • Privacy Policy
  • Open twitterOpen githubOpen rssOpen facebookOpen linkedin
  • d414e0f