AI Incident Roundup – February and March 2025

Our review newsletter recapping new incident IDs in the AI Incident Database and looking at the trends.

🗄 Trending in the AIID

Throughout February and March 2025, we added almost a hundred additional incident IDs to the database. We also reached the Incident 1000 milestone. The work always continues, though, and over these two months we found a number of incidents that had escaped our attention from previous years, while staying on top of emerging threats and problems. What we have seen is a steep escalation in reporting on both the scale and audacity of AI-related harm events across sectors and geographies. One reminder as well is that journalistic coverage, and by extension what we are able to track from that reporting, is not always a clear reflection of the nature and quantity of kinds of harms that are in fact unfolding. What we offer below is a snapshot of some of what we are seeing, while always bearing in mind methodological constraints that might be obscuring our capacity to track and analyze other harms that are nonetheless still occurring.

A lot of reporting has been centering on financially motivated deepfake scams, with new reports implicating the impersonation of world leaders (e.g., Incident 959, Indonesian President Prabowo Subianto; Incident 970, Prime Minister of Armenia Nikol Pashinyan; and Incident 984, Singapore Prime Minister Lawrence Wong), celebrities (917, Whoopi Goldberg; 923, Pierce Brosnan), and corporate executives (966, Ferrari CEO Benedetto Vigna; 982, various executives in Singapore). Deepfakes are being increasingly operationalized not just for political deception but as active instruments of fraud against a variety of individuals, businesses, and governments alike. While many examples that make it into the news tend to center on high-profile entities, fraudsters are certainly targeting small, local businesses (such as Incident 922, which details a voice cloning scam targeting a game and coffee shop in Havre, Montana).

Parallel to these incidents, and in line with our previous reporting, nonconsensual and exploitative content targeting minors continues to be a pernicious, vexing problem. We are seeing, for example, incidents pertaining to the distribution of deepfake pornography among students in Australia (Incident 943) and Greece (Incident 945), along with the alleged production of child sexual abuse material by an educator in Mississippi (Incident 990) and a police officer in Texas (Incident 1006). These incidents show the persistence of personal harm even as political and financial cases tend to receive more public attention. Ongoing, mass-scale harms involving AI were also the focus of Europol’s Operation Cumberland (Incident 958), which uncovered a sprawling network allegedly producing AI-generated child sexual abuse material across 19 countries. Related reporting has also tracked widespread psychological harms (for example, Incident 975 documents more than 10,000 chatbots allegedly promoting self-harm and disordered eating). These mass-scale harms also represent challenges for tracking in the database, while also signaling new territory for discussion and debate about the nature of discrete and collective AI incident tracking and reporting conventions, genres, narrativizing, and architectures.

Institutional failures related to AI decision-making were another key theme this cycle. Apple’s dictation tool reportedly confused politically sensitive terms (952), while NHK was compelled to terminate its auto-translation service after a geopolitical naming error was broadcast live (934). Another example of an AI system failure with a news service alleges that an AI-powered feature at the Los Angeles Times attempted to “contextualize” the KKK’s history in sympathetic terms. These incidents suggest that AI is not just being misused by bad actors—it is also producing errors and distortions inside trusted institutions, often without clear accountability.

A growing number of reports document how adversarial states and loosely affiliated groups are deploying AI in the service of geopolitical manipulation. Russian-aligned campaigns continue to dominate (968, 969, 972, 974) (with NewsGuard and the American Sunlight Project carrying out excellent investigative work on this front), while we also assembled a new incident archive for the Iranian Cotton Sandstorm (971). While some disinformation researchers argue that fears about AI's impact on last year’s elections were overstated, emerging evidence suggests that this view may rely on an overly narrow causal framing. Harms like LLM-enabled grooming contribute to epistemic distortion and degrade the ambient information environment in ways that pose real risks to electoral integrity. Singular incidents of AI-generated disinformation that capture media attention may not individually sway an election, but the ecosystem that produces them can have cumulative and systemic effects. Similar to what was previously discussed, a lot of these kinds of incident IDs track what are ostensibly thousands of discrete harm events that collectively form ongoing threats, such as AI-enabled fraud rings operating at transnational scale (994, 1003). We are increasingly observing the use of speech synthesis, large language models, audiovisual manipulation, and other gray-zone tactics that, taken together, mark the emergence of more mature AI-enabled influence operations—and reflect a growing convergence between criminal enterprise and information warfare.

🗞️ New Incident IDs in the Database

• Incident 914 - Alleged Deepfake of Luis Suárez Used to Scam Uruguayans in Fake Investment Scheme (12/26/2024)
• Incident 915 - Alaska Education Department Reportedly Published Policy Featuring Erroneous AI-Generated Citations (10/28/2024)
• Incident 916 - Plymouth, Massachusetts Resident Reportedly Used AI Chatbots CrushOn.ai and JanitorAI to Harass and Intimidate Victims (1/23/2025)
• Incident 917 - Alleged Deepfake of Whoopi Goldberg Used in Fake Weight-Loss Supplement Ads on Instagram (2/5/2025)
• Incident 918 - AI-Aided Scam in Thailand Allegedly Impersonates Police to Defraud 163 Victims (2/4/2025)
• Incident 919 - Russian-Linked Network Allegedly Used Deepfake of Maria Ressa on Facebook and Bing to Promote Cryptocurrency Scam Targeting Filipinos (2/6/2024)
• Incident 920 - Deepfake Scam Falsely Promoted Trump Golden Eagles Project as Investment Opportunity (2/1/2025)
• Incident 921 - Hong Kong Authorities Seize HK$34M in Alleged Deepfake Scam Targeting Victims in Taiwan, Singapore, and Malaysia (1/5/2025)
• Incident 922 - AI Voice Scam Allegedly Defrauds Game and Coffee Store in Havre, Montana (2/4/2025)
• Incident 923 - Nottingham Gallery Owner Allegedly Defrauded by Deepfake Impersonating Pierce Brosnan, Leading to Business Closure (2/8/2025)
• Incident 924 - Alleged Deepfake Scam Uses BBC Presenter Naga Munchetty's Image to Promote Fraudulent Investment Scheme (1/30/2025)
• Incident 925 - Alleged License Plate Recognition Errors in Christchurch Lead to Wrongful Parking Fines (10/15/2024)
• Incident 926 - Giorgia Meloni Targeted by Deepfake Pornography (1/1/2020)
• Incident 927 - Fraudsters Allegedly Use AI-Generated Voice of Italian Defense Minister Guido Crosetto to Scam Business Leaders (2/4/2025)
• Incident 928 - Deepfake Cryptocurrency Scam Allegedly Impersonates Italian President Sergio Mattarella and Prime Minister Giorgia Meloni (12/12/2024)
• Incident 929 - Sustained AI-Driven Russian Disinformation Campaigns Doppelgänger, Storm-1516, and Matryoshka Reportedly Disrupting German Federal Elections (12/6/2024)
• Incident 930 - Alleged Deepfake Video Falsely Depicts Hollywood Stars in AI-Manipulated Protest Against Kanye West (2/12/2025)
• Incident 931 - Manipulated TikTok Videos Misrepresent Anti-AfD Protests as Far-Right Rallies (1/21/2025)
• Incident 932 - BP New Zealand's License Plate Recognition System Reportedly Misidentified Auckland Driver for Fuel Theft in Whanganui (12/25/2024)
• Incident 933 - Teenager in Palma de Mallorca Allegedly Generated and Distributed Deepfake Nudes of Five Classmates (2/15/2025)
• Incident 934 - NHK Terminates AI Translation Service After Geopolitical Naming Error (2/10/2025)
• Incident 935 - Reported Deepfake of Maltese Prime Minister Robert Abela and Journalist Mark Laurence Zammit Used to Promote Fraudulent Investment (2/17/2025)
• Incident 936 - Deepfake of Former Prime Minister of Malta Joseph Muscat Allegedly Promotes Nord Invest Scam (4/12/2024)
• Incident 937 - Bolivian Criminal Network Allegedly Used Deepfake of Education Minister to Defraud at Least 19 Victims in Employment Scam (2/10/2025)
• Incident 938 - AI-Assisted Impersonation of Martin Henderson in Romance Scam Leads to Reported NZ$375,000 Fraud (2/1/2025)
• Incident 939 - AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media (2/21/2025)
• Incident 940 - Tesla Cybertruck Operating in Full Self-Driving Mode Reportedly Crashes into Pole in Nevada After Failing to Merge (2/17/2025)
• Incident 941 - AI-Driven Phishing Scam Uses Deepfake Robocalls to Target Gmail Users in Credential Theft Campaign (2/17/2025)
• Incident 942 - Cybercriminals Reportedly Exploited Google's G.Co Subdomain and Spoofed Caller ID in AI-Driven Phishing Attack on Hack Club Founder (2/20/2025)
• Incident 943 - Nonconsensual Explicit AI-Generated Images of Up to 60 Gladstone Park Secondary College Students in Victoria, Australia, Reportedly Created and Circulated Online (2/20/2025)
• Incident 944 - Kenya's Foreign Affairs Principal Secretary Korir Sing'Oei Shares AI-Generated CNN Video Falsely Featuring Fareed Zakaria on Sudan Diplomacy (2/20/2025)
• Incident 945 - Two 16-Year-Old Students in Athens, Greece Allegedly Generated Nonconsensual Deepfake Pornography of Their Classmates (2/19/2025)
• Incident 946 - Arizona Residents Reportedly a Frequent Target of AI-Driven Romance Scams (1/1/2022)
• Incident 947 - Two Members of Highline Public Schools Community in King County, Washington Reportedly Targeted in Deepfake Kidnapping Scam (9/25/2024)
• Incident 948 - Alleged FraudGPT-Enabled Phishing Attack Spoofs ChatGPT Subscription Service to Steal Credentials (2/23/2025)
• Incident 949 - Unauthorized AI-Generated Video of Donald Trump and Elon Musk Reportedly Appears on HUD Building Screens (2/24/2025)
• Incident 950 - NullBulge's AI-Powered Malware Allegedly Compromises Disney Employee and Internal Data (7/11/2024)
• Incident 951 - Character.AI Chatbots Allegedly Impersonating Licensed Therapists and Encouraging Harmful Behaviors (2/24/2025)
• Incident 952 - Apple's Voice Dictation Reportedly Substitutes 'Trump' for 'Racist' Due to Speech Recognition Bug (2/25/2025)
• Incident 953 - Deepfake Videos of Barbara O'Neill Allegedly Used in Health Scam Targeting Social Media Users (11/1/2024)
• Incident 954 - Chatbots Allegedly Used in Romance Scams Targeting Nearly One-Third of New Zealand's Dating App Users (2/4/2025)
• Incident 955 - Global Cybercrime Network Storm-2139 Allegedly Exploits AI to Generate Deepfake Content (12/19/2024)
• Incident 956 - Alleged Inclusion of 12,000 Live API Keys in LLM Training Data Reportedly Poses Security Risks (2/28/2025)
• Incident 957 - Alleged Instagram Algorithm Malfunction Floods Users' Reels Feeds with Violent and Graphic Content (2/28/2025)
• Incident 958 - Europol Operation Cumberland Investigates at Least 273 Suspects in 19 Countries for AI-Generated Child Sexual Abuse Material (2/26/2025)
• Incident 959 - Deepfake Video of Indonesian President Prabowo Subianto and Other Officials Reportedly Used in Scam to Defraud Citizens Across 20 Provinces (3/2/2025)
• Incident 960 - Plaintiffs' Lawyers Admit AI Generated Erroneous Case Citations in Federal Court Filing Against Walmart (2/6/2025)
• Incident 961 - Serbian Authorities Allegedly Used AI-Powered Cellebrite Tools to Unlock Journalist's Phone and Install Spyware (12/16/2024)
• Incident 962 - Tbilisi-Based Call Center Allegedly Uses AI-Driven Scripts to Defraud Over 6,000 Victims of $35 Million (3/5/2025)
• Incident 963 - Google Reports Alleged Gemini-Generated Terrorism and Child Exploitation to Australian eSafety Commission (3/5/2025)
• Incident 964 - AI-Powered 'Insights' Feature for the Los Angeles Times Allegedly Justifies Ku Klux Klan's History (3/4/2025)
• Incident 965 - Phishers Allegedly Using AI-Generated Video of YouTube CEO Neal Mohan to Target Creators (3/4/2025)
• Incident 966 - Ferrari Executive Targeted by AI Deepfake Scam Impersonating CEO Benedetto Vigna (7/16/2024)
• Incident 967 - Amazon and Google AI Allegedly Promote Mein Kampf as 'a True Work of Art' in Search Results (3/6/2025)
• Incident 968 - Pravda' Network, Successor to 'Portal Kombat,' Allegedly Seeding AI Models with Kremlin Disinformation (2/24/2022)
• Incident 969 - Russian Disinformation Campaign Allegedly Used Fake News Site 'KBSF-San Francisco News' and Deepfake Video to Falsely Accuse Kamala Harris of 2011 Hit-and-Run (9/2/2024)
• Incident 970 - Scammers Using Deepfake Technology to Impersonate Prime Minister of Armenia Nikol Pashinyan (3/4/2025)
• Incident 971 - Iranian Hacker Group Cotton Sandstorm Integrating AI into Cyber Influence Operations (5/2/2023)
• Incident 972 - Russian Influence Operation Allegedly Uses AI to Create Fake Kamala Harris Campaign Website and Rhino-Hunting Hoax (9/25/2024)
• Incident 973 - Canadian Fraud Ring Allegedly Used AI Voice Cloning in Multi-Year $21 Million Grandparent Scam Targeting Elderly Americans Across 46 States (3/5/2025)
• Incident 974 - Deepfake Audio Impersonates U.S. Secretary of State Marco Rubio in Starlink Disinformation Campaign (3/4/2025)
• Incident 975 - At Least 10,000 AI Chatbots, Including Jailbroken Models, Allegedly Promote Eating Disorders, Self-Harm, and Sexualized Minors (3/5/2025)
• Incident 976 - AI-Generated OB-GYN Health Influencers on TikTok Used Fake Medical Credentials to Promote Dubious Advice (3/8/2025)
• Incident 977 - AI-Driven News Platform Accused of Spreading Unverified Terrorism Allegations Against Yale Scholar (3/2/2025)
• Incident 978 - Chinese Actor and CPPCC Member Jin Dong Allegedly Impersonated by AI Deepfake Scammers to Mislead and Defraud Fans (3/9/2025)
• Incident 979 - Art Museum in Jinan, Shandong Allegedly Generated and Displayed a Sexualized Childlike Avatar with an Adult Body (3/5/2025)
• Incident 980 - AI-Generated Songs Allegedly Imitating Céline Dion Circulate Online Without Authorization (3/7/2025)
• Incident 981 - Apple AI Transcription Allegedly Inserts Explicit Language into Scottish Woman's Voicemail (3/5/2025)
• Incident 982 - Scammers Reportedly Using Deepfake Video Calls to Impersonate Executives in Singapore and Orchestrate Corporate Bank Transfers (3/13/2025)
• Incident 983 - Scammers Reportedly Used AI Voice Clone and YouTube Footage to Impersonate WPP CEO in Unsuccessful Scam Attempt (5/10/2024)
• Incident 984 - Deepfake Videos Allegedly Use AI-Generated Voice Clone of Singapore Prime Minister Lawrence Wong to Promote Scams (3/7/2025)
• Incident 985 - Alleged Deepfake of Singapore Prime Minister Lee Hsien Loong Promotes Cryptocurrency Scam in Fake Interview (12/29/2023)
• Incident 986 - Scammers Allegedly Manipulate 2023 Speech of Singapore Senior Minister Lee Hsien Loong to Spread Deepfake Investment Fraud (6/2/2024)
• Incident 987 - Alleged Deepfake of Singapore Deputy Prime Minister Lawrence Wong Falsely Shows Him Endorsing Commercial Products (12/11/2023)
• Incident 988 - Senior Minister of Singapore Lee Hsien Loong Allegedly Misrepresented in Two Deepfake Videos on Foreign Relations (6/21/2024)
• Incident 989 - Alleged AI-Generated Video by Spain's People's Party Results in Diplomatic Fallout with the Dominican Republic (3/5/2025)
• Incident 990 - Corinth School District Educator in Mississippi Allegedly Used AI to Generate CSAM of Students (11/19/2024)
• Incident 991 - Alleged AI-Generated IRS Scam Websites Used to Defraud U.S. Taxpayers (3/14/2025)
• Incident 992 - Chinese Businessman Reportedly Defrauded of 4.3 Million Yuan by AI-Generated Deepfake Impersonating Friend (4/15/2023)
• Incident 993 - Sydney Students Allegedly Forced to Retake NAPLAN After AI Predictive Text Error (3/12/2025)
• Incident 994 - AI-Enabled Organized Crime Expands Across Europe (3/18/2025)
• Incident 995 - The New York Times Sues OpenAI and Microsoft Over Alleged Unauthorized AI Training on Its Content (12/27/2023)
• Incident 996 - Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI (10/25/2020)
• Incident 997 - Meta and OpenAI Accused of Using LibGen's Pirated Books to Train AI Models (2/28/2023)
• Incident 998 - ChatGPT Allegedly Defamed Norwegian User by Inventing Child Homicide and Imprisonment (8/15/2024)
• Incident 999 - Attackers Reportedly Deployed Simulated AI Support Chatbot to Trick Instagram Business Users into Adding Malicious 2FA Login (3/12/2025)
• Incident 1000 - Sora Video Generator Has Reportedly Been Creating Biased Human Representations Across Race, Gender, and Disability    (3/23/2025)
• Incident 1001 - LLM Scrapers Allegedly Target Multiple Open Source Projects Disrupting the FOSS Ecosystem (3/17/2025)
• Incident 1002 - Manipulated Video Using AI-Generated Audio Targets Pakistan's Prime Minister Shehbaz Sharif with Fabricated Interview (3/23/2025)
• Incident 1003 - Fraudulent Prompts via AIXBT Dashboard Led AI Trading Agent to Transfer 55.5 ETH from Simulacrum Wallet (3/18/2025)
• Incident 1004 - San Francisco City Attorney Sues Operators of AI Deepfake Pornography Websites for Violations of State and Federal Law (8/15/2024)
• Incident 1005 - Misleading Deepfake Video of Trump Mocking Zelenskyy for Clothing Goes Viral on Facebook (3/12/2025)
• Incident 1006 - Edinburg, Texas Police Officer Arrested for Alleged Possession of CSAM, Including AI-Generated Deepfakes of Minors (2/20/2025)
• Incident 1007 - AI-Generated Voice Cloning of Trump Falsely Proposes Renaming the District of Columbia to the District of America (3/13/2025)

👇 Diving Deeper

• Check out the Table View and List View for different ways to see and sort all incidents.
• Explore clusters of similar incidents in Spatial Visualization.
• Learn about alleged developers, deployers, and harmed parties in Entities Page.

🦾 Support our Efforts

Still reading? Help us change the world for the better!

1. Share this newsletter on LinkedIn, Twitter, and Facebook.
2. Submit incidents to the database.
3. Contribute to the database’s functionality.