Skip to Content
logologo
AI Incident Database
Open TwitterOpen RSS FeedOpen FacebookOpen LinkedInOpen GitHub
Open Menu
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse
Entities

LLaMA

Incidents implicated systems

Incident 9396 Report
AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

2025-02-21

OpenAI reportedly uncovered evidence of a Chinese state-linked AI-powered surveillance campaign, dubbed "Peer Review," designed to monitor and report anti-state speech on Western social media in real time. The system, believed to be built on Meta’s open-source Llama model, was detected when a developer allegedly used OpenAI’s technology to debug its code. OpenAI also reportedly identified disinformation efforts targeting Chinese dissidents and spreading propaganda in Latin America.

More

Incident 7314 Report
Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

2023-12-01

Large language models are reportedly hallucinating software package names, some of which are uploaded to public repositories and integrated into real code. One such package, huggingface-cli, was downloaded over 15,000 times. This behavior enables "slopsquatting," a term coined by Seth Michael Larson of the Python Software Foundation, where attackers register fake packages under AI-invented names and put supply chains at serious risk.

More

Incident 9962 Report
Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

2020-10-25

Meta and Bloomberg allegedly used Books3, a dataset containing 191,000 pirated books, to train their AI models, including LLaMA and BloombergGPT, without author consent. Lawsuits from authors such as Sarah Silverman and Michael Chabon claim this constitutes copyright infringement. Books3 includes works from major publishers like Penguin Random House and HarperCollins. Meta argues its AI outputs are not "substantially similar" to the original books, but legal challenges continue.

More

Incident 10202 Report
Reportedly Unsafe Deployment of Llama.cpp Reveals Interactive AI-Generated CSAM Roleplay Prompts

2025-04-11

A study by UpGuard reports that misconfigured llama.cpp servers publicly exposed user prompts, including hundreds of interactive roleplay scenarios. Some prompts explicitly described fictional sexual abuse of children aged 7–12. While no real children were involved, the findings demonstrate how open-source LLMs can be exploited to generate AI-enabled child sexual abuse material (CSAM).

More

Related Entities
Other entities that are related to the same incident. For example, if the developer of an incident is this entity but the deployer is another entity, they are marked as related entities.
 

Entity

Developers using AI-generated suggestions

Incidents involved as Deployer
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Bar Lanyado

Incidents involved as Deployer
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

OpenAI

Incidents involved as Developer
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Google

Incidents involved as Developer
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Cohere

Incidents involved as Developer
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Meta

Incidents involved as both Developer and Deployer
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

Incidents involved as Developer
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

DeepSeek AI

Incidents involved as Developer
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

BigScience

Incidents involved as Developer
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Developers and businesses incorporating AI-suggested packages

Incidents Harmed By
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Alibaba

Incidents Harmed By
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Organizations that incorporated fake dependencies

Incidents Harmed By
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Software ecosystems

Incidents Harmed By
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Users downstream of software contaminated by hallucinated packages

Incidents Harmed By
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Trust in open-source repositories and AI-assisted coding tools

Incidents Harmed By
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

LLM-powered coding assistants

Incidents implicated systems
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

ChatGPT 3.5

Incidents implicated systems
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

ChatGPT 4

Incidents implicated systems
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Gemini Pro

Incidents implicated systems
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Command

Incidents implicated systems
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

CodeLlama

Incidents implicated systems
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

DeepSeek Coder

Incidents implicated systems
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

BLOOM

Incidents implicated systems
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Python Package Index (PyPI)

Incidents implicated systems
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

npm (Node.js)

Incidents implicated systems
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

GitHub

Incidents implicated systems
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Google Search / AI Overview

Incidents implicated systems
  • Incident 731
    4 Reports

    Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers

More
Entity

Chinese state-linked actors

Incidents involved as Deployer
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

More
Entity

Chinese Communist Party

Incidents involved as Deployer
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

More
Entity

Various open-source AI developers

Incidents involved as Developer
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

More
Entity

Chinese state security researchers

Incidents involved as Developer
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

More
Entity

Western social media communities

Incidents Harmed By
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

More
Entity

Social media users in Latin America

Incidents Harmed By
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

More
Entity

social media users

Incidents Harmed By
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

More
Entity

Opposition voices against the Chinese Communist Party

Incidents Harmed By
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

More
Entity

Chinese dissidents

Incidents Harmed By
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

More
Entity

Cai Xia

Incidents Harmed By
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

More
Entity

Peer Review

Incidents implicated systems
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

More
Entity

OpenAI models

Incidents implicated systems
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

More
Entity

ChatGPT

Incidents implicated systems
  • Incident 939
    6 Reports

    AI-Powered Chinese Surveillance Campaign 'Peer Review' Used for Real-Time Monitoring of Anti-State Speech on Western Social Media

More
Entity

Various generative AI developers

Incidents involved as both Developer and Deployer
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

EleutherAI

Incidents involved as both Developer and Deployer
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Bloomberg

Incidents involved as both Developer and Deployer
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

The Pile

Incidents involved as Developer
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

Incidents implicated systems
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Shawn Presser

Incidents involved as Developer
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Zadie Smith

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Writers

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Verso

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Stephen King

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Sarah Silverman

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Richard Kadrey

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Publishers found in Books3

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Penguin Random House

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Oxford University Press

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Over 170,000 authors found in Books3

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Michael Pollan

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Margaret Atwood

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Macmillan

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

HarperCollins

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

General public

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

  • Incident 1020
    2 Reports

    Reportedly Unsafe Deployment of Llama.cpp Reveals Interactive AI-Generated CSAM Roleplay Prompts

More
Entity

Creative industries

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Christopher Golden

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Authors

Incidents Harmed By
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

hugging face

Incidents implicated systems
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

GPT-J

Incidents implicated systems
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Books3

Incidents implicated systems
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

BloombergGPT

Incidents implicated systems
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Bibliotik

Incidents implicated systems
  • Incident 996
    2 Reports

    Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI

More
Entity

Users of llama.cpp servers

Incidents involved as both Developer and Deployer
  • Incident 1020
    2 Reports

    Reportedly Unsafe Deployment of Llama.cpp Reveals Interactive AI-Generated CSAM Roleplay Prompts

Incidents Harmed By
  • Incident 1020
    2 Reports

    Reportedly Unsafe Deployment of Llama.cpp Reveals Interactive AI-Generated CSAM Roleplay Prompts

More
Entity

llama.cpp

Incidents implicated systems
  • Incident 1020
    2 Reports

    Reportedly Unsafe Deployment of Llama.cpp Reveals Interactive AI-Generated CSAM Roleplay Prompts

More
Entity

public-facing HTTP API servers with insecure /slots configuration

Incidents implicated systems
  • Incident 1020
    2 Reports

    Reportedly Unsafe Deployment of Llama.cpp Reveals Interactive AI-Generated CSAM Roleplay Prompts

More

Research

  • Defining an “AI Incident”
  • Defining an “AI Incident Response”
  • Database Roadmap
  • Related Work
  • Download Complete Database

Project and Community

  • About
  • Contact and Follow
  • Apps and Summaries
  • Editor’s Guide

Incidents

  • All Incidents in List Form
  • Flagged Incidents
  • Submission Queue
  • Classifications View
  • Taxonomies

2024 - AI Incident Database

  • Terms of use
  • Privacy Policy
  • Open twitterOpen githubOpen rssOpen facebookOpen linkedin
  • ecd56df