概要: Large language models have reportedly hallucinated non-existent software package names, some of which were subsequently uploaded to public repositories and incorporated into real codebases. In one case, a package named huggingface-cli, which was purported to have been originally suggested by an AI model, was downloaded more than 15,000 times. This dynamic enables what security researchers have termed "slopsquatting," in which attackers register hallucinated package names and introduce potential malware into software supply chains.
Editor Notes: See Bar Lanyado's report at: https://www.lasso.security/blog/ai-package-hallucinations. See Spracklen, et al's preprint here, "We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs," here: https://arxiv.org/abs/2406.10279. See Zhou, et al's study, "Larger and more instructable language models become less reliable," here: https://doi.org/10.1038/s41586-024-07930-y.
Alleged: OpenAI , Meta , Google , DeepSeek AI , Cohere と BigScience developed an AI system deployed by Developers using AI-generated suggestions と Bar Lanyado, which harmed Users downstream of software contaminated by hallucinated packages , Trust in open-source repositories and AI-assisted coding tools , Software ecosystems , Organizations that incorporated fake dependencies , Developers and businesses incorporating AI-suggested packages と Alibaba.
関与が疑われるAIシステム: Python Package Index (PyPI) , npm (Node.js) , LLM-powered coding assistants , LLaMA , Google Search / AI Overview , GitHub , Gemini Pro , DeepSeek Coder , Command , CodeLlama , BLOOM , GPT-4 と GPT-3.5
インシデントのステータス
Risk Subdomain
A further 23 subdomains create an accessible and understandable classification of hazards and harms associated with AI
3.1. False or misleading information
Risk Domain
The Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental harms, and (7) AI system safety, failures & limitations.
- Misinformation
Entity
Which, if any, entity is presented as the main cause of the risk
AI
Timing
The stage in the AI lifecycle at which the risk is presented as occurring
Post-deployment
Intent
Whether the risk is presented as occurring as an expected or unexpected outcome from pursuing a goal
Unintentional