Incident 464: ChatGPT Provided Non-Existent Citations and Links when Prompted by Users

Description: When prompted about providing references, ChatGPT was reportedly generating non-existent but convincing-looking citations and links, which is also known as "hallucination".


New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History
Alleged: OpenAI developed and deployed an AI system, which harmed ChatGPT users.

Incident Stats

Incident ID
Report Count
Incident Date
Khoa Lam
Aligning Language Models to Follow Instructions · 2022

We've trained language models that are much better at following user intentions than GPT-3 while also making them more truthful and less toxic, using techniques developed through our alignment research. These InstructGPT models, which are t…

ChatGPT produces made-up nonexistent references · 2022

I was trying to find references to mathematical treatment of lists, and queried ChatGPT the following:

Please provide references that deal with the mathematical properties of lists

To which ChatGPT responded as follows:

    "Introduction to…
Why is Everyone Bashing ChatGPT? · 2022

Unfounded assumptions, bad advice, incorrect information—the biggest source of problems on the internet today is people blindly buying into hype. ChatGPT, which has taken the internet by the storm recently, seems to be making it a lot easie…

ChatGPT should be considered a malevolent AI and destroyed · 2023

“I’m sorry Dave, I’m afraid I can’t do that.”

These were the words that introduced most people in my generation to the concept of an AI gone rogue; HAL 9000 in the classic science fiction movie 2001: A Space Odyssey, eventually went insane …


A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.
Incident Date:
Description of Incident Circumstances

New text example with more than 80 characters. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Inputs / Outputs

New Input text

Test output text with markdown