Description: Researchers at Cornell reportedly found that OpenAI's Whisper, a speech-to-text system, can hallucinate violent language and fabricated details, especially with long pauses in speech, such as from those with speech impairments. Analyzing 13,000 clips, they determined 1% contained harmful hallucinations. These errors pose risks in hiring, legal trials, and medical documentation. The study suggests improving model training to reduce these hallucinations for diverse speaking patterns.
Entities
View all entitiesAlleged: OpenAI developed an AI system deployed by OpenAI , Whisper , Companies using Whisper and Organizations integrating Whisper into customer service systems, which harmed Individuals with speech impairments , Users whose speech is misinterpreted by Whisper , Professionals relying on accurate transcriptions and General public.
Incident Stats
Incident ID
732
Report Count
1
Incident Date
2024-02-12
Editors
Daniel Atherton
Incident Reports
Reports Timeline
news.cornell.edu · 2024
- View the original report at its source
- View the report at the Internet Archive
Speak a little too haltingly and with long pauses, and a speech-to-text transcriber might put harmful, violent words in your mouth, Cornell researchers have discovered.
OpenAI's Whisper -- an artificial intelligence-powered speech recognit…
Variants
A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.
Similar Incidents
Did our AI mess up? Flag the unrelated incidents
All Image Captions Produced are Violent
· 28 reports
Gender Biases in Google Translate
· 10 reports
Similar Incidents
Did our AI mess up? Flag the unrelated incidents
All Image Captions Produced are Violent
· 28 reports
Gender Biases in Google Translate
· 10 reports