Legal system
Incidents Harmed By
Incident 96011 Report
Plaintiffs' Lawyers Admit AI Generated Erroneous Case Citations in Federal Court Filing Against Walmart
2025-02-06
Lawyers Rudwin Ayala, T. Michael Morgan (Morgan & Morgan), and Taly Goody (Goody Law Group) were fined a total of $5,000 after their Wyoming federal lawsuit filing against Walmart cited fake cases "hallucinated" by AI. Judge Kelly Rankin sanctioned them, removing Ayala from the case and noting attorneys must verify AI sources. The filing, flagged by Walmart’s legal team, led to its withdrawal and an internal review.
MoreIncident 6154 Report
Colorado Lawyer Filed a Motion Citing Hallucinated ChatGPT Cases
2023-06-13
A Colorado Springs attorney, Zachariah Crabill, mistakenly used hallucinated ChatGPT-generated legal cases in court documents. The AI software provided false case citations, leading to the denial of a motion and legal repercussions for Crabill, highlighting risks in using AI for legal research.
MoreIncident 18293 Report
Arizona Court Releases DUI Suspect After AI-Assisted Police Report Raises Probable-Cause Questions
2026-10-29
An Arizona Department of Public Safety trooper used Truleo AI, a police report-drafting tool, to help prepare a probable-cause statement after arresting a driver near Phoenix. At an initial hearing, a Maricopa County court commissioner said he could not determine how much of the narrative was authored by AI without the officer's original account and ordered the driver released. Charges remained pending; no fabricated facts were established.
MoreIncident 7042 Report
Study Highlights Persistent Hallucinations in Legal AI Systems
2024-05-23
Stanford University’s Human-Centered AI Institute (HAI) conducted a study in which they designed a "pre-registered dataset of over 200 open-ended legal queries" to test AI products by LexisNexis (creator of Lexis+ AI) and Thomson Reuters (creator of Westlaw AI-Assisted Research and Ask Practical Law AI). The researchers found that these legal models hallucinate in 1 out of 6 (or more) benchmarking queries.
More