Skip to Content
logologo
AI Incident Database
Open TwitterOpen RSS FeedOpen FacebookOpen LinkedInOpen GitHub
Open Menu
Découvrir
Envoyer
  • Bienvenue sur AIID
  • Découvrir les incidents
  • Vue spatiale
  • Vue de tableau
  • Vue de liste
  • Entités
  • Taxonomies
  • Soumettre des rapports d'incident
  • Classement des reporters
  • Blog
  • Résumé de l’Actualité sur l’IA
  • Contrôle des risques
  • Incident au hasard
  • S'inscrire
Fermer
Découvrir
Envoyer
  • Bienvenue sur AIID
  • Découvrir les incidents
  • Vue spatiale
  • Vue de tableau
  • Vue de liste
  • Entités
  • Taxonomies
  • Soumettre des rapports d'incident
  • Classement des reporters
  • Blog
  • Résumé de l’Actualité sur l’IA
  • Contrôle des risques
  • Incident au hasard
  • S'inscrire
Fermer

Incident 704: Study Highlights Persistent Hallucinations in Legal AI Systems

Description: Stanford University’s Human-Centered AI Institute (HAI) conducted a study in which they designed a "pre-registered dataset of over 200 open-ended legal queries" to test AI products by LexisNexis (creator of Lexis+ AI) and Thomson Reuters (creator of Westlaw AI-Assisted Research and Ask Practical Law AI). The researchers found that these legal models hallucinate in 1 out of 6 (or more) benchmarking queries.

Outils

Nouveau rapportNouveau rapportNouvelle RéponseNouvelle RéponseDécouvrirDécouvrirVoir l'historiqueVoir l'historique

Entités

Voir toutes les entités
Alleged: Thomson Reuters et LexisNexis developed an AI system deployed by Legal professionals , Law firms et Organizations requiring legal research, which harmed Legal professionals , Clients of lawyers et Legal system.

Statistiques d'incidents

ID
704
Nombre de rapports
2
Date de l'incident
2024-05-23
Editeurs
Daniel Atherton
Applied Taxonomies
MIT

Classifications de taxonomie MIT

Machine-Classified
Détails de la taxonomie

Risk Subdomain

A further 23 subdomains create an accessible and understandable classification of hazards and harms associated with AI
 

7.3. Lack of capability or robustness

Risk Domain

The Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental harms, and (7) AI system safety, failures & limitations.
 
  1. AI system safety, failures, and limitations

Entity

Which, if any, entity is presented as the main cause of the risk
 

AI

Timing

The stage in the AI lifecycle at which the risk is presented as occurring
 

Post-deployment

Intent

Whether the risk is presented as occurring as an expected or unexpected outcome from pursuing a goal
 

Unintentional

Rapports d'incidents

Chronologie du rapport

+1
AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries
We asked ChatGPT for legal advice—here are five reasons why you shouldn't
AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries

AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries

hai.stanford.edu

We asked ChatGPT for legal advice—here are five reasons why you shouldn't

We asked ChatGPT for legal advice—here are five reasons why you shouldn't

theconversation.com

AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries
hai.stanford.edu · 2024

Artificial intelligence (AI) tools are rapidly transforming the practice of law. Nearly three quarters of lawyers plan on using generative AI for their work, from sifting through mountains of case law to drafting contracts to reviewing docu…

We asked ChatGPT for legal advice—here are five reasons why you shouldn't
theconversation.com · 2024

At some point in your life, you are likely to need legal advice. A survey carried out in 2023 by the Law Society, the Legal Services Board and YouGov found that two-thirds of respondents had experienced a legal issue in the past four years.…

Variantes

Une "Variante" est un incident qui partage les mêmes facteurs de causalité, produit des dommages similaires et implique les mêmes systèmes intelligents qu'un incident d'IA connu. Plutôt que d'indexer les variantes comme des incidents entièrement distincts, nous listons les variations d'incidents sous le premier incident similaire soumis à la base de données. Contrairement aux autres types de soumission à la base de données des incidents, les variantes ne sont pas tenues d'avoir des rapports en preuve externes à la base de données des incidents. En savoir plus sur le document de recherche.

Incidents similaires

Par similarité textuelle

Did our AI mess up? Flag the unrelated incidents

COMPAS Algorithm Performs Poorly in Crime Recidivism Prediction

A Popular Algorithm Is No Better at Predicting Crimes Than Random People

May 2016 · 22 rapports
Gender Biases in Google Translate

Semantics derived automatically from language corpora contain human-like biases

Apr 2017 · 10 rapports
Personal voice assistants struggle with black voices, new study shows

Personal voice assistants struggle with black voices, new study shows

Mar 2020 · 2 rapports
Incident précédentProchain incident

Incidents similaires

Par similarité textuelle

Did our AI mess up? Flag the unrelated incidents

COMPAS Algorithm Performs Poorly in Crime Recidivism Prediction

A Popular Algorithm Is No Better at Predicting Crimes Than Random People

May 2016 · 22 rapports
Gender Biases in Google Translate

Semantics derived automatically from language corpora contain human-like biases

Apr 2017 · 10 rapports
Personal voice assistants struggle with black voices, new study shows

Personal voice assistants struggle with black voices, new study shows

Mar 2020 · 2 rapports

Recherche

  • Définition d'un « incident d'IA »
  • Définir une « réponse aux incidents d'IA »
  • Feuille de route de la base de données
  • Travaux connexes
  • Télécharger la base de données complète

Projet et communauté

  • À propos de
  • Contacter et suivre
  • Applications et résumés
  • Guide de l'éditeur

Incidents

  • Tous les incidents sous forme de liste
  • Incidents signalés
  • File d'attente de soumission
  • Affichage des classifications
  • Taxonomies

2024 - AI Incident Database

  • Conditions d'utilisation
  • Politique de confidentialité
  • Open twitterOpen githubOpen rssOpen facebookOpen linkedin
  • ecd56df