CSETv0
What is the CSET Taxonomy?
The CSET AI Harm Taxonomy for AIID is the second edition of the CSET incident taxonomy. It characterizes the harms, entities and technologies involved in AI incidents and the circumstances of their occurrence. Every incident is independently classified by two CSET annotators. Annotations are peer reviewed and finally randomly selected for quality control ahead of publication. Despite this rigorous process, mistakes do happen, and readers are invited to report any errors they might discover while browsing.
Taxonomy Fields
Overall severity of harm Searchable in Discover App
- Negligible46 Incidents
- Minor19 Incidents
- Unclear/unknown16 Incidents
- Moderate12 Incidents
- Severe6 Incidents
Definition: An estimate of the overall severity of harm caused. "Negligible" harm means minor inconvenience or expense, easily remedied. “Minor” harm means limited damage to property, social stability, the political system, or civil liberties occurred or nearly occurred. "Moderate" harm means that humans were injured (but not killed) or nearly injured, or that financial, property, social, or political interests or civil liberties were materially affected (or nearly so affected). "Severe" harm means that a small number of humans were or were almost gravely injured or killed, or that financial, property, social, or political interests or civil liberties were significantly disrupted at at least a regional or national scale (or nearly so disrupted). "Critical" harm means that many humans were or were almost killed, or that financial, property, social, or political interests were seriously disrupted at a national or global scale (or nearly so disrupted).
Uneven distribution of harms basis Searchable in Discover App
- Race23 Incidents
- Sex13 Incidents
- Religion7 Incidents
- National origin or immigrant status6 Incidents
- Age5 Incidents
Definition: If harms were unevenly distributed, this field indicates the basis or bases on which they were unevenly distributed.
Harm type Searchable in Discover App
- Harm to social or political systems19 Incidents
- Psychological harm18 Incidents
- Harm to physical health/safety17 Incidents
- Harm to civil liberties16 Incidents
- Financial harm12 Incidents
Definition: Indicates the type(s) of harm caused or nearly caused by the incident.
System developer Searchable in Discover App
Definition: The entity that created the AI system.
Sector of deployment Searchable in Discover App
- Information and communication26 Incidents
- Transportation and storage13 Incidents
- Arts, entertainment and recreation13 Incidents
- Public administration and defence12 Incidents
- Administrative and support service activities7 Incidents
Definition: The primary economic sector in which the AI system(s) involved in the incident were operating.
Relevant AI functions Searchable in Discover App
Definition: Indicates whether the AI system(s) were intended to perform any of the following high-level functions: "Perception," i.e. sensing and understanding the environment; "Cognition," i.e. making decisions; or "Action," i.e. carrying out decisions through physical or digital means.
AI tools and techniques used Searchable in Discover App
- machine learning19 Incidents
- Facial recognition6 Incidents
- open-source6 Incidents
- natural language processing5 Incidents
- environmental sensing5 Incidents
Definition: Open-ended tags that indicate the hardware and software involved in the AI system(s).
AI functions and applications used Searchable in Discover App
- decision support10 Incidents
- autonomous driving9 Incidents
- recommendation engine9 Incidents
- Facial recognition8 Incidents
- image recognition8 Incidents
Definition: Open-ended tags that describe the functions and applications of the AI system.
Location Searchable in Discover App
- Global27 Incidents
- United States6 Incidents
- New Zealand2 Incidents
- Palo Alto, CA2 Incidents
- United Kingdom2 Incidents
Loading...
Definition: The location or locations where the incident played out.
Named entities Searchable in Discover App
Definition: All named entities (such as people, organizations, locations, and products - generally proper nouns) that seem to have a significant relationship with this event, as indicated by the available evidence.
Party responsible for AI system Searchable in Discover App
Definition: A list of parties (up to three) that were responsible for the relevant AI tool or system, i.e. that had operational control over the AI-related system causing harm (or control over those who did).
Harm nearly missed? Searchable in Discover App
Definition: Was harm caused, or was it a near miss?
Probable level of intent Searchable in Discover App
Definition: Indicates whether the incident was deliberate/expected or accidental, based on the available evidence. "Deliberate or expected" applies if it is established or highly likely that the system acted more or less as expected, from the perspective of at least one of the people or entities responsible for it. “Accident” applies if it is established or highly likely that the harm arose from the system acting in an unexpected way. "Unclear" applies if the evidence is contradictory or too thin to apply either of the above labels.
Human lives lost Searchable in Discover App
Definition: Marked "trur" if one or more people died as a result of the accident, "false" if there is no evidence of lives being lost, "unclear" otherwise.
Critical infrastructure sectors affected Searchable in Discover App
- Transportation10 Incidents
- Healthcare and public health4 Incidents
- Communications2 Incidents
- Government facilities2 Incidents
- Information technology1 Incident
Definition: Where applicable, this field indicates if the incident caused harm to any of the economic sectors designated by the U.S. government as critical infrastructure.
Public sector deployment Searchable in Discover App
Definition: "Yes" if the AI system(s) involved in the accident were being used by the public sector or for the administration of public goods (for example, public transportation). "No" if the system(s) were being used in the private sector or for commercial purposes (for example, a ride-sharing company), on the other.
Nature of end user Searchable in Discover App
Definition: "Expert" if users with special training or technical expertise were the ones meant to benefit from the AI system(s)’ operation; "Amateur" if the AI systems were primarily meant to benefit the general public or untrained users.
Level of autonomy Searchable in Discover App
Definition: The degree to which the AI system(s) functions independently from human intervention. "High" means there is no human involved in the system action execution; "Medium" means the system generates a decision and a human oversees the resulting action; "low" means the system generates decision-support output and a human makes a decision and executes an action.
Physical system Searchable in Discover App
- Software only66 Incidents
- Vehicle/mobile robot16 Incidents
- Consumer device7 Incidents
- Unknown/unclear2 Incidents
- Other:Medical system1 Incident
Definition: Where relevant, indicates whether the AI system(s) was embedded into or tightly associated with specific types of hardware.
Causative factors within AI system Searchable in Discover App
Definition: Indicates which, if any, of the following types of AI failure describe the incident: "Specification," i.e. the system's behavior did not align with the true intentions of its designer, operator, etc; "Robustness," i.e. the system operated unsafely because of features or changes in its environment, or in the inputs the system received; "Assurance," i.e. the system could not be adequately monitored or controlled during operation.
Full description of the incident
Definition: A plain-language description of the incident in one paragraph or less.
Short description of the incident
Definition: A one-sentence description of the incident.
Description of AI system involved
Definition: A brief description of the AI system(s) involved in the incident, including the system’s intended function, the context in which it was deployed, and any available details about the algorithms, hardware, and training data involved in the system.
Beginning date
Definition: The date the incident began.
Ending date
Definition: The date the incident ended.
Total financial cost
Definition: The stated or estimated financial cost of the incident, if reported.
Laws covering the incident
Definition: Relevant laws under which entities involved in the incident may face legal liability as a result of the incident.
Description of the data inputs to the AI systems
Definition: A brief description of the data that the AI system(s) used or were trained on.