CSETv0

What is the CSET Taxonomy?

The CSET AI Harm Taxonomy for AIID is the second edition of the CSET incident taxonomy. It characterizes the harms, entities and technologies involved in AI incidents and the circumstances of their occurrence. Every incident is independently classified by two CSET annotators. Annotations are peer reviewed and finally randomly selected for quality control ahead of publication. Despite this rigorous process, mistakes do happen, and readers are invited to report any errors they might discover while browsing.

Taxonomy Fields

Overall severity of harm Searchable in Discover App

Definition: An estimate of the overall severity of harm caused. "Negligible" harm means minor inconvenience or expense, easily remedied. “Minor” harm means limited damage to property, social stability, the political system, or civil liberties occurred or nearly occurred. "Moderate" harm means that humans were injured (but not killed) or nearly injured, or that financial, property, social, or political interests or civil liberties were materially affected (or nearly so affected). "Severe" harm means that a small number of humans were or were almost gravely injured or killed, or that financial, property, social, or political interests or civil liberties were significantly disrupted at at least a regional or national scale (or nearly so disrupted). "Critical" harm means that many humans were or were almost killed, or that financial, property, social, or political interests were seriously disrupted at a national or global scale (or nearly so disrupted).

Uneven distribution of harms basis Searchable in Discover App

Definition: If harms were unevenly distributed, this field indicates the basis or bases on which they were unevenly distributed.

Harm type Searchable in Discover App

Definition: Indicates the type(s) of harm caused or nearly caused by the incident.

System developer Searchable in Discover App

Definition: The entity that created the AI system.

Sector of deployment Searchable in Discover App

Definition: The primary economic sector in which the AI system(s) involved in the incident were operating.

Relevant AI functions Searchable in Discover App

Definition: Indicates whether the AI system(s) were intended to perform any of the following high-level functions: "Perception," i.e. sensing and understanding the environment; "Cognition," i.e. making decisions; or "Action," i.e. carrying out decisions through physical or digital means.

AI tools and techniques used Searchable in Discover App

Definition: Open-ended tags that indicate the hardware and software involved in the AI system(s).

AI functions and applications used Searchable in Discover App

Definition: Open-ended tags that describe the functions and applications of the AI system.

Location Searchable in Discover App

Definition: The location or locations where the incident played out.

Named entities Searchable in Discover App

Definition: All named entities (such as people, organizations, locations, and products - generally proper nouns) that seem to have a significant relationship with this event, as indicated by the available evidence.

Party responsible for AI system Searchable in Discover App

Definition: A list of parties (up to three) that were responsible for the relevant AI tool or system, i.e. that had operational control over the AI-related system causing harm (or control over those who did).

Harm nearly missed? Searchable in Discover App

Definition: Was harm caused, or was it a near miss?

Probable level of intent Searchable in Discover App

Definition: Indicates whether the incident was deliberate/expected or accidental, based on the available evidence. "Deliberate or expected" applies if it is established or highly likely that the system acted more or less as expected, from the perspective of at least one of the people or entities responsible for it. “Accident” applies if it is established or highly likely that the harm arose from the system acting in an unexpected way. "Unclear" applies if the evidence is contradictory or too thin to apply either of the above labels.

Human lives lost Searchable in Discover App

Definition: Marked "trur" if one or more people died as a result of the accident, "false" if there is no evidence of lives being lost, "unclear" otherwise.

Critical infrastructure sectors affected Searchable in Discover App

Definition: Where applicable, this field indicates if the incident caused harm to any of the economic sectors designated by the U.S. government as critical infrastructure.

Public sector deployment Searchable in Discover App

Definition: "Yes" if the AI system(s) involved in the accident were being used by the public sector or for the administration of public goods (for example, public transportation). "No" if the system(s) were being used in the private sector or for commercial purposes (for example, a ride-sharing company), on the other.

Nature of end user Searchable in Discover App

Definition: "Expert" if users with special training or technical expertise were the ones meant to benefit from the AI system(s)’ operation; "Amateur" if the AI systems were primarily meant to benefit the general public or untrained users.

Level of autonomy Searchable in Discover App

Definition: The degree to which the AI system(s) functions independently from human intervention. "High" means there is no human involved in the system action execution; "Medium" means the system generates a decision and a human oversees the resulting action; "low" means the system generates decision-support output and a human makes a decision and executes an action.

Physical system Searchable in Discover App

Definition: Where relevant, indicates whether the AI system(s) was embedded into or tightly associated with specific types of hardware.

Causative factors within AI system Searchable in Discover App

Definition: Indicates which, if any, of the following types of AI failure describe the incident: "Specification," i.e. the system's behavior did not align with the true intentions of its designer, operator, etc; "Robustness," i.e. the system operated unsafely because of features or changes in its environment, or in the inputs the system received; "Assurance," i.e. the system could not be adequately monitored or controlled during operation.

Full description of the incident

Definition: A plain-language description of the incident in one paragraph or less.

Short description of the incident

Definition: A one-sentence description of the incident.

Description of AI system involved

Definition: A brief description of the AI system(s) involved in the incident, including the system’s intended function, the context in which it was deployed, and any available details about the algorithms, hardware, and training data involved in the system.

Beginning date

Definition: The date the incident began.

Ending date

Definition: The date the incident ended.

Total financial cost

Definition: The stated or estimated financial cost of the incident, if reported.

Laws covering the incident

Definition: Relevant laws under which entities involved in the incident may face legal liability as a result of the incident.

Description of the data inputs to the AI systems

Definition: A brief description of the data that the AI system(s) used or were trained on.