Incident 44: Machine Personal Assistants Failed to Maintain Social Norms

Description: During an experiment of software personal assistants at the Information Sciences Institute (ISI) at the University of Southern California (USC), researchers found that the assistants violated the privacy of their principals and were unable to respect the social norms of the office.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History
Alleged: USC Information Sciences Institute developed and deployed an AI system, which harmed USC Information Sciences Institute.

Incident Stats

Incident ID
44
Report Count
1
Incident Date
2008-07-01
Editors
Sean McGregor

CSETv0 Taxonomy Classifications

Taxonomy Details

Full Description

From June to December 2000, researchers at the Information Sciences Institute (ISI) at the University of Southern California (USC) deployed a team of 12 software agents in their office to act as administrative assistants facilitating routine office operations. The agents, also known as Electronic Elves and nicknamed ‘Friday’, were designed to assist their human principal in scheduling meetings, facilitating informal meetings, auctioning group tasks, and ordering meals as a means to testing agent teamwork dynamics. During the experiment, the agents violated the privacy and social norms of the office by publishing their principal’s location information and revealing an employee value hierarchy used to deconflict meetings.

Short Description

During an experiment of software personal assistants at the Information Sciences Institute (ISI) at the University of Southern California (USC), researchers found that the assistants violated the privacy of their principals and were unable to respect the social norms of the office.

Severity

Negligible

Harm Type

Psychological harm

AI System Description

A team of software agents (agent organization) designed to facilate routine office operations, such as scheduling, informal collaboration, and meeting shared goals.

System Developer

University of Southern California Information Sciences Institute

Sector of Deployment

Administrative and support service activities

Relevant AI functions

Perception, Cognition

AI Techniques

Machine learning

AI Applications

decision support, resource optimization, personalization

Location

Marina del Rey, California

Named Entities

University of Southern California Information Sciences Institute

Technology Purveyor

University of Southern California Information Sciences Institute

Beginning Date

6/2000

Ending Date

12/2000

Near Miss

Near miss

Intent

Accident

Lives Lost

No

Data Inputs

Schedule data, cellphone GPS data

CSETv1 Taxonomy Classifications

Taxonomy Details

Harm Distribution Basis

none

Sector of Deployment

administrative and support service activities

Incident Reports

aaai.org · 2008

Abstract: Software personal assistants continue to be a topic of significant research interest. This article outlines some of the important lessons learned from a successfully deployed team of personal assistant agents (Electric Elves) in a…

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.