Incident 63: Google Photo Merge Decapitates Subject

Description: Google Photos' AI Assistant created a strange hybrid photograph when merging three different pictures from a ski trip.
Alleged: Google developed and deployed an AI system, which harmed Alex Harker.

Suggested citation format

Yampolskiy, Roman. (2018-01-25) Incident Number 63. in McGregor, S. (ed.) Artificial Intelligence Incident Database. Responsible AI Collaborative.

Incident Stats

Incident ID
63
Report Count
1
Incident Date
2018-01-25
Editors
Sean McGregor

Tools

New ReportNew ReportDiscoverDiscover

CSET Taxonomy Classifications

Taxonomy Details

Full Description

According to Android user Alex Harker, the Google Photos AI assistant created a strange hybrid of three photos he took on a ski trip. Two of the photos were of a forest landscape, while one photo was of Harker's friend. Google Photos created a stranged merged photo showing the friend's head half-merged behind the trees.

Short Description

Google Photos' AI Assistant created a strange hybrid photograph when merging three different pictures from a ski trip.

Severity

Negligible

AI System Description

Facial recognition software used by Google Photo Assistant

System Developer

Google

Sector of Deployment

Arts, entertainment and recreation

Relevant AI functions

Perception, Cognition

AI Techniques

Facial recognition

AI Applications

Facial recognition

Location

Banff, Alberta

Named Entities

Google Photos, Assistant, Android, Alex Harker, Alberta

Technology Purveyor

Google

Beginning Date

01/2018

Ending Date

01/2018

Near Miss

Unclear/unknown

Intent

Accident

Lives Lost

No

Data Inputs

Photographs

Incidents Reports

Google Photos’ intrepid Assistant just wants to help.

Part of the Android-maker’s photo app, it helps organize the thousands of photos stored on your phone. It can make little albums of places you went based on geolocation data, and through facial recognition can even organize albums about your friends, family and pets through. If you take a sequence of similarly composed photos, it can turn those into a little animation.

Or… it might turn your friend into a giant head lurking in a forest.

That’s what happened to Alex Harker, who snapped a few quick pictures of scenery and friends while skiing in Banff, Alberta:

Courtesy of Alex Harker/Imgur The originals, friends still intact.

Once the images were on his phone, Google Photo’s Assistant, correctly thinking that these three frames belonged together, got to work—but not in the way a human might expect.

Harker explained on Reddit:

I literally took like 3 pictures, one with them in, and two without them. And for some bizarre reason Google Assistant offered me a really strange panorama of the 3 photos spliced together.

The resulting image features Harker’s friend Matt hiding behind the trees, while his other friend has vanished completely.

After he discovered the glitchy panorama on his, he shared it on Reddit, where it quickly shot up to the top of the r/funny subreddit and has garnered over 188,000 upvotes.

Courtesy of Alex Harker/Imgur What the panorama looked like in Harker’s Google Photos app.

Observers online have remarked that while the image makes absolutely no sense, Assistant did a pretty good job of reconciling the edges of photos shot at differing perspectives. While there still is a giant head lurking beyond the pines, the foreground is nicely smoothed and cohesive.

Steve Dent of Engadget writes:

If you study the image closely, you can’t help but be impressed by the algorithm’s attention to detail. It did a pretty darn good job of masking out the trees to the subject’s left and inserting him “behind” them. It has also cropped his upper body beautifully to follow the contour of the slope, making it look like he’s hiding in a ravine, ready to tell punk skiers to slow the hell down.

While Google Photos and other AI powered image software have been found to make far worse mistakes, this is a goofier illustration of the ways in which ways tech tries to organize our lives.

Google Photos tried to fix this ski photo

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents

Biased Google Image Results

· 18 reports

FaceApp Racial Filters

· 23 reports