Citation record for Incident 265

Suggested citation format

Atherton, Daniel. (2021-04-01) Incident Number 265. in Lam, K. (ed.) Artificial Intelligence Incident Database. Responsible AI Collaborative.

Incident Stats

Incident ID
Report Count
Incident Date
Editors
265
2
2021-04-01
Khoa Lam

Incidents Reports

A former Uber Eats courier has brought legal action against the food delivery company, alleging he was unfairly dismissed because of the company’s “racist” facial recognition software.

Uber Eats drivers are required to take a selfie before starting a shift to verify their identity. However, Pa Edrissa Manjang said that the Uber Eats app told him to take multiple selfies a day because the software incorrectly thought he was someone else.

“Your algorithm, by the looks of things, is racist,” said Manjang, who is Black.

On Wednesday an employment tribunal rejected Uber’s motion to dismiss the discrimination claim, which means the case will move to a full hearing.

An Uber spokesperson told _UKTN _that “automated facial verification was not the reason for Mr Manjang’s temporary loss of access to his courier account”.

The spokesperson added that Uber Eats’ Real-Time ID Check is used for safety and security purposes.

“The system includes robust human review to make sure that we’re not making decisions about someone’s livelihood in a vacuum, without oversight,” the spokesperson said.

Manjang told the Guardian that Uber doesn’t “value” or “respect” its delivery drivers and that it acts in an “aggressive manner”.

“So, as a result, for the people that work for them, we are just numbers,” said Manjang.

AI bias

It’s not the first time that Uber has faced accusations that its facial recognition software is racially biased. In October last year, Abiodun Ogunyemi filed an employment tribunal case after Uber’s automated face-scanning software failed to recognise him, leading to his dismissal.

AI experts say that facial recognition systems generally are not as accurate at identifying people from ethnic minority backgrounds. They say that this is often because the data used to train the algorithm tends to skew towards white men, which makes the software more accurate at identifying people from that demographic compared to others.

Uber is not the only company facing legal issues over the use of facial recognition technology. Yesterday, Big Brother Watch filed a complaint against Southern Co-op over the use of the tech in 35 of its stores.

Clearview AI was fined £7.5m by the Information Commissioner’s Office (ICO) earlier this year for illegally collecting billions of images. The ICO is currently investigating if AI has a racial bias.

It comes at a time when the use of digital identification applications are gradually increasing across the country. Some cinemas have begun accepting digital IDs as a method of proving age.

Courier sues Uber Eats over 'racist' facial recognition dismissal

A delivery driver who is suing Uber Eats in London over his dismissal from the company and claims its facial recognition technology is racially biased says the company treats couriers as “numbers rather than humans”.

Pa Edrissa Manjang worked for Uber Eats between November 2019 and April 2021 while employed full-time as a financial assistant.

When Manjang first began working for the company he was not regularly asked to send in pictures of himself for verification purposes. However, these facial verification checks became more frequent.

Manjang was eventually dismissed from the company by email, when it claimed there were “continued mismatches” between the pictures he took to register for a shift and the one on his Uber work profile.

When Manjang suggested Uber ask a human being to review the photos, he was told that after “careful consideration” his account was being deactivated.

Speaking of his time working for Uber, Manjang said the experience “wasn’t good”. He said: “Anyone can tell you that working for the company isn’t good. They don’t respect or value the people they work with.

“They don’t value us. I know that a company exists to make profits, but [Uber] proceeds in an aggressive manner. So, as a result, for the people that work for them, we are just numbers.”

Manjang’s claim, supported by the App Drivers and Couriers Union (ADCU), is that he was dismissed on the back of racially biased facial recognition technology and, while working for Uber Eats, was targeted for heightened and excessive facial recognition verification checks that amounted to racial harassment.

On Wednesday, the east London employment tribunal refused Uber’s application to have a discrimination claim struck out. In the ruling, the judge said: “At the point of presentation there was a lack of clarity about what the processes were that had led to the deactivation. These were only brought to light in the grounds of resistance.”

Sign up to First Edition, our free daily newsletter – every weekday morning at 7am BST

Manjang says that although bringing legal action against the company has not been easy, he is hopeful he will win his claim. He said: “[The legal action] has been time-consuming and sometimes stressful. I’m lucky that my union is supporting me, but even then it has been difficult.

“But yes, I’m confident [about the claim].”

Yaseen Aslam, the president of ADCU, said Uber’s dismissal of Manjang, allegedly based on racially biased facial recognition software, was “offensive”.

Aslam added: “Despite these proceedings and a separate lawful data-access request, Uber has failed to come clean about the basic facts about why Pa Edrissa Manjang was dismissed before Uber surreptitiously reinstated him after the case was filed.

“Uber has proven to be irresponsible in the use of sensitive technology known to have inherent racial bias because not only was Pa misidentified by the technology but he was profiled for heightened and excessive checks because of his ethnicity. This is immoral and unacceptable.”

Facial recognition software has been proved to be controversial due to the possibility of leading to bias. Microsoft recently announced it would stop using some of its facial recognition AI tools.

A spokesperson for Uber Eats said: “Automated facial verification was not the reason for Mr Manjang’s temporary loss of access to his courier account. Our Real-Time ID Check is designed to protect the safety and security of everyone who uses the Uber app. The system includes robust human review to make sure that we’re not making decisions about someone’s livelihood in a vacuum, without oversight.”

Uber Eats treats drivers as ‘numbers not humans’, says dismissed UK courier