However, bias also appears for other unrelated reasons. A recent study into how an algorithm delivered ads promoting STEM jobs showed that men were more likely to be shown the ad, not because men were more likely to click on it, but because women are more expensive to advertise to. Since companies price ads targeting women at a higher rate (women drive 70% to 80% of all consumer purchases), the algorithm chose to deliver ads more to men than to women because it was designed to optimise ad delivery while keeping costs low.
But if an algorithm only reflects patterns in the data we give it, what its users like, and the economic behaviours that occur in its market, isn’t it unfair to blame it for perpetuating our worst attributes? We automatically expect an algorithm to make decisions without any discrimination when this is rarely the case with humans. Even if an algorithm is biased, it may be an improvement over the current status quo.
To fully benefit from using AI, it’s important to investigate what would happen if we allowed AI to make decisions without human intervention. A 2018 study explored this scenario with bail decisions using an algorithm trained on historical criminal data to predict the likelihood of criminals re-offending. In one projection, the authors were able to reduce crime rates by 25% while reducing instances of discrimination in jailed inmates.
Yet the gains highlighted in this research would only occur if the algorithm was actually making every decision. This would be unlikely to happen in the real world as judges would probably prefer to choose whether or not to follow the algorithm’s recommendations. Even if an algorithm is well designed, it becomes redundant if people choose not to rely on it.
Many of us already rely on algorithms for many of our daily decisions, from what to watch on Netflix or buy from Amazon. But research shows that people lose confidence in algorithms faster than humans when they see them make a mistake, even when the algorithm performs better overall.
For example, if your GPS suggests you use an alternative route to avoid traffic that ends up taking longer than predicted, you’re likely to stop relying on your GPS in the future. But if taking the alternate route was your decision, it’s unlikely you will stop trusting your own judgement. A follow-up study on overcoming algorithm aversion even showed that people were more likely to use an algorithm and accept its errors if given the opportunity to modify the algorithm themselves, even if it meant making it perform imperfectly.
While humans might quickly lose trust in flawed algorithms, many of us tend to trust machines more if they have human features. According to research on self-driving cars, humans were more likely to trust the car and believed it would perform better if the vehicle’s augmented system had a name, a specified gender, and a human-sounding voice. However, if machines become very human-like, but not quite, people often find them creepy, which could affect their trust in them.
Even though we don’t necessarily appreciate the image that algorithms may reflect of our society, it seems that we are still keen to live with them and make them look and act like us. And if that’s the case, surely algorithms can make mistakes too?
Maude Lavanchy is Research Associate at IMD.
This article was first published by The Conversation.