Associated Incidents
Image via Twitter
Parents know one particular challenge of raising kids all too well: teaching them to do what we say, not what we do.
A similar challenge has hit artificial intelligence.
As more apps and software use AI to automate tasks, a popular data-backed model, called "word embedding," has also picked up entrenched social biases.
The result is services like language translation spitting those biases back out in subtle but worrisome ways.
Earlier this year, for instance, examples of gender bias started cropping up on social media with Google Translate.
Try translating terms into English from Turkish, which has gender-neutral pronouns, and a phrase like o bir muhendis becomes he is an engineer, while o bir hemsire translates to she is a nurse.
Microsoft’s own translation service on Bing has a similar problem, thanks to the use of gender in grammar - think le and la in French, or der and die in German.
When Bing translates “the table is soft” into German, it offers the feminine die Tabelle, which refers to a table of figures.
“These gender associations are projected onto objects,” says Kate McCurdy, a senior computational linguist with the language-learning startup Babbel in Berlin, Germany, who discovered the bias in Bing.
“Objects that happen to be grammatically masculine are given masculine properties. It's learning to take gender stereotypes and project them into the whole world of nouns.”
Because of word embedding, a popular method of machine learning, translation algorithms are working off these biases, and so are other services like Google Search, as well as Netflix and Spotify recommendations.
“This is an approach that has taken off and is extremely widespread in the industry, and that’s why it’s so important to interrogate the underlying assumption,” says McCurdy.
Word embedding works by linking words to a vector of numbers, which algorithms can use to calculate probability. By looking at what words tend to be around other words, like “engineer,” the model can be used to figure out what other word fits best, like “he.”
The price of learning from reams of existing text and dialogue is that such models pick up the true-to-life imbalance between genders when it comes to jobs or opportunities.
A 2016 study that trained word-embedding models on articles on Google showed gender stereotypes “to a disturbing extent,” according to its researchers.
McCurdy says that there isn’t anything necessarily wrong with the word-embedding model itself, but it needs human guidance and oversight.
“The default now is to build these applications and release them into the wild and fight the fires when they come out,” she adds. “But if we were more deliberate about this and took things more seriously, we’d do more work to integrate a more critical perspective.”
Companies who are using the word-embedding model to make services for consumers also need more diverse programmers who are more likely to notice the risk of biases before they crop up.
“If we’re serious about having artificial intelligence make decisions that don’t end up with biases that we don’t want to reinforce, we need to have more diverse and critical people looking at this earlier on in the process.”