Why does Artificial Intelligence (AI) pick up biases?

Ver texto associado

AI Picks Up Racial and Gender Biases When Learning from What Humans Write

AI1 picks up racial and gender biases2 when learning language from text, researchers say. Without any supervision, a machine learning algorithm learns to associate female names more with family words than career words, and black names as being more unpleasant than white names.

For a study published today in Science, researchers tested the bias of a common AI model, and then matched the results against a well-known psychological test that measures bias in humans. The team replicated in the algorithm all the psychological biases they tested, according to a study from co-author Aylin Caliskan, a post-doc at Princeton University. Because machine learning algorithms are so common, influencing everything from translation to scanning names on resumes, this research shows that the biases are pervasive, too. 

An algorithm is a set of instructions that humans write to help computers learn. Think of it like a recipe, says Zachary Lipton, an AI researcher at UC San Diego who was not involved in the study. Because algorithms use existing materials — like books or text on the internet — it’s obvious that AI can pick up biases if the materials themselves are biased. (For example, Google Photos tagged black users as gorillas.) We’ve known for a while, for instance, that language algorithms learn to associate the word “man” with “professor” and the word “woman” with “assistant professor.” But this paper is interesting because it incorporates previous work done in psychology on human biases, Lipton says.

For today’s study, Caliskan’s team created a test that resembles the Implicit Association Test (IAT), which is commonly used in psychology to measure how biased people are (though there has been some controversy over its accuracy). In the IAT, subjects are presented with two images — say, a white man and a black man — and words like “pleasant” or “unpleasant.” The IAT calculates how quickly you match up “white man” and “pleasant” versus “black man” and “pleasant,” and vice versa. The idea is that the longer it takes you to match up two concepts, the more trouble you have associating them.

The test developed by the researchers also calculates bias, but instead of measuring “response time”, it measures the mathematical distance between two words. In other words, if there’s a bigger numerical distance between a black name and the concept of “pleasant” than a white name and “pleasant”, the model’s association between the two isn’t as strong. The further apart the words are, the less the algorithm associates them together.

Caliskan’s team then tested their method on one particular algorithm: Global Vectors for Word Representation (GLoVe) from Stanford University. GLoVe basically crawls the web to find data and learns associations between billions of words. The researchers found that, in GLoVe, female words are more associated with arts than with math or science, and black names are seen as more unpleasant than white names. That doesn’t mean there’s anything wrong with the AI system, per se, or how the AI is learning — there’s something wrong with the material.

1AI: Artificial Intelligence

2bias: prejudice; preconception

Disponível em <http://www.theverge.com/>. Acesso em: 18/04/2017.