Conversational AI Can Propel Social Stereotypes

As AI gets more talkative, designers need to consider the ethics of gendering not just voices but the finer nuances of speech patterns.
Men and Women symbols with robot heads
AI designers need to ensure they’re creating a more expansive world, and not replicating a close-mindedly gendered one. Illustration: Elena Lacey

Alexa, Siri, Watson, and their talking AI siblings serve to make our lives easier, but they also reinforce gender stereotypes. Polite, subservient digital secretaries like Alexa and Siri are presented as female. Assertive, all-knowing Jeopardy! champion Watson is most often referred to as “he.” New generations of AI are coming that will make this problem more significant, and much harder to avoid. As the field expands, designers need to ensure they’re creating a more expansive world, and not replicating a close-mindedly gendered one. Linguists can help them get there.

Last summer, UNESCO released a report warning against the “troubling repercussions” of gendered AI. The researchers recommended closer scrutiny of why many current speech-based AI systems, which interact with millions of people around the world, often default to speaking with a female voice, even though they may claim to be genderless. While any effort to explore and address the issue of AI and gender should be applauded, the report’s authors and others have missed a crucial point: It’s not just a matter of changing pronouns or vocal characteristics. To seriously attack the issue of gender stereotyping in AI, attention to a lot more than just the system’s voice is needed.

Today systems are moving from AI labs to industrial products that are conversational, far beyond the question-and-answer format of our pocket assistants. These new “social machines” will increasingly be able to become partners in multiperson, multimedia decision-making interactions. For example, rather than answering a single user’s query for the nearest Chinese restaurant, a conversational AI agent in the not-too-distant future will be able to engage with a group of people to help them choose where to go out to eat. Such an AI will participate as a member of the group: “Well if Bob and Bill want Chinese, and Mary likes Thai, why not the fusion place down the street?” it might say. Or it may even jump in more brashly: “OK, then let’s go to the fusion place.”

It is a given in linguistics that speech patterns in conversation invoke gender assumptions regardless of the speaker’s voice or appearance. For example, in standard American culture, men are described in the literature as more often “taking up space” in conversation: They interrupt more often, use more words, eschew some social politenesses, and speak with more evident certainty. Women, on the other hand, stereotypically speak less and more politely, give more affirmations and signs of listening, and suggest rather than dictate. In addition, tone, speed, word choice, and other small changes can change a participant’s perception of the speaker.

Where some have tried to address the issue by creating systems with genderless digital voices, they still miss a critical feature. Even in a voiceless chatbot, a user may attribute male or female gender based on these conversational features. In the previous restaurant example, the first suggestion would likely be seen as polite and female, while the latter assertion would typically be seen as male. Recent studies also show that these cues can outweigh whether a voice sounds stereotypically male or female and even contradict the direct assertions of a speaker, whether human or machine, with respect to their own identity. In AI terms, the fact that Siri replies “I don’t have a gender” has not changed the fact that people overwhelmingly conceive the program to be female.

Designers need to pay more attention to the ethical issues that emerge from these considerations. If new AIs continue to fall into current gender role stereotypes, then the stereotype of the passive and submissive woman versus the knowledgeable leader/expert man will be furthered. But designers could also be powerful agents of change, not just in our culture but especially in developing nations where the subjugated status of women is a growing international concern. Imagine the impacts of a business or medical adviser AI that presents as female and assistant companion AIs with default male speaking styles. More female-perceived AIs in expert roles could help evolve society’s perception and lead to women being more accepted in such positions.

Another future potential is to break away from the binary gender dichotomy altogether. A growing percentage of the world’s population does not identify as male or female, falling into categories that are just starting to be better recognized in mainstream society. This not only includes transgender individuals but also the large subpopulation that does not identify with a binary gender at all. For these marginalized groups, which for example have extremely high suicide rates, such AI systems could have a major impact. They could not only popularize the usage of the gender neutral they/them singular pronoun but also reflect the speech patterns of this community. As linguistic studies into nonbinary speech are only now emerging, AI designers partnering with linguistic researchers could benefit this community as well. For non-binary individuals, recognizing their way of speaking in AI role models would be invaluable.

The role of these conversational AIs in social change can also have impacts far beyond the gender binary. For example, AIs displaying traditionally African American speech styles could be important as role models for underrepresented minorities in fields ranging from online education to high-end scientific AI systems. There’s even a role for increased acceptance of immigrants that would result, for example, from North Americans interacting with more Latino voices or Europeans exposed to expertise in the form of Middle Eastern speech patterns.

AI researchers and educators are increasingly coming to terms with the need for understanding and teaching about the ethical impacts of choices made in AI’s data selection, analytic techniques, and deployment scenarios. However, if ethical AI is really going to change the thinking in the field, and to have a positive impact on the world, the more subtle impacts of choices in voice, speech patterns, and conversational styles must also be considered.


WIRED Opinion publishes articles by outside contributors representing a wide range of viewpoints. Read more opinions here. Submit an op-ed at opinion@wired.com.


More Great WIRED Stories