Voice Assistant Errors Amplify Racial Bias, Impacting User Psychology

Why It Matters

This research highlights a critical, yet often overlooked, aspect of user experience: the psychological impact of algorithmic bias. Designers must consider how system failures, particularly those with racial disparities, affect user emotions, self-perception, and trust in technology.

Key Finding

When voice assistants make more errors for Black users, these users experience increased self-consciousness, lower self-esteem, and more negative feelings towards the technology, unlike white users who do not show these effects.

Key Findings

• Black participants interacting with a high-error voice assistant reported increased self-consciousness.
• Black participants in the high-error condition showed lower self-esteem and less positive affect.
• Black participants rated the high-error voice assistant less favorably.
• White participants did not exhibit these disparate psychological responses across error rate conditions.

Research Evidence

Aim: To investigate whether speech recognition errors in voice assistants can elicit similar negative psychological effects as misunderstandings in cross-racial interpersonal communication.

Method: Controlled Experiment

Procedure: Participants (Black and white) were assigned to interact with a voice assistant programmed with either a high or low error rate. Psychological responses and technology ratings were then measured.

Sample Size: 108 participants

Context: Human-computer interaction, voice assistant technology, cross-racial communication.

Design Principle

Design for equitable performance: Ensure that technology functions reliably and without negative psychological impact for all user groups, regardless of race.

How to Apply

When developing or evaluating voice-enabled products, conduct rigorous testing with diverse user groups to identify and quantify performance differences in ASR. Implement bias mitigation strategies in ASR model training and deployment.

Limitations

The study focused on specific types of ASR errors and may not generalize to all forms of technological bias or all user demographics. The artificial nature of the programmed error rates might differ from real-world, naturally occurring errors.

Student Guide (IB Design Technology)

Simple Explanation: Voice assistants sometimes make more mistakes for Black users than for white users. This study found that when this happens, Black users feel more self-conscious, less confident, and dislike the technology more, while white users don't have the same negative feelings.

Why This Matters: This research shows that technology isn't always neutral. Design choices can unintentionally create unfair experiences and harm users' feelings and self-worth, especially for minority groups.

Critical Thinking: To what extent can designers be held responsible for the biases embedded within third-party AI components like ASR, and what ethical frameworks should guide their decision-making in such cases?

IA-Ready Paragraph: This research by Wenzel et al. (2023) demonstrates that disparities in voice assistant speech recognition accuracy can lead to significant negative psychological effects, such as increased self-consciousness and lower self-esteem, for Black users. This underscores the critical need for designers to ensure equitable performance across diverse user groups to avoid creating biased and harmful user experiences.

Project Tips

• When designing voice interfaces, consider how different accents or speech patterns might be recognized.
• Think about how technology failures could make users feel, especially if those failures are more common for certain groups.

How to Use in IA

• Reference this study when discussing the importance of user testing with diverse groups.
• Use it to justify the need for equitable performance in your design, not just functionality.

Examiner Tips

• Demonstrate an understanding of how algorithmic bias can have real-world psychological consequences.
• Show how you've considered diverse user needs and potential negative impacts in your design process.

Independent Variable: Error rate of the voice assistant (high vs. low), Participant race (Black vs. white).

Dependent Variable: Self-consciousness, Self-esteem, Positive affect, Technology ratings.

Controlled Variables: Type of voice assistant interaction, Pre-programmed error patterns.

Strengths

• Controlled experimental design allows for clear causal inference.
• Directly measures psychological outcomes, providing a deeper understanding of user impact.

Critical Questions

• How can designers proactively identify and mitigate potential racial biases in ASR systems during the design and development phases?
• What are the long-term implications of such biased interactions on user trust and adoption of voice technologies?

Extended Essay Application

• Investigate the psychological impact of AI bias in a specific domain (e.g., educational software, healthcare apps).
• Develop and test design interventions aimed at mitigating bias in AI-driven user interfaces.

Source

Can Voice Assistants Be Microaggressors? Cross-Race Psychological Responses to Failures of Automatic Speech Recognition · 2023 · 10.1145/3544548.3581357