AI-enhanced Assistive Technologies: Eliciting User Privacy Perspectives Requires More Than Just Reflection

Why It Matters

As AI becomes more integrated into assistive technologies, understanding user privacy concerns is paramount. This research highlights a gap between cognitive understanding of privacy risks and genuine emotional empathy, which is crucial for designing truly inclusive and ethical products.

Key Finding

While a participatory toolkit helped design students think about privacy issues with AI-powered assistive tools, it didn't make them feel more empathetic towards users, with some students underestimating the risks faced by older adults using these technologies.

Key Findings

• The participatory toolkit was effective in eliciting reflection on privacy risks associated with AI-enhanced tools.
• Students who used the toolkit were less concerned about the privacy risks faced by AAT users compared to their own perceived risks, often assuming more data leads to better technology.
• The toolkit demonstrated value in prompting reflection but did not successfully cultivate empathy towards AAT users.

Research Evidence

Aim: Can participatory elicitation toolkits effectively foster empathy and self-reflection among design students regarding the privacy perspectives of users of AI-enhanced adaptive assistive technologies (AATs)?

Method: Qualitative study using a participatory elicitation toolkit.

Procedure: Graduate students used a toolkit to reflect on end-user privacy perspectives related to AI-enhanced tools, specifically focusing on their own learning tools and older adults using AI-enhanced AATs. The study incorporated intersectional thinking into privacy elicitation.

Context: Design education for AI-enhanced adaptive assistive technologies.

Design Principle

Empathy cultivation is a distinct and necessary component of user-centered design for AI-driven technologies, requiring targeted interventions beyond information elicitation.

How to Apply

Incorporate role-playing, user interviews with a strong focus on emotional impact, and scenario-based exercises that require designers to step into the user's shoes to build genuine empathy.

Limitations

The study focused on graduate students and may not generalize to all design students or professionals. The specific design of the elicitation toolkit could influence outcomes.

Student Guide (IB Design Technology)

Simple Explanation: Just thinking about how users feel isn't enough; designers need to actually *feel* for them to create good, safe technology, especially with AI.

Why This Matters: Understanding user privacy and developing empathy are critical for creating responsible and ethical technology, especially as AI becomes more prevalent in assistive devices.

Critical Thinking: To what extent can empathy be 'taught' or 'fostered' through design tools, and what are the ethical implications if it cannot be fully achieved?

IA-Ready Paragraph: This research highlights that while participatory toolkits can facilitate reflection on user privacy with AI-enhanced technologies, they do not inherently foster empathy. Designers must actively implement strategies to cultivate genuine empathy, recognizing that assumptions about data and technology benefits can lead to underestimation of user risks, particularly for vulnerable populations.

Project Tips

• When researching user needs for AI-powered products, don't just ask about problems; explore the emotional impact and potential anxieties.
• Consider how your own identity and experiences might shape your perception of user risks, and actively seek to understand different perspectives.

How to Use in IA

• Use this research to justify the need for specific empathy-building activities in your design process, beyond standard user research methods.

Examiner Tips

• Ensure your design process includes explicit steps to develop empathy, not just gather information about user needs and privacy concerns.

Independent Variable: Use of a participatory elicitation toolkit.

Dependent Variable: Level of empathy and self-reflection regarding user privacy.

Controlled Variables: Participant's background (graduate students), type of AI-enhanced technology discussed (AATs).

Strengths

• Addresses a critical and emerging area of AI and assistive technology design.
• Highlights the distinction between cognitive reflection and emotional empathy.

Critical Questions

• What specific methods are most effective in fostering empathy for vulnerable user groups in AI design?
• How can we measure the 'success' of empathy-building interventions in design education?

Extended Essay Application

• An Extended Essay could explore the development and testing of novel empathy-building techniques for designers working with AI and marginalized communities, using this paper as a theoretical foundation for the limitations of current methods.

Source

Elicitation and Empathy with AI-enhanced Adaptive Assistive Technologies (AATs) · Journal of Problem Based Learning in Higher Education · 2023 · 10.54337/ojs.jpblhe.v11i2.7667