Equity Gaps in Health Tech Usability Testing Identified

Why It Matters

This research highlights a critical blind spot in the design and development of health technologies. By not proactively addressing the needs and participation of diverse user populations, designers risk creating products that are not only less effective but potentially unsafe for significant segments of the population.

Key Finding

The study found that both heuristic evaluations and usability testing for health technologies have significant potential failure points related to equity, particularly concerning the inclusion and consideration of marginalized user groups.

Key Findings

• Failure modes in heuristic evaluation were related to evaluator expertise and a lack of heuristics addressing marginalized groups' needs.
• Failure modes in usability testing were predominantly related to challenges in recruiting users from marginalized groups.
• Current methods may not uncover usability problems affecting marginalized groups.

Research Evidence

Aim: To identify potential failure modes in heuristic evaluation and usability testing methods for health technologies that could lead to inequitable outcomes for users.

Method: Failure Modes and Effects Analysis (FMEA) through expert workshops.

Procedure: Ten usability practitioners participated in three workshops to identify potential failure modes in heuristic evaluation and usability testing methods for health technologies, focusing on equity implications.

Sample Size: 10 participants

Context: Health technology usability evaluations, specifically for regulated medical devices.

Design Principle

Usability evaluations must be designed with equity at their core to ensure the safety and effectiveness of technologies for all users.

How to Apply

When planning usability evaluations for health technologies, explicitly map out potential equity risks and develop mitigation strategies for participant recruitment and evaluation criteria.

Limitations

The study focused on practitioners from a single consulting firm, potentially limiting the generalizability of findings across all usability evaluation contexts.

Student Guide (IB Design Technology)

Simple Explanation: When testing health technology, we need to make sure we are testing with and for everyone, not just the most common users, otherwise, the technology might not work well or could even be dangerous for some people.

Why This Matters: Ensuring that health technologies are usable and safe for all individuals, including those from marginalized communities, is crucial for achieving equitable health outcomes.

Critical Thinking: How might the findings of this study apply to the usability of non-health-related technologies, and what specific adaptations would be necessary?

IA-Ready Paragraph: This research highlights critical equity risks within health technology usability evaluations, indicating that standard methods may inadvertently exclude or fail to identify issues for marginalized user groups. Incorporating a proactive approach to equity in user recruitment and evaluation criteria is essential for ensuring the safety and effectiveness of designed solutions for all.

Project Tips

• Consider the diversity of your target user group from the outset of your design project.
• When planning user testing, think about how you will recruit participants who represent different backgrounds and needs.

How to Use in IA

• Reference this study when discussing the importance of inclusive user research and the potential pitfalls of standard usability testing methods in your design project.

Examiner Tips

• Demonstrate an understanding of how design choices in evaluation methods can impact user equity and safety.

Independent Variable: Usability evaluation methods (heuristic evaluation, usability testing).

Dependent Variable: Identification of equity-related failure modes.

Controlled Variables: Expertise of usability practitioners, focus on health technologies.

Strengths

• Utilizes a recognized risk analysis technique (FMEA) for a novel application in usability.
• Involves experienced practitioners in the field.

Critical Questions

• What specific heuristics could be developed to better address the needs of marginalized groups in health technology?
• What are the ethical considerations when designing recruitment strategies for vulnerable or marginalized populations?

Extended Essay Application

• An Extended Essay could explore the development and validation of a new, equity-focused usability evaluation framework for a specific type of health technology.

Source

Equity risks in usability evaluations of health technologies: A failure modes and effects analysis · Human Factors in Healthcare · 2026 · 10.1016/j.hfh.2026.100124