User Testing and Heuristic Evaluation Dominate HIS Usability Assessments

Why It Matters

Understanding the most common usability evaluation methods provides a benchmark for design practice. By knowing which techniques are frequently employed, designers can better anticipate industry standards and focus their efforts on refining these established approaches or exploring complementary methods.

Key Finding

The review found that user testing and heuristic evaluation are the go-to methods for evaluating hospital information system usability, often employing standardized questionnaires and checklists. Key areas of focus include user satisfaction and system efficiency.

Key Findings

• User testing (UT) and heuristic evaluation (HE) were the most frequently applied usability evaluation methods, each appearing in 34% of the reviewed studies.
• Standardized questionnaires (40%) and checklists (36%) were the most common tools, with SUS and Nielsen's heuristics being predominant.
• User satisfaction and efficiency were the most frequently assessed usability criteria.

Research Evidence

Aim: What are the most common usability evaluation methods and tools used for hospital information systems?

Method: Systematic Review

Procedure: A systematic search of multiple databases was conducted to identify studies evaluating the usability of hospital information systems. Data on evaluation methods, tools, usability criteria, and system subsystems were extracted and analyzed. Study quality was assessed using a modified Downs & Black checklist.

Sample Size: 41 studies

Context: Healthcare Informatics / Hospital Information Systems

Design Principle

Employ established and validated usability evaluation methods to ensure robust assessment of user interaction with complex systems.

How to Apply

In your next design project involving a complex software system, plan to incorporate both user testing and heuristic evaluation. Use the System Usability Scale (SUS) for quantitative feedback and Nielsen's 10 Usability Heuristics for expert review.

Limitations

The review primarily focused on studies published between 2016 and 2020, and a significant portion of the research originated from low- and middle-income countries, potentially limiting the generalizability of findings to all global contexts.

Student Guide (IB Design Technology)

Simple Explanation: When you need to check if a hospital computer system is easy to use, most people use two main methods: watching users try it out (user testing) and having experts look for common problems (heuristic evaluation).

Why This Matters: Understanding common evaluation methods helps you choose the most effective ways to test your designs and gather meaningful feedback for improvement.

Critical Thinking: While user testing and heuristic evaluation are prevalent, are there emerging or underutilized methods that could offer more comprehensive insights into HIS usability?

IA-Ready Paragraph: This systematic review highlights that user testing and heuristic evaluation are the most frequently employed methods for assessing the usability of hospital information systems. These methods, often supported by standardized questionnaires like the System Usability Scale (SUS) and established heuristic sets, provide valuable insights into user satisfaction, efficiency, and error reduction, informing iterative design improvements.

Project Tips

• When choosing evaluation methods for your design project, consider the nature of your product and the resources available.
• Document your chosen evaluation methods thoroughly, explaining why they are appropriate for your specific design.

How to Use in IA

• Reference this review when justifying the choice of user testing and heuristic evaluation as your primary usability assessment methods in your design project report.

Examiner Tips

• Demonstrate an understanding of why specific usability evaluation methods are chosen over others for a given design context.

Independent Variable: Usability evaluation methods (e.g., User Testing, Heuristic Evaluation)

Dependent Variable: Frequency of use, effectiveness in assessing usability criteria (satisfaction, efficiency, etc.)

Controlled Variables: Type of HIS subsystem evaluated, geographic region of study, publication year range

Strengths

• Comprehensive search strategy across multiple databases.
• Adherence to PRISMA 2020 guidelines for systematic reviews.

Critical Questions

• How might the specific context of healthcare (e.g., high-stakes environment, diverse user groups) influence the choice and effectiveness of usability evaluation methods?
• What are the potential biases introduced by the geographical distribution of the reviewed studies?

Extended Essay Application

• An Extended Essay could explore the comparative effectiveness of user testing versus heuristic evaluation for a specific type of digital product, drawing on the methodologies identified in this review.

Source

Usability evaluation methods for hospital information systems: a systematic review · BMC Health Services Research · 2025 · 10.1186/s12913-025-13745-y