Fuzzy AHP enhances e-learning system selection by incorporating stakeholder usability, maintainability, and deployability.

Why It Matters

This research highlights the importance of involving diverse stakeholders in the software selection process, particularly for educational technologies. By systematically assessing usability, maintainability, and deployability through a structured methodology, design teams can make more informed decisions that lead to better user adoption and system effectiveness.

Key Finding

A new evaluation technique, FOSSET, which uses Fuzzy AHP, was found to be effective in selecting e-learning systems by considering usability, maintainability, and deployability, and it simplifies the evaluation process by reducing the number of comparisons needed.

Key Findings

• The FOSSET technique is efficient and effective for evaluating e-learning systems.
• It reduces the number of pairwise comparisons needed, making it suitable for complex evaluations with many attributes.
• Stakeholder participation is crucial for selecting the preferred software product.

Research Evidence

Aim: How can a mixed-methods approach, integrating qualitative stakeholder input with a quantitative Fuzzy Analytical Hierarchy Process, be used to effectively evaluate and select free and open-source e-learning systems based on usability, maintainability, and deployability?

Method: Mixed-methods research (qualitative and quantitative)

Procedure: A Fuzzy Analytical Hierarchy Process (Fuzzy AHP) technique, named FOSSET, was developed and applied to evaluate two free and open-source e-learning systems. This involved gathering qualitative data through questionnaires and interviews with stakeholders, and quantitative data using the Fuzzy AHP algorithm to weigh and compare system attributes like usability, maintainability, and deployability.

Sample Size: 33 stakeholders

Context: Evaluation of e-learning systems in a developing country context (Open University of Tanzania)

Design Principle

System selection and design should be guided by a multi-criteria decision-making framework that incorporates diverse stakeholder perspectives and objective quality assessments.

How to Apply

When evaluating potential e-learning platforms for an organization, use a structured approach that includes user surveys and interviews, followed by a Fuzzy AHP analysis to rank options based on predefined criteria like ease of use, technical support, and integration capabilities.

Limitations

The study was conducted in a specific developing country context, which may limit generalizability to other regions or types of institutions. The effectiveness of the technique might vary with the complexity and number of attributes considered.

Student Guide (IB Design Technology)

Simple Explanation: This study shows that using a smart math method (Fuzzy AHP) along with asking users questions can help pick the best e-learning software by looking at how easy it is to use, fix, and set up.

Why This Matters: Understanding how to systematically evaluate software based on user needs is crucial for designing effective and adopted digital products.

Critical Thinking: To what extent can a purely quantitative approach, like Fuzzy AHP, fully capture the nuanced emotional and experiential aspects of user interaction with a design, and what are the potential drawbacks of relying heavily on such methods?

IA-Ready Paragraph: The selection of the final design concept was informed by a mixed-methods evaluation approach, drawing inspiration from Sanga's (2010) work on Fuzzy AHP for e-learning systems. This involved gathering qualitative feedback on usability and functional requirements from target users, which was then integrated into a quantitative analysis using a simplified multi-criteria decision-making framework to objectively compare design alternatives based on predefined criteria such as user experience, feasibility, and innovation potential.

Project Tips

• When evaluating different design options, consider using a multi-criteria decision-making tool.
• Ensure your evaluation criteria directly reflect user needs and system functionality.

How to Use in IA

• Use the Fuzzy AHP method to justify the selection of a particular design solution over others, referencing stakeholder feedback on usability and other key factors.

Examiner Tips

• Demonstrate an understanding of how to quantitatively assess qualitative user feedback.
• Clearly articulate the criteria used for evaluating design choices.

Independent Variable: E-learning system attributes (usability, maintainability, deployability)

Dependent Variable: Suitability/preference score for e-learning systems

Controlled Variables: Stakeholder group, evaluation context (Open University of Tanzania)

Strengths

• Integrates both qualitative and quantitative research methods.
• Proposes a novel technique (FOSSET) for software evaluation.
• Focuses on practical, user-relevant quality characteristics.

Critical Questions

• How might the weighting of criteria in Fuzzy AHP be influenced by the specific cultural or educational context of the users?
• What are the challenges in ensuring consistent and unbiased pairwise comparisons from a diverse group of stakeholders?

Extended Essay Application

• An Extended Essay could explore the application of Fuzzy AHP to evaluate different sustainable material choices for a product, incorporating expert opinions on environmental impact, cost, and performance.
• Investigate how Fuzzy AHP can be adapted to evaluate the user experience of different interface designs for a mobile application, considering factors like navigation ease, visual appeal, and task completion efficiency.

Source

A technique for the evaluation of free and open source e-learning systems · University of the Western Cape Electronic Theses and Dissertations Repository (University of the Western Cape) · 2010