Stakeholder input significantly enhances the validity of STEM attitude and persistence instruments.
Category: User-Centred Design · Effect: Strong effect · Year: 2018
Incorporating diverse stakeholder perspectives through concept mapping leads to more robust and representative instruments for measuring attitudes and persistence in STEM fields.
Design Takeaway
When designing tools to measure user attitudes or behaviors, actively involve a diverse group of stakeholders in the conceptualization and validation phases to ensure the tool's relevance and accuracy.
Why It Matters
Designing effective interventions or educational programs in STEM requires accurate measurement of student attitudes and persistence. By involving a wide range of stakeholders, designers can ensure that the tools used for assessment truly reflect the multifaceted nature of these constructs, leading to more targeted and impactful design solutions.
Key Finding
The study demonstrated that involving a wide array of stakeholders in the development process, using concept mapping, resulted in a more comprehensive and validated instrument for assessing attitudes and persistence in STEM.
Key Findings
- Concept mapping effectively organized stakeholder conceptualizations of attitude and persistence towards STEM.
- A participatory framework involving diverse stakeholders enhanced the development of a valid instrument (APT-STEM).
Research Evidence
Aim: How can a participatory concept mapping approach be used to develop and validate an instrument measuring attitudes and persistence towards STEM fields?
Method: Mixed Methods (Concept Mapping, Psychometric Validation)
Procedure: A concept mapping methodology was employed with a broad range of STEM stakeholders to collaboratively develop an item pool for measuring attitudes and persistence. This pool was then rigorously analyzed using qualitative and quantitative psychometric techniques, including Classical Test Theory (CTT), Item Response Theory (IRT), and Rasch Measurement Theory (RMT), to validate the resulting instrument.
Sample Size: 190 STEM stakeholders
Context: STEM education and workforce development
Design Principle
Participatory instrument development, informed by diverse stakeholder conceptualizations, yields more valid and reliable measures.
How to Apply
When developing surveys, questionnaires, or assessment tools related to user attitudes, behaviors, or experiences, initiate a concept mapping exercise with representatives from all relevant user groups and expert communities.
Limitations
The study focused on a specific set of STEM stakeholders; generalizability to other fields or stakeholder groups may vary. The concept mapping process itself can be resource-intensive.
Student Guide (IB Design Technology)
Simple Explanation: To make sure a survey or test about liking and sticking with science and math subjects is good, it's best to ask lots of different people involved (like students, teachers, and people working in science jobs) what they think and use their ideas to build and check the test.
Why This Matters: This research highlights how involving a variety of people in the design process leads to better tools for understanding user attitudes, which is crucial for designing effective products or services.
Critical Thinking: To what extent can the findings regarding STEM attitudes and persistence be generalized to other fields or non-cognitive skills?
IA-Ready Paragraph: The development of robust measurement instruments, such as those used to assess user attitudes or behaviors, benefits significantly from a participatory approach. As demonstrated by Sunny (2018), employing methods like concept mapping with diverse stakeholder groups can lead to a more comprehensive conceptual domain and enhanced instrument validity, ensuring that the tool accurately reflects the intended constructs.
Project Tips
- When developing a research instrument, clearly define the stakeholder groups relevant to your design project.
- Consider using visual tools like concept mapping to synthesize diverse stakeholder input.
How to Use in IA
- Reference this study when explaining the importance of stakeholder consultation in the development of research instruments for your design project.
Examiner Tips
- Demonstrate an understanding of how diverse perspectives can inform the development and validation of design tools.
Independent Variable: Stakeholder participation and concept mapping methodology
Dependent Variable: Validity and representativeness of the APT-STEM instrument
Controlled Variables: Psychometric validation frameworks (CTT, IRT, RMT)
Strengths
- Utilized a mixed-methods approach for comprehensive data collection and analysis.
- Involved a broad and diverse range of STEM stakeholders.
Critical Questions
- How might the specific composition of the stakeholder group influence the resulting conceptualization?
- What are the trade-offs between depth of stakeholder input and the practicality of instrument development?
Extended Essay Application
- This study provides a framework for developing and validating instruments in any field where understanding user attitudes and persistence is critical, such as in the design of educational technologies or engagement platforms.
Source
Stakeholders' Conceptualization of Students' Attitudes and Persistence towards STEM: A Mixed Methods Instrument Development and Validation Study. · OhioLink ETD Center (Ohio Library and Information Network) · 2018