Interactive Machine Teaching Enables Rapid Prototyping of Tangible Augmented Reality Experiences

Why It Matters

This approach democratizes AR development by enabling designers and researchers to quickly iterate on tangible AR concepts using familiar objects. It allows for more intuitive and context-aware interactions, moving beyond screen-based interfaces to richer, object-driven experiences.

Key Finding

The Teachable Reality system successfully enables users to build interactive AR prototypes by teaching it to recognize interactions with ordinary objects, making AR prototyping more accessible and versatile.

Key Findings

• Teachable Reality allows users to create functional AR prototypes using everyday objects without programming.
• The approach offers flexibility and generalizability for tangible AR applications.
• User studies and expert interviews confirmed a lowered barrier to AR prototyping and a flexible, general-purpose prototyping experience.

Research Evidence

Aim: Can interactive machine teaching be effectively used to prototype tangible augmented reality applications with everyday objects, bypassing the need for traditional programming?

Method: User study and expert interviews

Procedure: The researchers developed a tool called 'Teachable Reality' that uses vision-based interactive machine teaching to capture real-world interactions with everyday objects. Users could then define tangible and gestural interactions via a trigger-action interface to create functional AR prototypes. The effectiveness and usability of this tool were evaluated through user studies and expert interviews.

Context: Augmented Reality (AR) prototyping, Human-Computer Interaction (HCI)

Design Principle

Democratize complex technology prototyping by abstracting technical barriers and leveraging familiar interaction paradigms.

How to Apply

Use readily available objects (e.g., cups, books, hands) and a tool like Teachable Machine to train a system to recognize specific gestures or object states, then link these to AR outputs or behaviors for rapid prototyping.

Limitations

The effectiveness may depend on the quality of the object recognition model and the complexity of the desired interactions. Generalizability to all object types and interaction complexities may vary.

Student Guide (IB Design Technology)

Simple Explanation: You can make augmented reality (AR) prototypes that use real-world objects by 'teaching' a computer program to recognize how you interact with them, without needing to code.

Why This Matters: This research shows a way to create interactive AR experiences that feel more real and are easier to build, which can be very useful for design projects that involve new ways for people to interact with technology.

Critical Thinking: How might the accuracy and robustness of the machine teaching model impact the user experience and reliability of the AR prototype?

IA-Ready Paragraph: The research by Monteiro et al. (2023) on 'Teachable Reality' demonstrates a significant advancement in AR prototyping by utilizing interactive machine teaching. This approach allows for the creation of tangible AR applications using everyday objects without requiring programming expertise. By enabling designers to quickly train systems to recognize object-based interactions, it significantly lowers the barrier to entry for developing novel and intuitive AR experiences, offering a flexible and generalizable method for rapid prototyping.

Project Tips

• Consider using everyday objects as input for your AR prototypes.
• Explore tools that allow for 'machine teaching' to recognize user interactions.
• Focus on creating tangible interactions that feel natural and intuitive.

How to Use in IA

• Reference this study when discussing the prototyping of interactive AR systems, especially when using non-traditional inputs or aiming for rapid iteration.

Examiner Tips

• Demonstrate an understanding of how interactive machine learning can simplify complex prototyping tasks.
• Critically evaluate the limitations of object recognition and interaction mapping in your own design project.

Independent Variable: Use of interactive machine teaching for AR prototyping.

Dependent Variable: Ease of prototyping, flexibility of application, functionality of AR prototypes.

Controlled Variables: Type of everyday objects used, complexity of interactions programmed, AR display technology.

Strengths

• Novel approach to AR prototyping.
• Addresses the need for accessible AR development tools.
• Demonstrates practical application with diverse examples.

Critical Questions

• What are the ethical implications of using everyday objects in interactive AR systems?
• How does this method scale for professional AR development teams versus individual designers?

Extended Essay Application

• An Extended Essay could explore the development of a tangible AR interface for educational purposes, using Teachable Reality principles to allow students to interact with virtual models using physical objects.
• Investigate the user experience differences between marker-based AR and object-taught AR for a specific application domain.

Source

Teachable Reality: Prototyping Tangible Augmented Reality with Everyday Objects by Leveraging Interactive Machine Teaching · 2023 · 10.1145/3544548.3581449