Lay Post-Editing of Machine-Translated User-Generated Content is Feasible for Short Segments

Why It Matters

This research highlights an opportunity to leverage community expertise for content localization and quality assurance. By understanding the conditions under which lay post-editing is most effective, design teams can develop more efficient workflows for managing multilingual user-generated content, improving accessibility and user experience.

Key Finding

Community members can successfully edit machine translations, especially for shorter pieces of text, though the effort required can vary and isn't easily predicted by editor background.

Key Findings

• Lay post-editing is a statistically significant feasible concept for machine-translated user-generated content.
• Post-editing is successful for short segments requiring approximately 35% post-editing effort.
• No distinct post-editing patterns were identified for segments requiring more effort.
• Post-editing quality was largely independent of the measured profile characteristics of the post-editors.

Research Evidence

Aim: To investigate the feasibility and quality of lay post-editing for machine-translated user-generated content within an online community context.

Method: Mixed-methods approach

Procedure: The study involved lay post-editors (community members) correcting machine-translated text from English to German in a technology support forum. Quality was assessed quantitatively through error annotation, domain specialist evaluation, and end-user evaluation. Post-editing behaviour was analyzed qualitatively.

Context: Online technology support forums

Design Principle

Leverage distributed human intelligence for content quality and localization, particularly in user-generated environments.

How to Apply

When designing platforms with user-generated content that needs to be accessible in multiple languages, consider integrating a lay post-editing workflow, perhaps as a tiered quality assurance step.

Limitations

Variability in output quality was noted, and reasons for this variance were difficult to pinpoint. The study did not identify specific post-editing patterns for segments requiring significant effort.

Student Guide (IB Design Technology)

Simple Explanation: People who aren't professional translators can help fix machine translations, especially if the text isn't too long. This is a good way to make sure online content is understandable in different languages.

Why This Matters: This research shows how to use everyday users to improve the quality of translated content, which is important for making digital products accessible to a wider audience.

Critical Thinking: How can the variability in lay post-editing quality be mitigated to ensure a consistent user experience across different languages and content types?

IA-Ready Paragraph: This research demonstrates that lay post-editing of machine-translated user-generated content is a feasible approach, particularly for shorter text segments requiring approximately 35% post-editing effort. This suggests that design projects aiming to manage multilingual content can benefit from incorporating community-driven quality assurance mechanisms, optimizing for efficiency in these specific scenarios.

Project Tips

• Consider how user communities can contribute to content quality.
• Explore the trade-offs between machine translation, human editing, and community involvement.

How to Use in IA

• This study can inform the development of user interfaces for collaborative content creation and editing, especially in multilingual contexts.

Examiner Tips

• Evaluate the practical application of community-based quality control for translated content.
• Consider the ethical implications of relying on non-professional translators.

Independent Variable: Text segment length, post-editing effort required.

Dependent Variable: Post-editing quality, post-editing effort.

Controlled Variables: Machine translation engine, original language, target language, domain of content.

Strengths

• Addresses a gap in research by focusing on lay post-editors.
• Employs a mixed-methods approach for comprehensive analysis.

Critical Questions

• What are the long-term implications of relying on lay post-editors for content quality?
• How can the 'redefinition of quality' for user-generated content be practically implemented in design?

Extended Essay Application

• Investigate the impact of different user interface designs on the efficiency and quality of lay post-editing for machine-translated content.

Source

Community post-editing of machine-translated user-generated content · Arrow@dit (Dublin Institute of Technology) · 2015