Platform affordances enable coordinated hate raids, necessitating moderation-by-design.
Category: User-Centred Design · Effect: Strong effect · Year: 2023
The design of live streaming platforms, specifically their affordances, can inadvertently facilitate coordinated hate raids, highlighting the need for proactive moderation strategies embedded within the design process.
Design Takeaway
When designing interactive systems, anticipate how features might be misused for harassment and build in protective mechanisms from the start.
Why It Matters
Understanding how system features can be exploited for harassment is crucial for creating safer online environments. Designers must consider the potential for misuse of platform affordances and integrate moderation considerations from the outset of product development.
Key Finding
Live streaming platforms' design features can be exploited for coordinated attacks like hate raids, disproportionately harming marginalized streamers who lack sufficient platform support and struggle with existing moderation tools.
Key Findings
- Hate raids are human-bot coordinated attacks that exploit live streaming systems to target marginalized streamers.
- Marginalized streamers experience significant harm with inadequate platform support.
- Current moderation is primarily technical, yet streamers face challenges balancing moderation with participation due to their marginalized status.
Research Evidence
Aim: How do the affordances of live streaming platforms contribute to coordinated hate raids, and how can moderation-by-design principles mitigate these issues?
Method: Qualitative analysis
Procedure: The study analyzed discussions about hate raids on the r/Twitch subreddit to understand the phenomenon and its impact on streamers.
Context: Live streaming communities, online harassment, content moderation
Design Principle
Design for safety by considering potential misuse and integrating proactive moderation.
How to Apply
Before launching new features or platforms, conduct a 'misuse analysis' to identify potential avenues for harassment and design countermeasures.
Limitations
The study is based on discussions within a specific online community (r/Twitch) and may not represent all live streaming platforms or user experiences.
Student Guide (IB Design Technology)
Simple Explanation: The way a website or app is built can accidentally help people be mean to others, so designers need to think about safety when they create new features.
Why This Matters: This research shows that good design isn't just about making things easy to use, but also about making them safe and fair for everyone.
Critical Thinking: To what extent can designers be held responsible for the misuse of their creations, and where does user responsibility begin?
IA-Ready Paragraph: This research highlights the critical role of platform affordances in enabling coordinated online harassment, such as hate raids on live streaming platforms. The study found that system designs can inadvertently facilitate attacks, particularly against marginalized users, underscoring the necessity of a 'moderation-by-design' approach. This principle suggests that safety and moderation should be integral to the design process from its inception, rather than being an add-on, to proactively mitigate potential abuse and ensure a more inclusive user experience.
Project Tips
- When researching user needs, also consider potential negative uses of your design.
- Think about how your design could be used to exclude or harm certain groups.
How to Use in IA
- Use this research to justify the inclusion of specific safety features or moderation strategies in your design proposal.
- Reference this study when discussing the ethical considerations of your design project.
Examiner Tips
- Demonstrate an understanding of how platform design can impact user safety and community health.
- Show how you have considered potential negative consequences of your design choices.
Independent Variable: Platform affordances (features, design elements)
Dependent Variable: Incidence and impact of hate raids, streamer experience
Strengths
- Focuses on a novel and under-researched form of online harassment.
- Provides a theoretical lens (affordances) to explain the phenomenon.
- Offers practical design recommendations (moderation-by-design).
Critical Questions
- How can designers effectively anticipate and mitigate unforeseen forms of online abuse?
- What is the balance between platform control and user freedom of expression in online communities?
Extended Essay Application
- Investigate the affordances of a specific digital platform (e.g., a social media app, a gaming interface) and how they might be exploited for negative social interactions.
- Propose and prototype design modifications to enhance user safety and mitigate potential harassment within that platform.
Source
Hate Raids on Twitch: Understanding Real-Time Human-Bot Coordinated Attacks in Live Streaming Communities · Proceedings of the ACM on Human-Computer Interaction · 2023 · 10.1145/3610191