The research project "Using AI to Increase Resilience against Toxicity in Online Entertainment (ToxicAInment)", by Prof. Dr. Yannis Theocharis (Chair of Digital Governance), funded by the Bavarian Research Institute for Digital Transformation (BIDT), explores the spread of extremist, conspiratorial and misleading content on social media, investigating how this content is embedded through entertaining content. It aims to deepen the understanding of the impact of this content on user behavior by combining entertainment theories, visual communication and toxic language with AI methods. This project makes an important contribution to analyzing and combating online toxicity. More information can be found on the project page or in the BIDT press release.

With the aim of tackling the issue of harmful discourse online, we joined forces with the Bavarian State Ministry of Justice in order to build a community of practice. In a series of events of the Reboot Social Media Hub, we brought together academics and practitioners working on hate speech and other forms of hateful content online to discuss current problems and potential solutions.

Opening up the dialogue, our panel discussion with Teresa Ott (Hate Speech Officer at the Attorney General’s Office), Anna Wegscheider (lawyer at HateAid), and Svea Windwehr (policy analyst at Google), moderated by Georg Eisenreich, MDL (Bavarian State Minister of Justice) and Urs Gasser (TU Munich), gave around 100 guests that participated in the event deeper insights about the current state of the EU’s Digital Services Act and its impact on prosecutors, platforms, and users.  

Key insights from the discussion

While EU-wide harmonization through the DSA was identified as having great potential, it still faces setbacks when compared to the NetzDG, such as the lack of inclusion of deletion periods or concrete details on enforcement for hate speech violations. It was hence seen critical to explore ways to guarantee that the stronger and more actionable aspects of the NetzDG will still be available even if the DSA and its more ambiguous propositions are put in place. 

In general, the panelists noticed that the internal processes of major platforms regarding content moderation practices are still a "black box" for both law enforcement and victims of online hate. There was a wide consensus that this could and should be improved through expanded cooperation and communication between stakeholders from civil society, agencies, and platform operators. 

A final point was raised regarding public awareness of hate speech. Only 2 out of 10 online offenses are currently reported. To increasingly report and prosecute online hate, awareness of digital violence must be further increased - not only among the population but also in the judiciary and law enforcement. But as the number of reported cases increases, additional resources will then also become necessary for the responsible authorities to prosecute these instances. 

Session 1 focused on the latest insights on hate speech, incivility, and misogyny in online discourse. Based on inputs from Yannis Theocharis, Janina Steinert, and Jürgen Pfeffer (all TU Munich), participants discussed trade-offs between the need for different forms of content moderation vis-à-vis freedom of speech as a fundamental norm. While there was a consensus that a better understanding of the “grey zones” of hate speech, and how to deal with it, is needed, it was clear that some types of online behavior should not be normalized. It was also stressed that online hate is spread by a comparatively few, which are extremely loud and hence gain a lot of traction. This, in turn, has implications regarding whom policies to regulate harmful online content should be directed towards: the few haters or the large masses. 

Session 2 dealt with questions related to the implementation of the Digital Services Act concerning online hate. Following inputs from Till Guttenberger (Bavarian State Ministry of Justice) and Teresa Ott (Hate Speech Officer at the Attorney’s General Office), it was debated how to keep effective measures of the NetzDG alive once the Digital Services Act is enacted. A core topic focused on how future institutions and mechanisms should be designed. Participants were also wondering how to best increase awareness among victims and the public on ways to report hate speech.

Session 3 looked beyond the law to address uncivil discourse online. Christian Djeffal (TU Munich) spoke about content moderation together with users, while Sandra Cortesi (University of Zurich & Harvard University) gave an overview on how to educate children with the skills to navigate online discourses on social media. The big questions focused on finding sweet spots between education and regulation– which is probably not an “either/or” – as well as who is in the best position to create educational content highlighting that all relevant actors need to be on board. 

Partners & organization

The events were jointly organized by the TUM Think Tank, the Professorship for Public Policy, Governance and Innovative Technology, and the Bavarian State Ministry of Justice. Our special thanks go to our panelists Teresa Ott, Svea Windwehr, and Anna Wegscheider for sharing their expertise. We thank all participants for their input and engaged discussions as we are looking forward to continuing this conversation that we started. 

Impressions of the workshop © Thomas Gunnar Kehrt-Reese