ETHOS: a multi-label hate speech detection dataset

Hate speech (HS) is a form of insulting public speech directed at specific individuals or groups of people on the basis of characteristics, such as race, religion, ethnic origin, national origin, sex, disability, sexual orientation, or gender identity¹. This phenomenon is manifested either verbally or physically (e.g., speech, text, and gestures), promoting the emergence of racism and ethnocentrism. Because of the social costs arising out of HS, several countries consider it an illegal act, particularly when violence or hatred is encouraged [10]. Although a fundamental human right, freedom of speech, it is in conflict with laws that protect people from HS. Therefore, almost every country has responded by drawing up corresponding legal frameworks, while research which is related to mechanisms that try to remedy such phenomena has recently been done by the Data Mining and Machine Learning (ML) research communities [21].

Another important issue is that the occurrence of HS phenomena is emerging in the social media ecosystem, distorting their initial ambition of favouring communication between their corresponding members independently of geographical restrictions and enriching similar activities [48]. The anonymity of social media is the primary explanation for the growth of such phenomena, as is the deliberate avoidance of subsequent legislation. As a result, large corporations, such as Google and Facebook, are obligated to remove such violent content off their platforms. Therefore, Artificial Intelligence (AI) methodologies are employed to detect (semi-) automatically HS in real time, or even to prevent users from publishing similar content with appropriate warnings or bans.

The solution of quarantining in an online fashion has recently been demonstrated, trying to smooth the censorship and the possible harmful consequences of HS attacks [53], while learning from short-text segments is blooming in the last years [51]. Two of the most important features accompanying the short-text segments, sparseness, and the presence of noise [50], settle HS detection, a difficult task for the creation of fully automated solutions. Whereas problems of scalability arise when large quantities of data are simply collected without pre-processing or filtering. These points are of primary importance to this work.

To achieve high performance in real-world tasks, AI methodologies require balanced, accurate, and unbiased datasets. This requirement, however, is rarely met without applying proper annotation stages [6]. This is the direction in which our work aims to make a significant contribution, motivated by the HS use case, providing also a generic-based protocol that could be extended to a wide variety of learning tasks. To be more precise, the relevant literature currently contains numerous manually created HS datasets [58, 61]. However, since the majority of them were not carefully collected during the corresponding sampling stages, they are essentially large sets of annotated samples on which undesirable phenomena occur frequently. Specifically, highly imbalanced classes or redundant information prevent the subsequent implemented learning models from effectively harnessing the underlying patterns.

Moreover, by sampling the regions of feature space which express only a restricted level of uncertainty when unlabelled data are queried may settle the learning strategy myopic. All these phenomena violate the previously specified desired requirements resulting in solutions with low variance and/or high bias [55]. Furthermore, most of them are concerned with binary or multi-class classification tasks, while overlooking the more practical case of multi-label classification (MLL). Label dependencies and the semantic overlap that occurs on MLL cannot be ignored when protection from hateful comments is the main task. Since an online comment can fit to more than one defined label at the same time, rather than being limited to just one outcome, investigation of the latter scenario appears to be more effective (see Fig. 1). This aspect is also studied here, because the difficulties described previously are enforced under the MLL scenario.

A simple application that uses the MLL schema provided by the proposed HS dataset could be an assistance system for human staff reviewing comments on social media platforms. This would make it easier for the reviewer(s) to decide if the message contains HS content by providing more insights. For example, if a comment is presented as targeting people with disabilities, directed at a person, and encourages violence, it will be more helpful for the reader to conclude and condemn it for containing HS rather than being presented with a single label (e.g., ‘may contain HS’:{‘yes’,‘no’}). In terms of the ethical issues that emerge in the case of HS, it appears that a proper manipulation protocol is required for preventing possible defects. Such protocols have addressed wider or more focused research topics, such as news articles, although similar directions have recently been explored in the field of HS detection [42].

In this paper, we present the process of creating a dataset with two variations, a balanced binary and a multi-label one, with a step-by-step narrative, to avoid the consequences that typically occur in attempts with data that depend on social media platforms, and to increase the likelihood of mining more informative instances. Although the design of the proposed protocol can fit with any target domain indisputably, we are currently focusing on addressing the HS scenario and provide some insightful analysis of this use case. In this attempt, an existing dataset mined from popular social media platforms has been exploited, while a well-known crowdsourcing platform was used for validating the final result. The proposed annotation protocol’s effects are discussed in detail and visualised using explanatory methods. Following that, a series of experiments are being conducted to determine the baseline performance of this particular dataset using state-of-the-art (SOTA) techniques. From traditional ML algorithms and ensemble models to neural networks (NNs) with and without embeddings (emb) information, binary and multi-label experiments have been performed, inspired primarily by other similar approaches to presenting research datasets [2, 7, 30]. Experiments using Transformers (BERT and DistilBERT) were also included, because they appear to be extremely promising in many text-related machine learning tasks, with evidence of outstanding performance in hate speech detection, as well [27]. Despite the limited size of the investigated dataset, its careful design during the active sampling stage and the consistency of the included samples were proven beneficial based on our results.

Our ultimate ambition, by describing the total procedure and providing the corresponding dataset, is to foster any interested researchers/businesses to take into consideration an approach that attempts to transform the existing insulting environment of social media into a non-hate, inclusive online society. Adoption of the proposed annotation protocol into different scientific fields could prove quite beneficial, especially when the knowledge acquired by oracles during annotation may be ambiguous. The assets also gained by examining the HS problem through a multi-label view help us clarify the harasser’s actual motivations and lead to more targeted comments when dedicated platforms try to inform the corresponding victims [10]. And, of course, the insights gained through such protocols could enhance the ability of ML learners to generalise when applied to different datasets that contain similar classification categories, despite the limited size of the proposed dataset over which they are trained. The proposed strategy of actively creating a balanced dataset, preserving the informativeness of each class and minimising the redundancy of the included instances, constitutes the key asset of our protocol. Our in-depth experiments support our hypotheses, particularly regarding the most difficult classes to detect.

The rest of this paper is structured as follows: Sect. 2 includes several well-documented attempts to address the HS problem using samples gathered from related sources. The proposed annotation protocol is defined in Sect. 3, followed by some extended single/multi-label classification experiments in Sect. 4, which demonstrate the discriminating ability of several algorithms under consideration. Sect. 5 presents a few studies with a variation of the original dataset and two additional datasets. Finally, Sect. 6 discusses the more crucial assets of the proposed dataset, and the annotation protocol, also regarding the recorded experiments, reporting later some remarkable future points that could be further investigated.

A collection of 16,914 hate speech tweets was introduced in a study of how different features improve the identification of users that use analogous language online [58]. Out of the total number of messages, 3383, 1972, and 11,559 concerned sexism, racism, and did not include HS, respectively, while were sent by 613, 9, and 614 users. The corpus was generated by a manual tweet search, containing popular slurs and terms related to sexual, religious, gender, and ethnic minorities to include samples that are not offensive regardless of the inclusion of such words. The main drawback here is the access to the text of the tweets only through the public Twitter API.

Another dataset (D1) [7] contains 24,783 tweets, manually classified as HS (1,430), offensive but not HS (19,190), and neither hate nor offensive speech (4163) by Figure-Eight’s² members. The data were gathered again via the Twitter API, filtering tweets containing HS words submitted to Hatebase.​org. The outcome was a sample of 33,548 instances, while 85.4 million tweets were collected from the accounts of all users. A random sample of this collection led to the final dataset. Nevertheless, this dataset lacks diversity in terms of HS content. For example, the gender-based HS tweets are biased towards women, while the greatest number of them contains ethnicity content.

Research focusing on the identification of misogynistic language on Twitter uses a dataset called Automatic Misogyny Identification (AMI) [12] with 4000 annotated comments and binary labels. Apart from this labelling mode, every comment is defined by two extra fields. The first one concerns the type of misogynistic behaviour: stereotype, dominance, derailing, sexual harassment, discredit, or none (if the tweet is not misogynous). The second one concerns the subject of the misogynistic tweet: active, when it attacks a specific target (individual), passive, when it denotes potential receivers (generic), and again none, if there is no misogyny in the tweet.

The largest online community of white nationalists, called Stormfront, was used to form another dataset [8]. The content in this forum revolves around discussions of race, with various degrees of offensiveness, included. The annotation of the samples is at the sentence level, which is a technique that keeps the smallest unit containing hate speech and reduces noise. The dataset contains 10,568 sentences that are classified as HS (1119 comments) or not (8537 comments), as well as two supplementary classes, relation for sentences that express HS only when related to each other and skip for sentences which are not in English or do not contain any information as to be accordingly classified. Furthermore, information like the post-identifier and the sentence’s position in the post, a user identifier, and a sub-forum identifier, as well as the number of previous posts the annotator had to read before deciding over the sentence’s category are also recorded. The samples were picked randomly from 22 sub-forums covering diverse topics and nationalities.

A dataset introduced by Fox News [16] consists of 1528 Fox News users’ comments (435 hateful), which were acquired from 10 discussion threads of 10 widely read Fox News articles published during August 2016. Context information is considered extremely important, so details such as the screen name of the user, all the comments in the same thread and the original article, are also included.

A recent multi-lingual work (D2) [34], a trilingual (English, French, and Arabic) dataset with tweets, was created attempting to mine similar expressions of 15 common phrases over these languages, focused on different sources of obscene phrases (e.g., more sensitive topic-based discussions based on locality criteria). After tackling some linguistic challenges per separate language, and a strict rule set that was posed to human annotators from the Amazon Mechanical Turk platform to ensure trustworthy feedback, a pilot test set was provided. Having gathered the necessary evaluations, another one reconstruction of the label set was applied, before the final formulation of 5647 English, 4014 French, and 3353 Arabic tweets was reached, annotated over 5 separate tasks. Apart from the binary directness of each tweet that was tackled better by single-task language models, the rest 4 classification tasks, which included at least 5 label gradations, were clearly boosted via multitask single/multi-language or single/multi-lingual models.

The issue of cyberbullying has been recently investigated too, where the skewed distribution of positive and negative comments was tackled by tuning a cost-sensitive linear SVM learner over various combinations of joined feature spaces and obtaining similar performance on both English and Dutch corpus [54]. Additionally, an investigation of recognising the role of each participant during such phenomena took place, while a qualitative analysis raised the difficulty of reducing misclassification scores when irony exists in offensive comments.

A small collection of 454 YouTube comments annotated as HS (120) or not (334) was introduced by the creators of the Hatebusters platform [3], which aims to establish an online inclusive community of volunteers actively reporting illegal HS content on YouTube. This dataset, through semi-supervised learning, was evolving in the Hatebusters platform, improving the predictivity of the ML models. However, this unpremeditated expansion of the dataset led to a more redundant variant of its original form. We use the initial collection of Hatebusters’ data as a seed to the protocol that we propose in the following section.

There is clearly a lot of work and numerous publicly available datasets for hate speech identification. Nevertheless, the majority of these works fail to address a few critical issues. The first is related to data imbalance, and by that we mean not just numerical imbalance across classes, but also semantic imbalance between samples of the same class. Another issue that many works overlook is the heterogeneity of data sources. While most works focus on and gather data from platforms such as Twitter and Facebook, comments from platforms such as YouTube and Reddit, which include a significant amount of hate speech content, remain underrepresented.

To overcome the key weaknesses of the existing collections of HS instances, we introduce a small, yet fairly, informative dataset, ETHOS, that does not suffer from issues such as imbalanced or biased labels (e.g., gender), produced appropriately following the proposed protocol. Considering the aforementioned popular approaches of mining similar datasets for tackling with HS problem, we assume that an appropriate pre-process of initially collected data could improve in general their overall utilisation under ML or AI products, improving the total fitness of data quality, blending data mining techniques related with the field of Active Learning [44], such as query strategy and crowdsourcing platforms. The overview of the proposed annotation protocol is visualised through a flowchart in Fig. 2. The finally obtained dataset is the outcome of a three-stage process, which we describe shortly in the current section.

To evaluate ETHOS, after pre-processing the data, we used a variety of algorithms in binary/multi-label scope to present the baseline performance in this dataset. For the purpose of providing the unbiased performance of each algorithm, we performed nested-cross-validation [57] evaluation, using a variety of parameter setups, for every algorithm except NNs, where we applied tenfold cross-validation [17]. In addition, we binarise the values of each label, which are initially discrete in a range of [0,1], to the {0,1} classes using the rule “If value \(\ge 0.5 \rightarrow 1\) Else value \(\rightarrow 0\)”. More in-depth details follow next.

After setting the baseline performance of ETHOS in multiple ML algorithms, in both binary and multi-label scope, this section aims at highlighting some interesting views and aspects of its usefulness over other learning tasks. First, we fulfil our experimental soundness by setting a fair comparison between a balanced subset and a random subset of ETHOS capturing useful insights under a 1-vs-1 evaluation stage. Second, we examine how the ETHOS dataset can generalise over separate HS datasets when it is applicable. Thus, we transfer its discriminative ability obtained by the proposed underlying representation through training proper ML models. These experiments have been conducted for two well-known datasets on binary (D1) [7] (2017) and multi-label (D2) [34] (2019) level, as described briefly in Sect. 2, commenting the produced results regarding the aspects that we had initially posed and providing accurate explanations about any mismatches over this attempt.

The provision of a new well-designed dataset to the public on a specific subject is always considered a significant contribution [19, 47]. In this sense, our HS dataset, called ETHOS, collected from social media platforms, could be reused by the ML and AI communities. Alleviating redundant information through balancing the proposed dataset between fine-grained classes through a fine-tuned learner and an Active Learning scheme benefited us both from the aspect of less human-laborious effort and, of course, by scoring good learning rates despite the limited cardinality of our collected instances. Redundancy reduction has been shown to be quite beneficial for a variety of learning tasks. More specifically, the proposed protocol offers us a balanced dataset with a rich quality of included instances for both binary and multi-label HS problems. At the same time, our experimental procedure revealed that a proper balance has been achieved between the discriminative ability of the learners, both traditional and neural networks, and the computational resources consumed.

The issue of imbalanced data collection has also affected the performance of similar works, where the need for proper manipulation is clearly stated [18, 33]. The solution of proactive learning has been applied in the latter approach, trying to match the expertise of each human annotator with the most appropriate unlabelled instances. Based on this, the negative effect of harmful annotations can be seriously avoided. This asset should be carefully explored and adopted by our side before enlargement of the current dataset takes place or new data collection attempts get started. We must emphasise once more that, despite the relatively small size of the ETHOS dataset, the human resources invested in adequate labelling cannot be overlooked (2 consecutive months of daily querying of the targeted databases, human annotation in 2 stages, input by a crowdsourcing process). Thus, besides the need for high-quality annotators, mining informative instances that retain the ability to discriminate between hate speech examples, both in binary and multi-label classification tasks, is of high importance. The conducted experiments verify our assumptions following our straightforward protocol, since the learning performance of various models is satisfactory, especially these based on embeddings. Simultaneously, a proof-of-concept of how to exploit the ETHOS dataset’s learning capacity was provided, serving as a seed dataset for generalising to similar hate speech detection datasets.

Some promising directions of our work are mentioned here, trying to take advantage of its assets and the baselines that were posed. The main issue, the shortage of collected data, is a fact that depends on the limitations that occur during exploiting crowdsourcing platforms (e.g., restricted budget, users’ traffic) and the further costs that are induced by the human-intensive stage of actively selecting instances that keep a balanced profile of the target dataset on a daily basis. Investigating the related literature, we have mined some clever ideas that tackle this limitation. Another constraint of the annotation protocol is the requirement for a single annotator in the first stage. This restriction derives from the fact that we intended the annotator to remember the comments approved in the dataset, not let extremely similar comments in, and aggregate semantically diverse comments for each category. However, in future work, we want to overcome this constraint using deep learning techniques to assess the similarity of incoming comments with approved comments and enable multiple annotations by the first stage of the protocol.

We record here the case where an annotation process has been designed using a game-based approach, motivating the human oracles to contribute to assigning sentiment labels to a variety of Twitter instances, surpassing the monetisation incentive [14]. Further enrichment of this dataset could also be carried out, integrating either multi-lingual resources for capturing even more hate speech occurrences, or applying data augmentation techniques [45]. From the perspective of the ML models that we used, pre-processing stages—such as feature selection mechanisms [49] or methods for creation of semantic features [46]—which are established in the realm of short-text input data, could improve the obtained results, and retain interpretability properties in specific cases.

In addition, the ETHOS can be combined with various similar HS datasets—as we stated here initially with two different data collections—for evaluation reasons. The development of hybrid weakly supervised HS detection models, merging semi-supervised and active learning strategies under common frameworks, alleviating human intervention based on decisions over the gathered unlabelled instances that come solely from the side of a robust learner [23, 60], constitutes another very promising ambition. Online HS detection and prevention tools, such as Hatebusters among others, are highly favoured by such approaches. The impact of such detection tools could have been very beneficial in terms of enforcing social awareness and addressing effective ethical issues [1, 10]. Furthermore, it would be interesting to investigate how our annotation protocol can be used to collect a multi-modal hate speech dataset [28], as well as how our collected balanced dataset can enhance tasks like multi-lingual hate speech detection [29].

Finally, the fact of examining ETHOS under the spectrum of multi-labelled nature appears favouring to reviewers on social media platforms, facilitating informative suggestions for HS comments regarding the level of violence, the target of comments, and the categories of HS that are present. However, this is not a multi-purpose HS detection dataset, as the mined comments are based on social media. This means that the corpus contains relatively small sentences. Thus, models trained on this dataset may fail to detect HS in documents on a larger scale without segmentation. On the other hand, the general structure of the proposed protocol could be applied to a variety of learning tasks, especially on large databases, towards better predictions and less intensive annotation [11]. Last but not least, examination of alternative query sampling strategies that support inherent MLL could have proven quite beneficial regarding both the reduction of human effort and the enrichment of attempts like the proposed one [26].