Brown Hands Aren’t Terrorists: Challenges in Image Classification of Violent Extremist Content

The Internet eases the broadcasting of data, information, and propaganda. The availability of myriad social media has turned the spotlight on violent extremism and expanded the scope and impact of ideology-oriented acts of violence. Automated image classification for this content is a highly sought-after goal, yet raises the question of potential bias and discrimination in case of incorrect classification. A requirement for addressing, and potentially counter-acting, bias, is the existence of a reliable training dataset. To demonstrate how such a dataset can be developed for highly sensitive topics, this article operationalizes the process of human-coding images posted on the open social web by violent religious extremists into four master categories and four subcategories. We concentrate on the group ISIS due to their prolific digital content creation. The developed training dataset is used to train a convolutional neural network to automatically detect extremist visual content on social media and determine its category. Using inter-coder reliability, we show that the training data can be reliably coded despite highly nuanced data and the existence of various categories and subcategories.