Brown Hands Aren’t Terrorists: Challenges in Image Classification of Violent Extremist Content

Margeret Hall, Christian Haas

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

The Internet eases the broadcasting of data, information, and propaganda. The availability of myriad social media has turned the spotlight on violent extremism and expanded the scope and impact of ideology-oriented acts of violence. Automated image classification for this content is a highly sought-after goal, yet raises the question of potential bias and discrimination in case of incorrect classification. A requirement for addressing, and potentially counter-acting, bias, is the existence of a reliable training dataset. To demonstrate how such a dataset can be developed for highly sensitive topics, this article operationalizes the process of human-coding images posted on the open social web by violent religious extremists into four master categories and four subcategories. We concentrate on the group ISIS due to their prolific digital content creation. The developed training dataset is used to train a convolutional neural network to automatically detect extremist visual content on social media and determine its category. Using inter-coder reliability, we show that the training data can be reliably coded despite highly nuanced data and the existence of various categories and subcategories.

Original languageEnglish (US)
Title of host publicationDigital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. AI, Product and Service - 12th International Conference, DHM 2021, Held as Part of the 23rd HCI International Conference, HCII 2021, Proceedings
EditorsVincent G. Duffy
PublisherSpringer Science and Business Media Deutschland GmbH
Pages197-207
Number of pages11
ISBN (Print)9783030778194
DOIs
StatePublished - 2021
Event12th International Conference on Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management, DHM 2021, Held as Part of the 23rd HCI International Conference, HCII 2021 - Virtual, Online
Duration: Jul 24 2021Jul 29 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12778 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference12th International Conference on Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management, DHM 2021, Held as Part of the 23rd HCI International Conference, HCII 2021
CityVirtual, Online
Period7/24/217/29/21

Keywords

  • Algorithmic bias
  • Image processing
  • Mixed methods
  • Social media
  • Violent extremist organizations

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Brown Hands Aren’t Terrorists: Challenges in Image Classification of Violent Extremist Content'. Together they form a unique fingerprint.

Cite this