Criminal tendency detection from facial images and the gender bias effect

Mahdi Hashemi, Margeret Hall

Research output: Contribution to journalArticlepeer-review

25 Scopus citations

Abstract

Explosive performance and memory space growth in computing machines, along with recent specialization of deep learning models have radically boosted the role of images in semantic pattern recognition. In the same way that a textual post on social media reveals individual characteristics of its author, facial images may manifest some personality traits. This work is the first milestone in our attempt to infer personality traits from facial images. With this ultimate goal in mind, here we explore a new level of image understanding, inferring criminal tendency from facial images via deep learning. In particular, two deep learning models, including a standard feedforward neural network (SNN) and a convolutional neural network (CNN) are applied to discriminate criminal and non-criminal facial images. Confusion matrix and training and test accuracies are reported for both models, using tenfold cross-validation on a set of 10,000 facial images. The CNN was more consistent than the SNN in learning to reach its best test accuracy, which was 8% higher than the SNN’s test accuracy. Next, to explore the classifier’s hypothetical bias due to gender, we controlled for gender by applying only male facial images. No meaningful discrepancies in classification accuracies or learning consistencies were observed, suggesting little to no gender bias in the classifier. Finally, dissecting and visualizing convolutional layers in CNN showed that the shape of the face, eyebrows, top of the eye, pupils, nostrils, and lips are taken advantage of by CNN in order to classify the two sets of images.

Original languageEnglish (US)
Article number2
JournalJournal of Big Data
Volume7
Issue number1
DOIs
StatePublished - Dec 1 2020

Keywords

  • Convolutional neural network
  • Deep learning
  • Facial images
  • Image classification
  • Machine learning
  • Personality traits

ASJC Scopus subject areas

  • Information Systems
  • Hardware and Architecture
  • Computer Networks and Communications
  • Information Systems and Management

Fingerprint

Dive into the research topics of 'Criminal tendency detection from facial images and the gender bias effect'. Together they form a unique fingerprint.

Cite this