Improving measurements of similarity judgments with machine-learning algorithms

Jeffrey R. Stevens, Alexis Polzkill Saltzman, Tanner Rasmussen, Leen Kiat Soh

Research output: Contribution to journalArticlepeer-review

Abstract

Intertemporal choices involve assessing options with different reward amounts available at different time delays. The similarity approach to intertemporal choice focuses on judging how similar amounts and delays are. Yet we do not fully understand the cognitive process of how these judgments are made. Here, we use machine-learning algorithms to predict similarity judgments to (1) investigate which algorithms best predict these judgments, (2) assess which predictors are most useful in predicting participants’ judgments, and (3) determine the minimum number of judgments required to accurately predict future judgments. We applied eight algorithms to similarity judgments for reward amount and time delay made by participants in two data sets. We found that neural network, random forest, and support vector machine algorithms generated the highest out-of-sample accuracy. Though neural networks and support vector machines offer little clarity in terms of a possible process for making similarity judgments, random forest algorithms generate decision trees that can mimic the cognitive computations of human judgment making. We also found that the numerical difference between amount values or delay values was the most important predictor of these judgments, replicating previous work. Finally, the best performing algorithms such as random forest can make highly accurate predictions of judgments with relatively small sample sizes (~ 15), which will help minimize the numbers of judgments required to extrapolate to new value pairs. In summary, machine-learning algorithms provide both theoretical improvements to our understanding of the cognitive computations involved in similarity judgments and intertemporal choices as well as practical improvements in designing better ways of collecting data.

Original languageEnglish (US)
Pages (from-to)613-629
Number of pages17
JournalJournal of Computational Social Science
Volume4
Issue number2
DOIs
StatePublished - Nov 2021

Keywords

  • Classification
  • Intertemporal choice
  • Judgment
  • Machine learning
  • Similarity

ASJC Scopus subject areas

  • Transportation
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Improving measurements of similarity judgments with machine-learning algorithms'. Together they form a unique fingerprint.

Cite this