Comparing survey ranking question formats in mail surveys

Jolene D. Smyth, Kristen Olson, Allison Burke

Research output: Contribution to journalArticlepeer-review

3 Scopus citations


Although questions that ask respondents to rank-order a list of items can be analytically valuable, responding to ranking questions typically requires a good deal of cognitive effort. This is especially true in mail questionnaires where the advantages of electronic response formats available in web surveys are inaccessible. In this article, we examine two alternative formats for ranking questions in mail surveys. Using a nationally representative mail survey of U.S. adults, this article experimentally compares ranking formats in which respondents write numbers in boxes versus selecting items for the most and second most important issues using a grid layout. Respondents to the numbering format were more likely to provide usable data, although one-third of respondents in this format still did not follow instructions correctly. Substantive responses differed somewhat across formats. Less educated respondents had difficulty with both formats, resulting in substantively different conclusions about preferences across formats for this group. A numbering format is more effective than a most-second most grid format for collecting ranking data in mail surveys.

Original languageEnglish (US)
Pages (from-to)502-516
Number of pages15
JournalInternational Journal of Market Research
Issue number5
StatePublished - Jan 1 2018


  • Mail survey
  • Questionnaire design
  • Ranking
  • Visual design

ASJC Scopus subject areas

  • Business and International Management
  • Economics and Econometrics
  • Marketing


Dive into the research topics of 'Comparing survey ranking question formats in mail surveys'. Together they form a unique fingerprint.

Cite this