Do interviewer postsurvey evaluations of respondents' engagement measure who respondents are or what they do? A behavior coding study

Antje Kirchner, Kristen Olson, Jolene D. Smyth

Research output: Contribution to journalReview articlepeer-review

14 Scopus citations


Survey interviewers are often tasked with assessing the quality of respondents' answers after completing a survey interview. These interviewer observations have been used to proxy for measurement error in interviewer-administered surveys. How interviewers formulate these evaluations and how well they proxy for measurement error has received little empirical attention. According to dual-process theories of impression formation, individuals form impressions about others based on the social categories of the observed person (e.g., sex, race) and individual behaviors observed during an interaction. Although initial impressions start with heuristic, rule-of-thumb evaluations, systematic processing is characterized by extensive incorporation of available evidence. In a survey context, if interviewers default to heuristic information processing when evaluating respondent engagement, then we expect their evaluations to be primarily based on respondent characteristics and stereotypes associated with those characteristics. Under systematic processing, on the other hand, interviewers process and evaluate respondents based on observable respondent behaviors occurring during the question-answering process. We use the Work and Leisure Today Survey, including survey data and behavior codes, to examine proxy measures of heuristic and systematic processing by interviewers as predictors of interviewer postsurvey evaluations of respondents' cooperativeness, interest, friendliness, and talkativeness. Our results indicate that CATI interviewers base their evaluations on actual behaviors during an interview (i.e., systematic processing) rather than perceived characteristics of the respondent or the interviewer (i.e., heuristic processing). These results are reassuring for the many surveys that collect interviewer observations as proxies for data quality.

Original languageEnglish (US)
Pages (from-to)817-846
Number of pages30
JournalPublic Opinion Quarterly
Issue number4
StatePublished - Dec 12 2017

ASJC Scopus subject areas

  • Communication
  • History
  • Sociology and Political Science
  • General Social Sciences
  • History and Philosophy of Science


Dive into the research topics of 'Do interviewer postsurvey evaluations of respondents' engagement measure who respondents are or what they do? A behavior coding study'. Together they form a unique fingerprint.

Cite this