The effects of respondent and question characteristics on respondent answering behaviors in telephone interviews

In a standardized telephone interview, respondents ideally are able to provide an answer that easily fits the response task. Deviations from this ideal question answering behavior are behavioral manifestations of breakdowns in the cognitive response process and partially reveal mechanisms underlying measurement error, but little is known about what question characteristics or types of respondents are associated with what types of deviations. Evaluations of question problems tend to look at one question characteristic at a time; yet questions are comprised of multiple characteristics, some of which are easier to experimentally manipulate (e.g., presence of a definition) than others (e.g., attitude versus behavior). All of these characteristics can affect how respondents answer questions. Using a landline telephone interview, we use cross-classified random effects logistic regression models to simultaneously evaluate the effects of multiple question and respondent characteristics on six different respondent behaviors. We find that most of the variability in these respondent answering behaviors is associated with the questions rather than the respondents themselves. Question characteristics that affect the comprehension and mapping stages of the cognitive response process are consistently associated with answering behaviors, whereas attitude questions do not consistently differ from behavioral questions. We also find that sensitive questions are more likely to yield adequate answers and fewer problems in reporting or clarification requests than nonsensitive questions. Additionally, older respondents are less likely to answer adequately. Our findings suggest that survey designers should focus on questionnaire features related to comprehension and mapping to minimize interactional and data quality problems in surveys and should train interviewers on how to resolve these reporting problems.