Researchers at the University of California, Riverside found that surveys containing nearly repetitive questions may result in unfavorable outcomes, including unreliable data.
Released in the Journal of Marketing Research, the study used mouse- and eye-tracking to trace the decision processes of participants as part of four experiments.
“The authors posit that every time a respondent answers an additional elicitation question, two things happen: (1) they provide information about some parameter(s) of interest, such as their time preference or the partworth for a product attribute, and (2) the respondent increasingly adapts to the task—i.e., using task-specific decision processes specialized for this task that may or may not apply to other tasks,” according to the study.
The authors noted that people are less willing to answer questions with slight variations compared to other questions, often providing similar answers to all questions.
“Marketers, policymakers, and researchers who rely on long surveys to predict consumer or voter behavior will have more accurate data if they craft surveys designed to elicit reliable, original answers,” the study’s authors suggested.
“The research suggests that to maximize the validity of preference measurement surveys, researchers could use an ensemble of methods, preferably using multiple means of measurement, such as questions that involve choosing between options available at different times, matching questions, and a variety of contexts.”
