…the Insights Won’t Matter to Your Business
If the short survey is still asking consumers questions that don’t matter to them, then the answers will still be distorted by our respondents’ boredom. What can we do as researchers to improve the situation?

Many of us in market research, especially those in methodologist and data science rolls, come from an academic background. We’re trained to think deeply about the implications of our assumptions for our statistical models. We test and design instruments, develop best practices, and deploy surveys, using the tools of social science to mine data and insights for our corporate partners. But in all of this high-minded science, we’re missing some simple lessons about survey design that may lead us, and our clients, astray.

To put it simply, too many of our questions don’t matter to our respondents. We’ve worked for years as an industry to make surveys shorter and our respondent experiences better, to develop better models and statistical techniques. But if our respondents simply don’t care about the answer to our questions, then we can never be sure about the quality of our data. Rather than thinking about respondent engagement as a user experience design challenge to be solved by programmers, survey writers and other researchers need to take responsibility for asking our respondents questions that matter to them.

When researchers pile on objectives, modules, and attribute batteries, we’re taking up valuable survey real estate, but we’re also taxing the energy our respondents have to care about their answers. I may have strong opinions about the first 2 minutes of questions about a grocery category, but at minute 5 there’s a good chance I’ve stopped caring about which frozen pizza brand is the most “inquisitive.”  Most likely, the responses in the aggregate will likely follow the same general trend as earlier, more “high-level” questions, rewarding the most popular brands in proportion to their competitive standing.

One often-used solution to this problem is to make our surveys shorter. While this can work to a point, long surveys are only a small part of the issue. If the short survey is still asking consumers questions that don’t matter to them, then the answers will still be distorted by our respondents’ boredom. If bored, disengaged respondents are giving us bad answers, what can we do as researchers to improve the situation?

Read Full Article Here