The Center for Strategy Research, Inc. Vol 7 Issue 8   December 2011


As a fascinating speaker from this year’s Market Research Event explained, more choice, in all situations and for all people, isn’t necessarily a good thing. Today’s newsletter looks at the implications of this perspective for those of us engaged in market research design.

And speaking of choice, all of us at CSR wish all of you a joyous holiday season and a healthy and prosperous 2012 – however you choose to celebrate!

Read on for more…

Julie Brown

Mark Palmerino
Executive Vice President

The Choice is Yours – 3 Recommendations for Improving Research
Having recently returned from last month’s Market Research Event in Orlando, both my brain and my laptop are overflowing with things to think about, recommendations to implement and processes to improve.

As always, the conference was stocked with excellent, thought-provoking speakers. For me, one of the highlights was Sheena Iyengar, Columbia Business School professor and author of The Art of Choosing.

Considered one of the world’s leading experts on choice (and the often counter-intuitive truths about the way we make decisions), Professor Iyengar’s presentation was fascinating. And while it wasn’t about market research per se, as I sat there taking notes, I couldn’t help but think that much of what she shared relates directly to research and survey design.

With that in mind, I give you three of her main recommendations and the implications, as we see them, for constructing research projects:
  1. Cut: Having too many choices hinders our ability to make decisions at all, let alone good ones.

    While Iyengar’s name may not ring a bell for you, you’ve no doubt heard of her most famous study: jam preferences among supermarket shoppers. The study found that when shoppers were given 24 jams to sample and a coupon to then purchase one, only 3% of samplers did. When another group of shoppers was given just six sample choices and a coupon, a staggering 30% did.

    Why? According to Iyengar, choices require “cognitive energy” — too many choices at one time causes people to fatigue and, as a result, make fewer decisions.

    It’s important to remember this concept when building closed-ended surveys — giving respondents too many options can be taxing, leading to less-than-optimal results and higher rates of attrition. To the extent possible, try and stay within the 5–9 range of options per question.

    By the same token, surveys that are too long overall also lead to diminished results (7–10 minutes is best for online surveys; twice that for closed-ended phone surveys).
  1. Categorize: Group options together to reduce fatigue.

    Wine stores separate red from white; restaurant menus group items by course; even the zoo groups animals by continent. Grouping like things reduces fatigue and makes decision-making easier.

    In terms of survey design, when cutting choices is simply not an option, we recommend grouping similar choices and using a stepped, questioning approach.

    For example, a survey regarding post-exercise beverage preferences might contain 30 possibilities, from Diet Coke, to fresh-squeezed orange juice, to bottled water. That many options can easily overwhelm respondents.

    Breaking it into a two-step process, however — first offering a choice between “soda, juice or water” and then offering more specific options, depending upon the category chosen — is an easier mental task for participants.

    Not only does the stepped approach reduce the number of options presented at any one time, it cuts the number of choices overall, since those who choose “soda,” for example, never see the juice or water choices.

    It’s worth noting as well that this structure, while increasing the number of overall questions in an instrument, may in fact reduce its average completion time. Allowing people to make more decisions more quickly can be faster overall than if they are required to wade through many options per question.
  1. Condition for complexity: Teach people to succeed easily.

    In one study relating to choice, participants were asked to “configure a vehicle” using a computer interface. Along the way, on an option-by-option basis, they were told to either accept the default selection or choose something different. The results revealed that when more complicated options (e.g., body color) were presented earlier in the process than simpler choices (e.g., sunroof or not?), participants were more likely to begin accepting the default selections sooner.

    The point here is subtle, but important: Surveys don’t just gather data… they teach respondents about the instruments themselves as they interact with it.

    Simpler questions early on can condition participants to believe — consciously or not — that the overall task is easier than they might otherwise think. Asking gender, for example, prior to a question regarding income level may give participants a more positive view of the survey overall.

    From a design perspective, this suggests that the order of the questions can be critical, both in terms of the quality of the answers received and the likelihood to complete.
Here’s the Twist: As Professor Iyengar explained so persuasively, more choice, in all situations and for all people, isn’t necessarily a good thing.

In survey design in particular, it’s worth remembering that fatigue, confusion and boredom work against our best efforts to gather accurate and meaningful data. Cutting, categorizing and conditioning for complexity are three principles that go a long way in optimizing research results.

— Mark

Click here to share this newsletter with a colleague.

Professor Iyengar explained that for Americans in particular, choice is important to our egos, allowing us to feel in control, differentiate ourselves, and identify with other people and different groups.

To the extent possible that your research technique allows people to express their motives, you’re going to be more successful. In this regard, qualitative methodologies, which allow for a much greater degree of open-ended questioning, give participants the space to:

  • Say things in their own words. By removing most of the filter of “the instrument,” respondents in an open-ended survey can speak more freely and expressively.
  • Explain why they choose what they choose. Questions such as “Why did you want that?” and “What benefit did you get?” give researchers insights that would likely never be uncovered with a closed-ended approach.
  • Improve the research for future phases. Open-ended questions reveal information to the researcher that may be useful in terms of cutting, categorizing and conditioning in quantitative phases of the research. We may assume, for example, that color is the best way to categorize a survey about wine. In-depth discussions may reveal, however, that price, region, taste, age or something else entirely leads to a more meaningful grouping for our target audience.
If choice is an expression of our internal selves, more freedom to choose how one responds will provide a more accurate view of feelings and preferences.


The Choice is Yours – 3 Recommendations for Improving Research

Mixology (Putting research into practice)

Twist and Shout

About Us

We are pleased to announce that Marissa Glowac, who joined CSR in late 2005 as a Research Manager, has been promoted to Senior Research Manager, effective 1/1/2012.

A graduate of Colby College, with a graduate degree from UMass, Marissa has participated in and managed a wide variety of qualitative and quantitative engagements over more than ten years of experience in the research industry.

In her tenure with CSR, Marissa has managed many of our most complex engagements, including Prudential Insurance’s annual study of employee benefits, one of its best-known Thought Leadership studies. (A copy of the 2011 study can be accessed by clicking here.)

Please join us in congratulating Marissa in her success!

“Once again, we come to the Holiday Season, a deeply religious time that each of us observes, in his own way, by going to the mall of his choice.”

— Dave Barry

Problems? Click here to send us an email with your request.
The Center for Strategy Research, Inc. (CSR) is a research firm. The “Twist” to what we offer is this: We combine open-ended questioning with our proprietary technology to create quantifiable data. As a result, our clients gain more actionable and valuable insights from their research efforts.

Understanding What People Really Think

WordPress Lightbox Plugin