The Center for Strategy Research, Inc. Vol 2 Issue 1   January 2006

Happy New Year and Welcome!

You don’t need to be Emily Post to know that too many guests at an event can detract from even the best setting.

And yet, many companies put the success of their research projects at risk when they also attempt to overpopulate them — with questions! Read on for our take on why this is problematic.

As always, your thoughts and comments are appreciated.

Julie Brown

Mark Palmerino
Executive Vice President

Don’t Crowd the Pool

One of our company’s largest clients is based in Miami. Of course it’s January here in New England, and my icy toes and I are well aware of our good fortune whenever we visit this client — sometimes several times — during the winter.

As if that isn’t upsetting enough to those of you who also hail from snow-bound climates, I would also like to mention that on my last trip to Miami, I was invited to a beautiful pool party!

Interestingly, what made it so beautiful was that in addition to the glorious weather, the number of guests was limited. My experienced hosts knew that even an Olympic-sized swimming pool gets less and less inviting as more hot, thirsty people crowd around it.

If you’re still with me (and believe me, I know that all this talk of pool parties is enough to drive any driveway- shoveling citizen to reach for the delete key), you’re probably wondering what any of this has to do with research. Good question.

It’s simply this: Loading up a research study with too many objectives, or, the frequent result, too many questions, is the research manager’s equivalent of an overcrowded pool party.

I lost count long ago of the number of times a research manager has introduced a question pattern with little correlation to the objectives of a given study with: “Well, we thought, since you have these people on the phone, you could ask this too.”

Because while this may seem to make sense at first blush, in practice, it’s just the opposite. The temptation to take advantage of the presence of a willing survey participant to ask as many questions as the respondent has time and patience for adds little useful data, often adds cost, and may even reduce the value of the questions that are on topic.

Here’s a good example that came up in a recent study we undertook for a leading media conglomerate. This conglomerate, whose sponsorship of the survey was not disclosed, structured the questionnaire to start by asking about diversity initiatives within the Fortune 1000 companies to which this survey was directed.

After about 10 minutes however, the question pattern changed completely — driven by the motivation to “take advantage of the fact you have these folks on the line” — to a series of questions about media use and newspaper/newsmagazine readership patterns.

It didn’t go well. Three reasons why: Transition, Target, and Transparency.

  1. Transition. When the subject changed abruptly, it was difficult to manage the transition from one topic to another within the interview. Even the most well-crafted transition verbiage requires time to deliver and a concurrent increase in respondent patience (i.e. decreased completion rates, resulting in increased costs).
  2. Target. It turned out that much of the target audience to whom the media use questions (part two of the survey) were directed, were entirely clueless about diversity initiatives within their corporation (part one of the survey). Therefore, their level of engagement for the first 10 minutes of the interview suffered and the reliability of the answers was suspect. Others simply told us to go talk to the HR or Diversity officer about the questions, a detour which resulted in two interviews rather than one as planned
  3. Transparency. Some number of executives terminated as soon as we got to the media questions, expressing frustration that two entirely unrelated areas were being merged into one survey. As intelligent and sophisticated buyers and users of research themselves, it was transparent that the client was trying to load multiple objectives into one conversation.

In summary, I can’t emphasize enough the importance of not adding questions to an interview that don’t relate directly to a study’s main objectives. Like a poorly planned pool party, too much overcrowding prevents anybody from getting the results they’re hoping to achieve.

Thanks for reading. Now, I’m headed back to Miami and away from New England winters!

— Julie

Click here to share this newsletter with a colleague.

Mixology (Putting research into practice)

When an Answer, Isn’t!

“There is no greater impediment to the advancement of knowledge than the ambiguity of words.”
– Thomas Reid, Scottish Philosopher

In a previous Mixology, we discussed the ambiguity that can arise from a poorly worded question. Another area where ambiguity occurs is in the answers that respondents give.

In general, allowing people to respond in their own words to the researcher’s questions of interest leads to a much better understanding of what they think, and why they think it. This insight is critical to making the right decision with regard to the product or service being researched.

However, there are times when a respondent’s “answer” isn’t as full of useful information as we’d like. It is prudent, therefore, for interviewers to be sensitive to the ambiguity inherent in many words and be prepared with probes designed to help the respondent clarify what he means.

In the context of good open-ended interviews, many words can be ambiguous. Examples include: good, bad, service, convenient and so on. The following are actual quotes from four different respondents, all of whom are describing the exact same bank. These illustrate both how ambiguous a concept like “convenience” can be and how judicious probing can clarify what the respondent really means. (Note: Probes by the interviewer are in square brackets, respondent definitions of “convenient” are in italics.)

“They’re the largest in our area. They’ve gobbled up every other bank around. They’re convenient, lots of branches.

Convenience.” [Meaning what?] “Close to our office. If they weren’t just across the street, I’d switch to the closest one.”

Convenience.” [Meaning?] “They’re a smaller bank, which makes it more personalized. Yet, they’re big enough to allow me the convenience of getting everything I need from one bank.” [How close are they?] “Oh, not very.”

“I would say convenient because they don’t charge us a fee for our Escrow accounts, which most banks do. So that was another major decision.” [How close are you?] “Oh, we do our banking by phone. I don’t even know where they are.”

While it does take some training, interviewers can and should become sensitive to ambiguous words. When such a word is detected, a simple probe like “What do you mean by (ambiguous word)?” is often all it takes to encourage a respondent to explain what he means. With this clarification, deeper insights are obtained and better business decisions made.

— Mark


Don’t Crowd the Pool

Mixology (Putting research into practice)

Twist and Shout

About Us

Last week we e-mailed each of you an invitation to share your feedback about this e-newsletter. If you have not yet done so, please click the link below to participate in a very short survey we are using to solicit feedback from “Research With A Twist” recipients.

As a “thank you” for a few moments of your time, we will donate $5.00 to the American Red Cross. We very much appreciate your opinions of our newsletter and how we can improve it.

Just follow this link to share your ideas with us.

“Not everything that can be counted counts and not everything that counts can be counted.”

— Albert Einstein

Enter your email here to subscribe to “Research with a Twist

Problems? Click here to send us an email with your request.
About Us
The Center for Strategy Research, Inc. (CSR) is a research firm. We combine open-ended questioning with our proprietary technology to create quantifiable data.


(617) 451-9500
Understanding What People Really Think

WordPress Lightbox Plugin