The Center for Strategy Research, Inc. Vol 1 Issue 2   October 2005


In our last newsletter [click here to see it], we focused on the importance of asking open-ended, instead of closed-ended, questions, when trying to get a better understanding of what people really think.

This month we address the other end of the spectrum: the unfettered, boundary-less, open-ended question.

As always, your thoughts and comments are appreciated.

Julie Brown

Mark Palmerino
Executive Vice President

Things That Make You Go “Huh?”

I picked up my college friend Rose at the Boston airport last week. She lives in Florida, and together we drove up to Portsmouth, New Hampshire for the weekend, to spend a few days with two other college buddies of ours.

You don’t need to recall every scene from The Big Chill to imagine what it was like. Lots of laughing and joking, almost as much drinking and a constant barrage of questions in all directions, as we made up for lost time over a short weekend.

After two days of this — and once again as comfortable with each other as we were back in college — we moved easily from the simple (e.g. “Where do you want to go for dinner?”), to the not so simple (e.g. “Why didn’t you ever marry that guy?”) — literally on a minute-by-minute basis.

On the third day, however, something happened. Another woman — Millie — arrived, and she joined us for dinner that night.

I didn’t waste any time. “What’s your five-year plan?”, I asked Millie, temporarily forgetting that she was new to the group.

She put down her fork and just stared at me. “What’s that supposed to mean?!”, she said.

I think you can see what I’m getting at. This type of question — both out of context and way too personal — caused Millie to clam up. Without the history of two intense days together that the rest of us shared, Millie didn’t know what to make of my inquiry.

And yet, in research the same kind of thing is forced upon hapless respondents all the time. This creates a number of problems:

  • The respondent may choose not to answer at all. This is particularly likely to occur when a question is asked before establishing rapport. It’s one reason why it’s important to ask demographic questions at the end of a survey. Even when demographics need to be part of the screening process, it’s important to introduce them in a way that doesn’t throw respondents off.
  • The answers the respondent gives may be vague, evasive, or even purposely untrue, and/or they may disengage from the interview. Particularly in business to business interviewing, it’s important to realize that respondents sitting at their desks are easily distracted. Disengagement is easy, and therefore, it’s important to create an interview guide where questions flow naturally from one to the next. Proper lead-in and context helps keep the respondent mentally and physically engaged in the interview.
  • Even if the respondent answers in a direct and honest manner, he or she may have misunderstood the context of the question. Without a clear understanding of the objectives of a survey and of each question (or, at least, each set of questions), it’s possible that a respondent will answer one or more questions in ways that don’t address the issues. And if that happens, you’ve spent time and effort (and most likely money) getting information that will prove worthless.

While the opportunities to ask — and get answers to — well-phrased open-ended questions are often limitless, it’s important to first set the stage. This is not to close off options, but to give your conversation partner the opportunity to address the issues that you’re investing in, to direct the conversation in a natural and useful way, and, to give your respondent time to think about the issues. This allows both interviewer and respondent the benefit of time to ensure a clear understanding of what the survey participant really thinks.

— Julie

Click here to share this newsletter with a colleague.

Mixology (Putting research into practice)

Sometimes, what appears to be a straightforward survey question leads to answers that are less than helpful. A good example of this is when a question begins with “How…”

“How does this happen you ask?”

I reply: “Quite easily!”

Which is an illustration of the problem.

Let me start, however, by pointing out that asking “how” does not necessarily lead to problems. The following kinds of “how” questions work quite well:

How satisfied are you with the service you receive?
How easy was it to accomplish the purchase?
How important is a knowledgeable representative to you?

When “how” is followed by an adjective or adverb, the question tends to work well. However, take a look at the following question:

How did you accomplish the purchase?

A reasonable, and common, answer to that kind of question is “very easily” or “with great difficulty.” This is because the respondent interpreted the question to mean something like: “How easy was that?”

If the purpose of the question, on the other hand, is to get at “in what ways” the purchase was accomplished, or what were the steps that were taken to purchase the item, an answer like “very easily” doesn’t really help us. While well-trained interviewers, who know the goal of the question, can tactfully redirect the respondent by rephrasing the question, ambiguity of this sort is still inefficient. Further, with certain modes of survey delivery, for example, Internet surveys, rephrasing is not possible and a good opportunity for learning is missed.

In general, when you are faced with a question where “how” is followed by a verb (e.g., do, is, will, can, etc.), the question is very likely ambiguous. In that case, dropping the word “how” and replacing it with a phrase like “in what manner” or “in what ways” will often lead to more useful and actionable answers.


Things That Make You Go “Huh?”

Mixology (Putting research into practice)

Twist and Shout

About Us

Our question from last month — “Why did we name the newsletter, “Research With a Twist”? — resulted in lots of responses (thank you)!

We had two favorites.

The first from a client in New York City who told us: “It’s what you do: Research with something a little different.”

The second from a marketing consultant in Baltimore who wrote: “Because you like martinis!”

You’re both correct, and thanks to all of you who responded!

CSR Executive Vice President, Mark Palmerino will be speaking at the Nursing Research Council on November 2nd in Winchester, Massachusetts.

His talk, “Bedside manner – not just for doctors and nurses any more!”, will discuss the keys to gaining the most insight from research respondents.

Mark has been consulting with members of the group for more than a decade, and helps them design and implement research projects that impact the quality of patient care.

“If we knew what we were doing, it wouldn’t be called research, would it?”

— Albert Einstein

Enter your email here to subscribe to “Research with a Twist