Research with a Twist
blue lineVol. 9, Issue 4, June 2013
 CSR - Center for Strategy Research
 In This Issue…

twistAndShoutTwist and Shout


Please visit CSR’s booth at LIMRA’s annual Marketing and Research Conference, Wednesday 5/29 to Friday, 5/31, at Disney’s Yacht & Beach Club Resort, Orlando, FL.

Julie, Mark and Marissa hope to see you there!

 Quote of the Month
“Laugh as much as you choose, but you will not laugh me out of my opinion.”

– Jane Austen, Pride and Prejudice

 click here to subscribe to research with a twist
Questions? Click here to send us an email with your request.
(617) 451-9500


We’re all familiar with market research conducted to discover how satisfied customers are with products, services or experiences they’ve had with particular companies. Research conducted before the fact, however (i.e., prior to the launch of a product or service), is equally essential.

Today’s newsletter, penned by our SVP of Operations, Jennifer Lacy, looks at three common biases that keep companies from conducting proper market research prior to development and launch.

signature - Julie
Julie Brown

signature - Mark
Mark Palmerino

Executive Vice President
Julie and Mark
articleOnePride and (Poverty and) Prejudice
As you might imagine, the work we do here at CSR – conducting large, multifaceted market research engagements for a number of clients simultaneously – necessarily involves some fairly sophisticated systems for managing data, participants and processes.

We pride ourselves on doing this well. That said, we do confess to the occasional goof.

Last week, for example, one of our teammates was working on transcribing a series of recorded interviews. As he got into the work, however, he realized he didn’t have a full list of participants. So he sent off an e-mail to another CSR staffer, looking for the list. She thought she had sent it; he was certain she hadn’t.

Back and forth it went, until finally, the transcriber found the list in an earlier e-mail, included as an additional tab in one of the Excel spreadsheets – a tab that he had originally missed.

It was a simple oversight, and one which, thankfully, only wasted a small amount of time for those involved. And yet it served as a reminder of what happens when other, often much larger, research-related oversights occur.

This prompted us to consider and categorize three common reasons for research oversights that we often see, which we’ve named, with apologies to Jane Austen, biases of Pride, of Poverty, and of Prejudice.

In these cases, companies risk a lot more than just a few extra emails back and forth: wasted effort, failed launches, negative market feedback and other, equally dire consequences.

One oversight that we encounter quite often, in fact, has to do with not doing research at all. Here we’re referring to situations in which companies press forward with new product or service launches without first understanding the competitive landscape and/or market need for whatever it is they intend to offer.

Three Flavors of Bias

We’re all familiar, of course, with market research conducted to discover how satisfied customers are with products, services or experiences they’ve had with particular companies. Research conducted before the fact, however (i.e., prior to the launch of a product or service), is equally essential.

We can’t tell you how many times we’ve been invited to a kick-off meeting for research engagements that have been prompted by a product or service that had launched with disappointing results. Had the research been undertaken earlier – to verify what the internal product development or innovation team believed was true – everyone would have been better off.

In our experience, there are three types of biases that keep companies from conducting proper market research prior to development and launch:
  1. Pride Bias.

    Many companies skip the research phase because they believe they already know what the market needs. They’ll say things such as, “We know our customers,” “This will definitely be a winner,” or “Our market is dying for this service.”

    This is what we refer to as “pride bias” – a belief that research is unnecessary (if not somewhat insulting to us as experts in the field), due to one’s familiarity with the market. After all, if we talk to our customers every day and live and breathe our marketplace, shouldn’t we already have a good sense of what’s needed?

    Maybe. The problem is that there’s a difference between collecting hearsay from a non-representative sample and conducting unbiased, well-designed research. Maybe customers aren’t telling you what they really think. Maybe the topics that you discuss and hear about are just a small slice of the total picture. Maybe the structure of your distribution channel is causing you to view the world in a certain (flawed) way.

    Timing plays a role here as well. Company lore regarding “the marketplace” tends to evolve slowly, whereas trends can shift on a dime. Even if what you believe to be true was in fact true at one point, relying solely on “what we already know” puts you in the position of driving forward while looking in the rearview mirror.

    When you allow pride bias to drive research decisions – eschewing unbiased information in favor of your current understanding of the world – you open the door to a dazzling array of easily avoidable problems.
  1. Poverty Bias.

    You’ve probably heard the saying, “If you think education is expensive, try ignorance.” Well, the same could be said for not conducting pre-launch research on the grounds of it being too costly.

    When you consider how much time, money and effort will be expended on meetings, planning, design, development, production, marketing, public relations and more in an effort to bring a new product or service to market, spending a little bit to test the features and concept within that market is a small price to pay.

    Fixing problems after the fact – if not scratching a launch entirely – is invariably more expensive.
  1. Preconception Bias.

    Also known as confirmation bias, this refers to research that is conducted “with prejudice” – that is, with a prejudgment regarding the final outcome. Each of us comes to the table with a set of beliefs, of course. Unless these are understood and managed, however, these preconceived notions can sway both the development and interpretation of the research.

    We worked with a company, for example, that developed multivitamins. As we were designing the research, our client informed us that their advertising agency had a particular theme it wanted to test, for use in the roll-out campaign. In the course of our research (in-depth, one-on-one interviews supplemented with focus groups) the theme in question did in fact come up. And each time it did, the agency folks jumped for joy, citing the mention of the theme as “proof of the concept.”

    Upon further investigation, however, it was a non-starter. The theme was already strongly associated with a direct competitor and its mere existence in no way suggested it would work with our client’s product. The agency people had trouble hearing this, thanks to their confidence in the brilliance of their creative concepts.

    Research done badly can be as damaging to an effective rollout as research that is not done at all. Given the biases we all have – whether subconscious, tied to a particular agenda, or some combination – it’s in the best interests of all concerned to get these on the table up front and ensure that the work is conducted and interpreted objectively (see Mixology below for more on this).
Here’s the Twist: Every product or service rollout involves a great deal of time and effort on the part of those involved. It also carries with it a certain (sometimes huge) amount of market risk for the company itself.

While dice-rolling may be an amusing diversion in one’s spare time, it has no place in your next launch. Minimize the unknowns prior to rollout by sidestepping these three common biases.

– Jennifer

mixologyMixology (Putting Research into Practice)martini
Preconception bias may be the most dangerous of the three described above, since unlike the other two, this bias actually involves research. As a result, it can give team members a false sense of having listened to the marketplace.

To combat this, we offer three suggestions:
  1. Establish disprovable hypotheses.

    In the multivitamin example above, it was easy for people to listen for a particular word and, when it was spoken, declare victory, ignoring the way in which it was actually being used. In these types of situations, we will often develop alternative hypotheses (before the fact) and apply statistically-based measurements to determine which ideas or themes are the strongest.

    For example, rather than simply considering the utterance of a word as support for a theme, we will be more specific: How often is this word used in connection with our product vs. that of a competitor? In what contexts is the word used? Are there other words that are even more closely associated with our client’s products or services?

    Only if the differences are significant enough do we consider this to be real evidence for taking action.
  1. Enlist alternative viewpoints.

    Bringing in others who disagree with prevailing assumptions offers a balancing influence. These may be people within the company who hold a competing point of view or who are simply not steeped in the issues at hand.

    In an insurance company, for example, you may bring in someone who knows group life products and involve them in research related to a new dental plan. This type of fresh perspective will help to tease out preconception bias.
  1. Engage external, third party sources.

    At the risk of appearing self-serving, we hasten to point out the value in involving those with a broader perspective – beyond the company or even industry. In addition to their external viewpoint, third party partners may be more willing to challenge, say, a senior executive who may be championing a particular direction. In general, they are also not tied to a particular outcome the way internal team members may be.

    These individuals are likely to have a greater willingness to speak their mind and an ability to see things that may be obscured by company walls.
aboutUsAbout Us
The Center for Strategy Research, Inc. (CSR) is a research firm. The “Twist” to what we offer is this: We combine open-ended questioning with our proprietary technology to create quantifiable data. As a result our clients gain more actionable and valuable insights from their research efforts.
understanding what people really think
WordPress Lightbox Plugin