Research with a Twist
blue lineVol. 9, Issue 5, July 2013
 CSR - Center for Strategy Research
 In This Issue…

twistAndShoutTwist and Shout


And speaking of worthwhile events, look for us at LIMRA’s 2013 Group and Worksite Benefits Conference in Philadelphia, this September!

 Quote of the Month
“In God we trust, all others must bring data.”

– W. Edwards Deming 

 click here to subscribe to research with a twist
Questions? Click here to send us an email with your request.
(617) 451-9500


“Big Data” is all the rage. Some people have even suggested that Big Data, with its ability to offer so much information, so quickly, signals the end of “traditional” market research.

As you might have guessed, we don’t agree. Today’s edition of Research With a Twist explains why.
signature - Julie
Julie Brown

signature - Mark
Mark Palmerino

Executive Vice President
Julie and Mark
articleOneHoney, I Shrunk the (Big) Data
We returned home last month from LIMRA’s 2013 Marketing and Research Conference. It was held at Disney’s beautiful Yacht and Beach Club resort in Orlando and, as usual, it did not disappoint (in either the learning or the entertainment department).

Among the many themes circulating throughout this three-day event, one that we heard and talked about quite a bit, was “Big Data”: What it is, why it matters, what its impact has been and will be on the market research industry.

Regarding that last point, there was even some suggestion that with so much information now “widely available,” Big Data will soon put an end to the need for market research. We don’t agree.

Why not? Well, the way we see things, there are (at least) four significant limitations to replacing market research with Big Data, and regarding its use as a reliable, predictive tool. More on this below, but first, a little background on what Big Data is and why it’s become so popular in recent years.

What is “Big Data” and why does it matter?

In short, the term “Big Data” refers to the exponential growth and volume of information being created and captured across nearly every aspect of our lives. From finance, to health care, to packaged goods and beyond, most of the transactions that we as consumers are involved in on a daily basis are in (or can be converted to) digital form and, therefore, are available for analysis.

Big Data is the term used to describe all this information. From a market research perspective, its existence raises the question of how it can best be used to understand past activities and predict future behavior.

For those who have been involved in the areas of data mining and predictive analytics over the last several decades, the concept of Big Data may sound familiar. What’s different now, however, is the volume, scope and speed at which data is gathered, thanks to the combination of increased computing power together with the ubiquity of communications and information via social media, online transactions, mobile devices and the like.

And so, the thinking goes, maybe we no longer need formal, “traditional” market research. After all, with so much data at our fingertips, all we need to do is organize it, analyze it and use it to guide decisions.

To which we say, “Not so fast.”

Because while it’s true that there are now mountains of information growing exponentially every day, the sheer volume of data doesn’t by itself lead to insight. Indeed, one need look no further than the ongoing NSA scandal to see the inherent disconnect between collecting information (legally or not) and using it intelligently to predict behavior (e.g., the Boston Marathon attack).

Specifically, here are some reasons why Big Data as a substitute for market research falls short:
  1. It’s based on historical behavior.

    Big Data is all about what’s already happened: What was purchased, who searched on a particular phrase, which tweet themes are trending, etc. It’s broad, absolutely, but inherently shallow.

    So, for example, American Airlines may be able to calculate the number of New Yorkers who purchase tickets to Panama after conducting a Google search on “winter vacations.” The danger arises, however, when AA then attempts to take action – to boost sales, for example – based on this information. There may be any number of reasons why people aren’t buying more – Price? Political unrest? Panama was replaced by Costa Rica as “the” destination for “beautiful people” last week? There’s no way to know from past behavior alone.

    Big Data‘s bias towards the past is particularly limiting when working with products, services or concepts that are “disruptive,” i.e., fundamentally different than what has come before. Historical behavior is of little help, for example, when estimating the future actions of a competitor; contemplating the impact of changes in the business environment; or evaluating a product idea that’s never been thought of before.

    The point is, and just as your financial advisor warns that “past results are not a reliable predictor of future returns,” simply knowing that someone did, said, or purchased something in the past cannot always be expected to reliably tell you what he or she is likely to do next. For this, and particularly in a world where things change quickly, we need the kind of robust and consistent tools that market research brings to bear.
  1. The samples may not be representative.

    With social media-based data in particular, it’s easy to get caught up in the appeal of listening in on thousands – or millions – of virtual conversations as a means of understanding “what’s really happening.”

    But this channel is by no means representative of the general population. Not only does it skew towards certain demographics, our inability to screen participants based on their actual behavior – witness, for example, how TripAdvisor has felt the need to “certify” that those providing ratings for hotels actually stayed at the location they are rating – prevents us from verifying the validity of comments and suggestions made.

    Add to this the fact that online conversations are easily swayed by the voices and opinions of the loudest and most passionate, and you’ve got the potential to make critical business decisions based on what amounts to an unmoderated, unrepresentative, untested, online focus group.
  1. The insights are only as good as the quality of the information.

    Much of the data being analyzed today comes from the internal systems of the companies that are using the information. The problem, of course, is that the data may not be accurate.

    We worked with a client a few years ago, for example, in which half its sample data regarding purchase patterns was wrong, something we only discovered when we began conducting one-on-one interviews with customers. Had the company’s internal data been taken as fact, any analytics and decisions resulting from review of that data would have been inaccurate.

    Without the checks and balances that are generally built in and are fundamental to the way quality market research is conducted, information is presented and used at face value, and subject to errors in packaging, data entry, salesperson documentation – basically, any error that can be made by any individual along the customer experience route.
  1. It’s hard to get to “Why.”

    Big Data is massive, but generally very thin. As market researchers working with clients to help them generate insights and make intelligent business decisions, we want the opposite (Little Data?): Profound and in-depth.

    Remember, our goal is not number crunching, it’s insight – understanding the “Why?” There are only so many hours in the day to work on a particular question or project and, if you spend 7 out of 8 gathering, organizing and slicing data – and only one developing the “A-ha’s” that flow from it – you’re disproportionately favoring the wrong side of the equation.

    In-depth research – particularly qualitative, one-on-one – is overwhelmingly focused on the desired end result: Making connections and putting the pieces together in an intelligent way, so that businesspeople can use these insights to take informed action.
Here’s the twist. Big Data is not entirely without merit – but, as described above, it’s far from a panacea and its existence is in no way a signal that market research is on its way out.

In practice, we recommend an AND rather than an OR approach: Tap into what Big Data has to offer, but keep these four limitations in mind as you move forward.

mixologyMixology (Putting Research into Practice)martini
As researchers, we need to ensure that we are not caught up in the whirlwind of Big Data analysis and, instead, that we leave enough room to develop valid insights.

Two suggestions:
  1. Don’t confuse volume with understanding. Understand the limitations of Big Data and stay focused on the insights you seek.
  1. Use Big Data for its strengths. While weak in the area of “Why,” Big Data can help quite a bit with the other 4 W’s: Who, What, Where and When. When used strategically you can, as one LIMRA conference attendee shared, find answers to questions you didn’t even realize you had.
aboutUsAbout Us
The Center for Strategy Research, Inc. (CSR) is a research firm. The “Twist” to what we offer is this: We combine open-ended questioning with our proprietary technology to create quantifiable data. As a result our clients gain more actionable and valuable insights from their research efforts.
understanding what people really think
WordPress Lightbox Plugin