The Center for Strategy Research, Inc. Vol 4 Issue 8   September 2008


It’s September, and with the presidential election less than two months away, today’s Research with a Twist explores why good research, like accurate polls, requires more than just one cut at the data.

As always, please click here to send us your thoughts and comments.

Julie Brown

Mark Palmerino
Executive Vice President

Research, Like Election Polls, Demands Repetition

The political conventions are (thankfully!) over and as we turn the corner away from summer, we all know what’s coming next: Lots of debates, a barrage of campaign ads, and 24/7 coverage of everything from Sarah Palin’s sporty glasses to Joe Biden’s hair — with the occasional issue of substance thrown in for good measure.

We’ll also hear about “The Polls.” Lots of them. Different hypotheticals with a variety of permutations… all intended to uncover the answer to a very simple question:

“Who will be the next President of the United States?”

It’s so simple a question, in fact, that it begs another: Why not just conduct one poll, one time, and be done with it?

In other words, if we want to know who’s going to be elected in November, why can’t we just ask today, get the answer, and go back to watching football (Go Patriots!) as we do during every non-election year autumn?

It’s a rhetorical — even ridiculous — question, of course. And yet digging in a bit to the reasons beneath the value in conducting political polls over and over, offers important insights into why we should also consider conducting market research studies more than just once.

The problem with conducting just one presidential poll is…

  • People change their minds when presented with new information. I made up my mind about whom I would vote for months ago and at this point, it would take a train wreck (with my candidate driving) to send me the other way. Others, however, will remain near or on the fence until the very end. As new information is revealed — witness the recent Sarah Palin buzz — those in the middle may be swayed.

    The presidential landscape tends to breed more and faster new information than, for example, the car-buying marketplace. That said, it’s easy to see how the spike in the price of gas over the past 12 months has had a significant impact on the way a 2008 consumer makes a vehicle purchase decision. Research conducted in mid-2007 in this regard may no longer be valid.

  • People change their minds when the issue is emotionally charged. Even if the facts have been out there all along, when a topic (or candidate or product) feels personal, all kinds of bias comes into play. Deep-rooted personal baggage related to race, gender, money, religion and more rises to the surface. As people dig deeper and discuss issues with friends, relatives and others, rapid and significant swings in opinion may occur.

    If you’re conducting market research on an emotionally charged topic — questions about how people make financial planning decisions, for example — you may also want to test the waters more than once. Here as well, news regarding the economy, the war and the cost of health care, to name just a few, can move opinions quickly.

  • Sampling error introduces uncertainty. Sampling error relates to the statistical likelihood that the research results you’re seeing reflect the reality underneath. And while the press likes to turn small, temporary leads in the polls into large, blaring headlines, the truth is that many of these indications of “who’s ahead” often fall within the error range.

    Conducting polls or market research more than once, over time, can mitigate some of the sampling bias uncertainty. (See Mixology below for more on the concept of margin of error and sampling bias.)

  • What you say and what you do may not be the same thing. Political polls, by their very nature, ask people to predict a future action that they plan to take. What the polls don’t reveal, however — and this is in no small part due to the closed-ended nature of nearly every poll — is how likely it is that a stated intention will lead to actual behavior.

    Whether due to a reluctance to reveal a prejudice (“I don’t want to admit that I would never vote for an African American… a woman… a senior citizen…, etc.”), or simply a last minute change of heart, saying and doing are not the same thing. Multiple polling, with questions asked in different ways, attempts to close the gap.

    Longitudinal market research, particularly the kind which asks “Why?” in addition to “What?”, can also add more insight into how people will behave when faced with a buying decision.
A presidential election, with its once-every-four-years, winner-take-all construct, is admittedly more dramatic than the typical consumer or business purchase decision. And while a presidential poll is not market research, the need for taking more than one snapshot of a rapidly changing world makes sense in both cases.

Best of luck to your candidate in November (assuming you’ve even decided yet)!

— Julie

P.S. Do you already know for whom you’ll be voting in the presidential election? Click “reply” now to tell us, Yes, No or Undecided!

Click here to share this newsletter with a colleague.

In a presidential election, with no middle ground between winning and losing, a few percentage points in a close race can make all the difference.

That’s why it’s important when interpreting poll data — frankly, any research data — to understand the “Margin of Error,” a concept which relates to the degree of certainty of results.

For example, when you look at this Gallup Poll data regarding the McCain / Obama race, you see up at the very top that the Margin of Error is 2% (“MoE +/- 2”).

Statistically speaking, this means that any number associated with either candidate could very well be 2 points higher or two points lower than indicated. A 45% score, for example, is really a four point range, from 43% to 47%.

Now look at the daily race. Given the 4% range for each man’s numbers, a given candidate only has a statistically significant lead when he moves more than four percentage points ahead of his opponent. As you can see, leads of that magnitude, going back more than a year, have rarely existed and never for long.

Ignore the polls? No way. Just make sure the lead in the headline is supported by the data.


Research, Like Election Polls, Demands Repetition

Mixology (Putting research into practice)

Twist and Shout

About Us

Next month, on the morning of Tuesday, October 14, Mark will be co-presenting with Neil Marcus (MetLife’s Director of Institutional Business Market Research) at The Market Research Event in Anaheim, California.

The presentation, “Measuring Satisfaction of Your Largest Clients: How to Do It Right,” focuses on best practices related to how MetLife measures satisfaction, delivers an outstanding client experience and produces highly useful information for internal staff in the process.

Stop by and say hello if you’re attending the event!

“A new poll showed that if the election were held today, people would be confused because it is normally held in November.”

— Kevin Nealon

Problems? Click here to send us an email with your request.
The Center for Strategy Research, Inc. (CSR) is a research firm. We combine open-ended questioning with our proprietary technology to create quantifiable data.

Understanding What People Really Think

WordPress Lightbox Plugin