The Center for Strategy Research, Inc. Vol 5 Issue 6   September 2009


Hello!

Michael Jackson’s untimely death this summer shocked the world. It also served as a good example of why our most widely-used statistical tools – mean, median and mode, to be exact – may hide more than they reveal.

Please click “reply” to tell us what you think!


Julie Brown
President

Mark Palmerino
Executive Vice President



Don’t Be Mean to Michael
It’s always a bit of a shock when we learn that a well-known person has died; even more so when that person’s death comes without warning.

Indeed, for many of us, the untimely deaths of such American icons as President Kennedy, his brother Robert, Martin Luther King and Elvis Presley, were so unexpected, that we can clearly remember where we were and what we were doing when we heard the news.

Now, many are predicting that Michael Jackson’s surprise death this past June 25th will be added to this short list of “Where were you when you heard the news?” memories that we all share.

As for me, I was getting ready to shut down e-mail for the evening when the news arrived at about 6:00 p.m. here on the East Coast. The first thing I did was send e-mails to several friends and family members.

The responses started coming back almost immediately and I remember being struck by their passion and intensity:

Some were negative, with comments like “Good riddance” and “He only escaped jail because of fame and fortune.” Others, in contrast, spoke of shock, sadness and grief upon hearing that such an unmatched talent had died so young.

Everyone, it seemed, had an opinion. And every opinion — whether positive or negative — was a strong one.

As I reviewed the e-mails as they continued to arrive over the next few days, I realized I was witnessing a research phenomenon played out in real life. To be precise, had I asked this question…

“On a scale of 1 to 11, where 1 means you are not at all grieved at the passing of Michael Jackson, and 11 means you are very grieved at his death, what number best represents your reaction to this news?”

… I could easily characterize the responses as follows:

1, 1, 1, 1, 2, 2, 3, 9, 9, 10, 10, 10, 11, 11, 11

And, being a researcher, I started thinking about the meaning behind the responses, and what to make of them. I wondered how, statistically, they might best be analyzed and which if any of the three most common statistical tools — mean, median and mode — could be used to best represent what I was hearing.

The “mean,” of course, is a simple average: All values are added and the sum is then divided by the number of values. Applying this process to the responses above, we would calculate a mean of 6.1 (sum of 92 divided by 15 responses).

But as you can see, this tells us very little (in fact it’s downright misleading) regarding what my friends and relatives really thought. A mean response of 6.1 suggests a neutral response overall; not one of these people responded neutrally to this question.

A mean is also limited in its ability to be used as a comparative measurement tool. For example, had I asked a similar question regarding the other celebrity who passed away on that same day…

“On a scale of 1 to 11, where 1 means you are not at all grieved at the passing of Farrah Fawcett, and 11 means you are very grieved at her death, what number best represents your reaction to this news?”

… I suspect that the responses I would get, about a very beautiful woman who was seen by many as a mid-level actress with some emotional challenges, would be as follows:

5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 7, 7, 7, 11

(The 11 would be offered by my brother, who kept her ubiquitous poster draped across a wall in our basement for most of his youth.)

In the case of Farrah, and despite the very different feel of the responses regarding her death, the mean is 6.2 (sum of 87 divided by 14 responses) — nearly the same as Michael’s 6.1. And yet, to suggest that these two sets of data reflect similar realities would clearly be incorrect.

Unfortunately, the other two common calculations — median and mode — don’t fare much better:
  1. The median is the “middle” value in the list of numbers; the number where there are exactly the same number of values above the result as there are below. In the case of the King of Pop, the Median is 9; for Charley’s Angel, it is 6. Here as well, the median doesn’t add any illumination, and, in the case of the King of Pop, is just as misleading as the Mean.
  1. The mode is the value that occurs most often (if no number is repeated, then there is no mode for the list). In the Gloved One’s case, the mode is 1. For Farrah, there are two modes — 5 and 6. Certainly, in the case of the former, the mode is also off target.
Here’s the Twist: Simple calculations, while relatively easy to compute and present, often do more to hide the truth than reveal it. Relying on any one number (especially the mean) to explain complex situations, whether that be the reaction to Michael Jackson’s death or satisfaction with your products and services, can easily result in your missing a “Thriller” of a complete story.

— Julie

Click here to share this newsletter with a colleague.

At CSR, we believe strongly in the power of qualitative, conversational research, and in particular, in its ability to disambiguate the potentially contradictory or inconsistent direction that quantitative-only research often produces.

Here as well, Michael Jackson serves as a good example. Given the introduction to this newsletter, the timing of it relative to the recency of MJ’s death, and our repeated use of his many nicknames, we are probably all on the same page as to which “Michael Jackson” I’m asking about.

But, while he is certainly the most famous of the group, there are many (many) people, famous and not, who share the same moniker.

So, what if we posed the same question some day in the future, when his untimely death will (hopefully?) have ceased to dominate the headlines? At that point, a wide range of responses to the closed-ended question about reaction to the death of Michael Jackson could reflect a much more fundamental issue at work: i.e. “Which Michael Jackson?”

Simply asking, therefore, about how one feels about the passing of “Michael Jackson,” particularly as more and more time passes, could lead to some confusion:

We could continue with the rest of Wikipedia’s list of two dozen famous and infamous Michael Jacksons, but I’m sure you see our point: Quantitative results on their own often cannot tell the whole story, and are often dependent on contexts that are out of the control of the researcher.

Additional examination of quantitative results, clarified through appropriate qualitative support, is far more likely to provide the insight necessary to drive business decisions, than can quantitative results on their own.

 

Don’t Be Mean to Michael

Mixology (Putting research into practice)

Twist and Shout

About Us


We’re delighted to announce that Mark will be speaking at LIMRA’s Group Benefits Leadership Conference at Boston’s Marriott Long Wharf in September.

Together with one of our clients, Mark will discuss how our innovative approach to a compelling research issue helped this client better understand the challenges that members of key sales and distribution teams face.



He uses statistics as a drunken man uses lampposts — for support rather than for illumination.

— Andrew Lang



Problems? Click here to send us an email with your request.
The Center for Strategy Research, Inc. (CSR) is a research firm. The “Twist” to what we offer is this: We combine open-ended questioning with our proprietary technology to create quantifiable data. As a result, our clients gain more actionable and valuable insights from their research efforts.

 
Understanding What People Really Think


WordPress Lightbox Plugin