The Center for Strategy Research, Inc. Vol 1 Issue 3   November 2005


Welcome!

Research projects, like Thanksgiving feasts, have a tendency to attract a lot of participants. Today we talk about why, in both cases, it’s important to limit the size of the guest list.

As always, your thoughts and comments are appreciated.


Julie Brown
President

Mark Palmerino
Executive Vice President



How Many Guests Is Too Many Guests?

Before I was married, I lived in Charlestown, Massachusetts. I owned a small condo in this parking- and privacy-deprived part of Boston. How small? Let’s just say that a cocktail party of 10 would necessarily include claustrophobia. And yet one year at Thanksgiving, a friend and I had the brilliant idea that we would entertain 16 guests for dinner.

Not long after this idea was hatched, 16 grew to 24.

Distant cousins were visiting from out-of-town; someone asked a best buddy who would otherwise be alone; my brother, who had said he planned on going to his in-laws’, just couldn’t bear to miss seeing his sister make a fool of herself entertaining such a big crowd for her first Thanksgiving.

On the night before the big day, as I attempted to place 24 rented chairs around a too small rented table, I knew I had a problem. While this kind of scrunched together arrangement is familiar to many of us who grew up with large extended families, it was not quite the stuff of Martha Stewart Living.

Thankfully, it all worked out in the end. A few people didn’t show up, and there was enough wine and laughter flowing to mask the overcrowding. Still, as we debriefed the next day, my friend and I agreed on one thing: There were just too many people at the table that year.

By the same token, inviting too many teammates to take part in the implementation of a research project is the research manager’s equivalent of too many guests at the table.

We all recognize that building internal consensus is critical to conducting research that is organizationally useful and broadly accepted. But, the challenges that arise when you’ve got too many people involved include:

  1. Ineffective questionnaires. What often happens when you invite a number of smart, experienced executives to participate in survey construction is that each has his or her own sense of how issues should be approached and/or how questions should be worded. After a short time, it becomes more politic for the questionnaire to incorporate “everyone’s” ideas, rather than single out one as the best and only.

    The result: a questionnaire that is redundant (the same issues approached from different directions), frustrating for the respondent (who perceives the same issues being explored over and over again), and inefficient.

  2. Project delays. Involving a large number of people in survey design results in delays in three critical ways:
    • The team generally needs to wait until everybody can “get to” reviewing the survey.
    • Reconciling the edits and comments of more than one person takes additional time.
    • Team members without a background in market research may not understand how the survey outcomes may be used. This is especially true when the survey design involves more specialized research techniques, such as conjoint analyses. In such cases, additional time is needed to ensure that everybody understands the possible outcomes and their potential uses.

  3. Internal dissent over final results. A few years ago, we worked on a research project where some team members were not fully informed about all the applications and outcomes of a particular research process. Because of this lack of understanding going in, the final results were difficult to explain and gain acceptance on by the team, and organization, at the end of the project.

In the same way that we don’t want to seat 18 at a table that comfortably fits 12, completing a research project effectively often means limiting the frequency of feedback, the timing of the feedback, and the number of “feedbackers.”

This is not to minimize the importance of consensus-building. That said, if you want timely and in-budget completion of your research projects, be sure to plan for the involvement of teammates as you build your budgets and calendar.

— Julie

Click here to share this newsletter with a colleague.

Mixology (Putting research into practice)

And, while we’re on the topic of how many people to include in designing the research, how many respondents should we invite to the research “party” itself? What are the rules of thumb for determining the minimum number of respondents for each sub-group, or cell, in a study?

The short answer is that when comparing two groups, 30 people per cell or sub-group is about right.

Why 30? Good question. 30 in each group is generally large enough to give you a reasonable chance of detecting a difference between the two groups and yet not so large that you’re wasting resources.

In the realm of statistics, groups of this size leads to an 80% (more or less) confidence level. This is another way of saying that if there is in fact a “moderately” large difference between the two groups in the population, we have an 80% chance of actually detecting it. Not perfect certainly, but four out of five times, accurate. If 80% isn’t good enough, increasing the numbers to 50 per group will net you nearly 100% chance of detecting a moderately large difference.

Of course, there are all kinds of “ifs,” “ands,” and “buts” that impact this number, but it’s usually a pretty good place to start. No matter how you slice it, there is always a direct correlation between group size and accuracy, and of course, a proportionate impact on cost.

— Mark

 

How Many Guests Is Too Many Guests?

Mixology (Putting research into practice)

Twist and Shout

About Us


An article entitled “Efficacy of Treating Osteoarthritis with Chiropractic Care,” co-authored by Mark Palmerino, has been accepted for publication in the Journal of Manipulative and Physiological Therapeutics (JMPT).

Mark has worked (and continues to work) with the primary investigator, Dr. Kathleen Beyerman of Winchester Hospital, on numerous projects.



“Whenever I find myself in the cellar of affliction, I always look about for the wine.”

— Samuel Rutherford, 1600’s Scottish minister



Enter your email here to subscribe to “Research with a Twist


Problems? Click here to send us an email with your request.
About Us
The Center for Strategy Research, Inc. (CSR) is a research firm. We combine open-ended questioning with our proprietary technology to create quantifiable data.

 

(617) 451-9500
Understanding What People Really Think


WordPress Lightbox Plugin