The Center for Strategy Research, Inc. Vol 8 Issue 1   February 2012


The Super Bowl is front and center this week, and as usual, there are plenty of market research lessons to be mined.

Today’s edition takes a look at a recent high-profile missed kick and what it suggests for those of us who tackle data rather than people for a living.

Julie Brown

Mark Palmerino
Executive Vice President

Avoiding the Choke: 3 Suggestions for Better Research
The Choke. It’s a classic sports paradigm.

The wide-open missed layup at the end of a close basketball game. The two-foot golf putt that veers wide on the final hole. The easy groundball that rolls through the first baseman’s legs, when instead, it should have been the game’s final out (don’t get me started).

And now, with Super Bowl XLVI on tap for Sunday, February 5th, you’ve probably seen or heard about The Choke’s most recent incarnation: Baltimore Ravens kicker Billy Cundiff’s miss of what appeared to be a “routine” field goal in the waning seconds of the AFC Championship game on January 22nd, which ended in the Ravens’ 23-20 loss to the New England Patriots.

A missed kick that, instead of sending the game into overtime, sent the Ravens to the locker room as losers.

And while there’s no shortage of opinions regarding what really happened in those final seconds to cause the miss, we have three of our own observations, each of which has a parallel in the world of market research:
  1. Keep your head in the game.

    Cundiff says the scoreboard didn’t indicate the correct number of downs remaining. Some players say the coaches kept calling for Cundiff and he wasn’t immediately ready. Whatever the reasons for the last-minute confusion, it’s inexcusable at this level of play and at this point in the season.

    As professional researchers, we also need to minimize confusion in the work we do, particularly at points where specialists get involved and/or the “ball changes hands.”

    For example, do you give your analysts plenty of notice regarding the delivery of data and expectations for turnaround, or do you drop it off at the last minute with a request that it be “ready in the morning?”

    Do you keep senior people involved in the process from start to finish (we do) or do you hand off along the way to less experienced staff with the hope that nobody fumbles?

    Keeping your head in the research game means playing as a team and making sure everyone is clear on their roles, clear on the timetable, and clear on expectations.
  1. Don’t be afraid to call a timeout.

    In postgame interviews, Ravens coach John Harbaugh confessed that the idea of calling a timeout, “never occurred to me.” He wasn’t aware of the problem. Ironically, standing on the opposite side of the field and outside the immediate fray of the Ravens bench, the Patriots coaching staff noticed the confusion and smartly decided not to call time out.

    As researchers — particularly in today’s instant, 24/7, social media-enhanced environment — we’re under pressure to deliver things quickly, to meet a schedule.

    The problem here, as in football, is that when anyone does research under pressure, it’s easier to make mistakes and to miss what should be obvious danger signals. In one’s eagerness to get a survey into the field, for example, important questions can be missed or programming might not be done quite right.

    But the volume and ubiquitous nature of data available today doesn’t by itself improve research results. The instruments created, the process developed and the analytics applied need to be well-considered, a step which invariably takes time.

    The Ravens didn’t think to call time out, despite the obvious confusion. Do you?
  1. One step at a time.

    32-yard field goals are easy (well, for professional football kickers, anyway). As Cundiff offered after the game, “It’s a 32-yard field goal, I’ve probably made thousands of those [in games and practice].”

    But he missed it when it mattered. Did it seem too easy to him? Was the coaching staff already focused on its overtime strategy? Whatever the team was thinking, it really isn’t all that surprising — it’s natural to look beyond the routine, albeit necessary, task at hand.

    Research also has a set of foundation elements that can easily be missed. I received an e-mail from a hotel company, for example, asking me to comment on my stay. I’d never been there (I cancelled at the last minute), but nowhere did they confirm that I had in fact been a guest.

    Appropriate vetting of participants, accurate analysis of data, and properly structuring skip and logic patterns in conversations and surveys are just three examples of simple research fundamentals that, if missed, are capable of sending your project to the locker room.
Here’s the Twist. Unlike football games, most well-planned research projects aren’t won or lost with game-saving tactics in the final seconds. That said, with the application of a few lessons and insights from the world of sports, we can avoid becoming the next headline-grabbing choke of the day!

(And, oh yeah, Go Pats!!)

— Julie

Click here to share this newsletter with a colleague.

One practical and time-tested means of obviating the need for last minute heroics is the “slow launch.”

The premise is simple: Rather than go out at full speed from the start with a research project, only to uncover flaws in approach or methodology after the fact, the slow launch begins by fielding just a portion of the study. This smaller set is then reviewed, allowing for adjustments before going to full launch.

Things to keep in mind:

  • Build it into your schedule. A slow launch and the subsequent analysis and adjustment can add two or three days to the process. By explicitly building it into the research calendar from the start, you relieve much of the pressure to “just get it going” which may be there come launch time.
  • Shoot for 5–10% of total completes. While you’ll likely gain some insight into final results, that’s not the purpose of the slow launch… it’s to test what you’ve created. You need enough participants and variation to test all branches of the instrument accurately.

    Too few people and you’re not properly testing. Too many and you’re “burning” participants whom you may have to throw away if significant changes are made. In our experience, 5–10% tends to be about right.
  • Take the instrument through the paces. Things to look for include errors in programming; surveys timing out too soon or too late (either can indicate mistakes in your assumptions); and high dropout rates.
  • Become a habitual slow launcher. While there are circumstances in which this initial step is not necessary (e.g., very simple surveys, surveys that are essentially identical from year to year), we do these as a matter of course. A slow launch is ideal for uncovering problems, making adjustments and, when necessary, calling a time out!


Avoiding the Choke: 3 Suggestions for Better Research

Mixology (Putting research into practice)

Twist and Shout

About Us

Patriots' USB

And speaking of winning Patriots drives, we’d like to send you a winning Patriots drive. Specifically, an NFL-themed USB drive for the team of your choice.

To be considered, simply click “reply” and send us your prediction for the final score of Sunday’s Super Bowl, along with the name of the team that should be proudly displayed on your new drive. The three closest guesses will receive the drive of their choice within 2 weeks of the Super Bowl’s conclusion.

(Please note: Giants USBs are known to have a high failure rate and are easily crushed.)

“Finish last in your league and they call you Idiot. Finish last in medical school and they call you Doctor.”

— Abe Lemons

Problems? Click here to send us an email with your request.
The Center for Strategy Research, Inc. (CSR) is a research firm. The “Twist” to what we offer is this: We combine open-ended questioning with our proprietary technology to create quantifiable data. As a result, our clients gain more actionable and valuable insights from their research efforts.

Understanding What People Really Think

WordPress Lightbox Plugin