My company is frequently called on to conduct qualitative research projects that are fraught with peril. That is, to test advertising, concepts, packaging, retail store designs or other communications.
Disaster looms, with peril approaching from all directions. The agencies who have created the work are understandably on edge. Reactions to their creative output inevitably affect client relationships, and at times, dramatically so. The history of marketing abounds with “blame the agency” stories when work doesn’t test well or market share is declining through no fault of the advertising, logo or packaging.
That’s because client reputations are also on the line. Marketing people don’t want to report to management that months of agency fees and the costs of running the research have all been a “failure.”
This, of course, is ridiculous. Creative work done in good faith, responsibly and on strategy, is never guaranteed to connect with potential consumers. We push agencies to deliver “breakthrough” advertising, which by its very nature, is risky. The essence of innovation – as well as great art, advertising included – is disruption, forcing people out of their comfort zones. Therefore, the bigger chances we drive agencies to take, the larger the probability of failure.
Exasperating this problem is the high likelihood of snatching defeat from the jaws of victory. Sometimes success is right in front of us, but we just can’t recognize it.
This, in part, is rooted in the tendency – perhaps the very human need – for people to be critical. A recent article from the Atlantic, The Art Of Recognizing Good Ideas, And Why Managers Are So Bad At It, helps explain:
“There are two basic modes of judgment: criticism and praise. The former consists of identifying a subject’s flaws; the latter of noting its merits.
“In most settings, criticism tends to dominate. For any idea or book or movie or what have you, the question that people discuss is what’s wrong with it, why it didn’t live up to expectations. Often, one gets the feeling that the criticism isn’t dispensed in an effort to engage with the work but as a demonstration of the critic’s smarts, the implicit argument being that he or she is sharper and more discerning than the work’s creator.”
The author of the piece, Rebecca J. Rosen, goes on to talk about how the TV show Seinfeld was rejected by “executive after executive at NBC” before finally finding a champion and getting made:
“The managers…were too focused on whether Seinfeld looked like what had succeeded in the past to recognize its novel brilliance. Years of experience had trained them to believe that a certain type of show would be successful, and biased them against something that broke that mold.”
In the same vein, respondents could be looking at the next iPhone and never have a clue. If it doesn’t fit into their past experience or current worldview, critic mode can easily take over and lead to rejection of the new idea. Moreover, the very nature of inviting people to research and telling them how interested we in their opinions can easily be taken as cues to “act smart” and say “intelligent things.” Rosen’s observation that people want to seem “sharper and more discerning than the work’s creator” – not to mention fellow research participants – is in full force here.
Obviously, we need to know if something isn’t working. But we must acknowledge the human tendency to feel superior through criticism. There are many times when negative feedback is not a truly honest, emotional response to the work.
Exacerbating this bias toward negativity is the need for agencies and clients come out of research with “winners.” Walking away from studies where concepts or communications failed to connect with consumers can be a painful experience for all involved.
It’s interesting how well-conducted research, yielding incredible insights, is frequently deemed “bad” when creative work “loses.” We all might agree in principle that the true purpose of research to learn, but “winning” always seems to trump learning. And when reactions aren’t those that we had hoped for, the blame game begins. Clients blame agencies for producing “bad work.” Agencies and clients may both blame the researcher for asking the “wrong questions” or letting the voices of those vocal “negative” respondents drown out the “positives.” Unfair, unproductive and foolish all around.
We’re squeezed from one end by the need to “win” and the other by the ease with which respondents can get negative and downright nasty.
Fortunately, the right mindset, along with sound research techniques, can help us avoid this trap entirely. I’ve written about some of these methods many times in the past, most recently in 3 Ways to Co-Create With Consumers For Breakthrough Strategy & Innovation. I won’t go into them in detail again here, but the key is setting a workshop-like environment that requires active, constructive, respondent participation through creative exercises. Asking direct “yes” or “no” questions or soliciting “ratings” for stimulus can put get respondents to that negative critic place rather quickly.
The proper mindset for researchers, clients and agencies is equally important. Never one to pass up an opportunity to quote David Ogilvy, an excellent starting point is his wonderful observation on how marketing executives are reluctant to use their own judgment, instead relying on research “as a drunkard uses a lamp post for support, rather than for illumination.”
We should always agree on three, simple things at the outset of every study.
- Enlightenment is our goal. The research can guide us, but a superficial reliance on any qualitative or quantitative study to pick winners and losers is unwise. Moreover, the definition of “great research” is “great learning,” not just that what we tested “won.” Conversely, it’s not bad research if our concepts “lost.”
- “Failure” is always an option. We pay a lot of lip service to this notion, especially when we talk about innovation. Article after article is written on how it’s not just okay, but healthy to fail, for without failure there is no learning. It’s one of those beliefs we love in theory but seldom in practice. Sure, agencies are capable of churning out some bad work. But if you’re the client, you’ve acknowledged the work is on strategy and you’ve given your approval to take it to research, then you own that work as much as the agency does, “good” or “bad.” The learning itself must be appreciated. It would be wonderful if one round of creative work was all it ever took to get it right, but that’s not realistic, especially if we’re taking chances with the work.
- Go into qualitative research with an understanding of human nature and a plan to maximize passionate, constructive participation that will minimize the tendency for respondents to play “marketing critic.” Straightforward questions don’t work. Make sure you go into the research with well-thought out, provocative stimulus, and leverage creative exercises to get people “thinking out loud” in order to get beyond the obvious and truly understand their motivations.
