by Bob Walker | Aug 10, 2019 | Business strategy, Marketing research, NPS
In May 2019, The Wall Street Journal published a comprehensive article on the use – and misuse –of customer satisfaction programs that heavily rely on a single measure known as the “Net Promoter Score” or NPS*. Titled “The Dubious Management Fad Sweeping Corporate America”, the authors shared a review of 40,000 earnings call transcripts for 688 companies that are or were in the S&P 500 from 2003-2018. Last year (2018) the words “net promoter” or “NPS” were cited more than 150 times in conference calls by 50 S&P 500 companies. That’s four times as many mentions, and three times as many companies, from five years earlier. Last year, “net promoter score” was mentioned in 56 proxy filings. Some companies, including American Express, Best Buy, and Citigroup list NPS as a compensation metric, alongside share price and EPS. So what is really going on here?
Regrettably, the Net Promoter Score or NPS has morphed into something well beyond its scope. Originally intended to foster product and service improvement, many corporate leaders now treat NPS as a pseudo-financial metric. Management seems to believe that they must follow peer group practices, including the use of benchmarks collected by their own competitors. Misinformed stock analysts add pressure by implying that companies who do not use NPS must be out of touch with their customers.
The companies that continue to use NPS as a mantra are apparently oblivious to its corrosive effects. Much like “teaching to the test”, they ignore the fact that the measure can be manipulated. For example, a recent trip to my local Honda service center produced a text message from the Service Manager (see inset). He implored me to give him high ratings. What larger goal does this accomplish? Knowing that a survey is coming, management is unwittingly diverting his time and creating anxiety – and missing valuable insight into my service experience. And what if the initial technician who took my car had an attitude? Do I tag the Service Manager with that? A far better approach would have been to ask: “What aspects of your visit to Honda need to be changed or improved?” Or, more specifically, probe on multiple dimensions of my visit so that the employee is not associated with my evaluation of the entire experience.
Of course, the error can go in the opposite direction: companies can collect feedback that is too narrow and not actionable. Another example: I recently had to ship something, so I went to Staples and bought some bubble wrap to protect my item. Staples sent me a customer satisfaction survey asking me about my “bubble wrap experience”! Seriously?
NPS May Capture Intent, But Not Actual Behavior
Regrettably, management teams incorrectly believe that they need just one measure (NPS) to predict sales growth, profitability, and retention. This misperception also diminishes the role of insights professionals and research departments who are often closest to the customer. Many of you may recall that the NPS buzz began with Fred Reichheld’s 2003 HBR article “The One Measure You Need To Grow”. His quote: “The only path to profitable growth may lie in a company’s ability to get its loyal customers to become, in effect, its marketing department” became gospel. A “willingness to recommend” formed the basis of the NPS question, and an entire industry was born. The assumption was that a recommendation can drive both growth and retention.
This is certainly plausible — if such recommendations are ever made — but recommendations are, in fact, rare and intent is not the same as action. Other self-reported measures of behavior, such as purchase intent, have long been known to be directionally associated with in-market behavior, but the magnitude varies tremendously and is highly sample-dependent. Over the last 15 years, research has also failed to prove a clear connection with one’s willingness to recommend and profitability, growth, or retention. And the assumption that any of this can be accomplished at near-zero cost is also dubious.
Common sense must be used, and adjustments made, as companies learn more about their customers and business performance. Virtually all businesses require more than a single measure to understand customer needs. As I have previously written (“Rethinking NPS”), customer feedback is essential; it is NPS that is inadequate.
It seems that nothing will be able to stop the NPS locomotive. According to the WSJ, J.D. Power has signed an agreement with Bain & Company to become the officially recognized authority for benchmarking the “Net Promoter Score”. Much like J.D. Power’s customer satisfaction awards in automobiles and other categories, can “NPS awards” be far behind? In its next life, NPS will no doubt morph into support for advertising claims and other puffery. For devotees of W. Edwards Deming, this should feel familiar: NPS will become a slogan, and more manipulated than ever.
Have Courage: Use Measures That Matter
But what to do? If you are in leadership, focus on how internal processes can be improved, rather than (mis)placing the responsibility on individuals or your front line staff. An employee is responsible for bringing problems to leadership’s attention, but your job is to improve the process itself. Be wary of salespeople claiming to offer magical and simplistic answers to difficult problems. Look at the big picture and choose from appropriate (multiple) measures linked to customer happiness. Build your own normative database to best meet the needs of your unique business. One size does not always fit all, and it will cost money to truly understand what is going on with your customers and your business. But at least you will have measures that make sense.
At Surveys & Forecasts, LLC, we have been doing just that for 25 years.
* [1] Net promoter score or NPS is a registered trademark of Satmetrix Systems, Inc. a privately held experience management company headquartered in San Mateo, CA. Along with the Bain & Company and Fred Reichheld, Satmetrix Systems co-developed the Net Promoter Score (NPS).
by Bob Walker | Jan 5, 2019 | Marketing and strategy, Marketing research, Strategic research
I always tell my children that life is not linear. Neither is business. Things rarely go in a straight line, and companies can never sustain super-normal growth in either sales or stock price. Yet we were somehow surprised this week that Apple reported significant declines in iPhone sales. But should we have been that shocked? Apple has been making superficial changes to the iPhone now for several years. Didn’t Apple urge us to “think different”? Had they done any research to anticipate this obvious shift in consumer preference?
I continue to be amazed at the perceptual blinders that companies, or financial analysts, wear. The pattern is very familiar: management ignores any initial signs of trouble, they reassure themselves (and consumers, employees, and financial analysts) that all is well. In Apple’s case, they try to deflect by saying “look over here” (for example, by refusing to release unit sales). Denial and the refusal to believe the facts are powerful forces of inertia in every organization. Denial is a basic human instinct.
It is also the innovator’s dilemma (see Clayton Christensen): entrenched interests force managers to set aside obvious facts in favor of perpetuating the status quo. History is replete with examples. Hewlett-Packard never wanted to introduce ink-jet printers (the margins were too low). CVS, not doctors, conceptualized Minute Clinic (primary care that was “good enough”). Intel is overshooting processor power even though most consumers will never need it (although AI may change this dynamic). Chevy recently dropped the Volt, an electric car for which there was minimal underlying consumer demand. And why didn’t Sears become Amazon?
Apple is a great company and they will probably recover. A reset was needed eventually. But it raises the larger question of why companies stubbornly ignore facts about their customers, markets, and competitors? Why do managers assume that they already know everything that could potentially affect their business?
Great research is needed now more than ever.
I believe that it comes down to one basic factor: fear. But what does that really mean? Fear of having to think outside of normal patterns… a fear of disruption, which may force people to behave differently, which can be uncomfortable… a fear of having to adapt to a new way of working… a fear of a job change, or even worse – a job loss.
But you can’t anticipate or plan for what you can’t see. Don’t assume that you know everything that might affect your business. If you are in a position to recommend marketing or customer research, advocate strongly for it. Your company’s survival depends on seeing what is happening out in the real world!
I suggest a few simple rules for paying attention to what is going on in the world:
-
Actively listen! Talk to as many of your customers as you can, qualitatively or quantitatively.
-
Construct honest and non-biased questions to uncover real understanding, and identify the real dynamics in your market.
-
Validate your findings with enough respondents to give you the confidence you’ll need to make the hard decisions. Individual or small group feedback is a great start, but always verify, verify, verify.
-
When doing external (non-customer) research, work with reputable sample providers who have the expertise to manage consumer or business panels. To use a gardening analogy, the sample is the soil. If the sample is wrong, everything else is compromised.
-
Have an opinion about what your research results mean and implications for the business. Get out in front of the story; add value. If you abdicate your role, others will write the story for you – and then wonder why you’re there.
-
Let management question the results – but also make sure that they accept the findings. Management buy-in is essential.
Management philosopher Peter Drucker preached that all organizations have two functions: innovation and marketing. Marketing research is at the heart of both. Never stop asking questions, and do so with the frequency required for your industry.
Follow these simple guidelines and you will have an insightful, productive, and happy new year!
Let’s continue the discussion.
by Bob Walker | Nov 13, 2018 | DIY software, Marketing research
So by now many of you have heard the news: Qualtrics is being bought by Germany’s SAP for $8 billion in cash. I also received an email from Ryan and Jared Smith announcing the news.
But what exactly did SAP get for its money?
For the first nine months of 2018, Qualtrics generated $289.57 million in revenue ($386 annualized), up 40% from a year earlier. It posted a $2.3 million operating profit in the same period ($3.0 million annualized), nearly double its year-earlier operating profit (but is still paltry). The SAP price values Qualtrics at 14x forward revenue (not earnings). Based on earnings and an $8 billion price tag, Qualtrics’ PE is 2700. SAP itself has a PE of 24. The market seems to like it; SAP shares were up about 3% today. But does this make sense?
There are only so many ‘large’ customers ($100K+ in fees a year) that can continue to fuel a 40%+ growth rate. Much will depend on how much running room Qualtrics gets from SAP. The love fest will last a quarter or two, but soon the vulture capitalists will get paid, the growth curve will flatten, and the cost accountants will swarm in. Then redundant positions (in sales, marketing, and support) will be eliminated. Prices will also go up, and the quality of support will decline, as staff reductions kick in and the acquisition costs weigh down SAP’s P&L. It’s a familiar story.
Culture also matters. With SAP’s 95,000 employees and German influence, the Qualtrics hipster culture will fade. The relatively few Qualtrics employees can’t swim against the SAP tide for long. And there will be many companies that will now see Qualtrics as a conflict: those using Oracle, Microsoft, or Salesforce won’t want SAP snooping around, and thus they will hesitate. Many SMBs who may have considered Qualtrics will now have second thoughts, as they never envisioned bringing in such a behemoth.
The $8 billion price tag is an amazing achievement – congratulations to all. However, Qualtrics as a brand will persist only as long as SAP leaves it alone.
But it’s impossible not to play with that shiny new toy they just paid so dearly for.
by Bob Walker | Oct 3, 2018 | Marketing and strategy, Marketing research
There is an old joke about sharing research results: management’s response is either “I could have told you that”, or “I don’t believe it”. Humorous, but too often true.
Yet if a company has spent serious money on research, perhaps management should listen more closely to the full story. As researchers, we know all too well that any research project will usually confirm some obvious things. For example, a brand’s overall position or market share, key benefits, or common usage occasions are standard by-products of many studies.
But what happens when we peel back the layers a little more, and drill into more subtle differences? That is where we can uncover new learning and turn doubters into believers.
Recently, a well-known management consulting firm was hired to develop a new growth strategy for a major pain reliever brand with over $300 million in sales and a 30% market share. The consulting firm’s research convincingly showed that 60% of the brand’s usage occasions were for muscular pain. It was therefore argued that the product should concentrate all of its advertising dollars on this single use indication. By going narrow, it was argued, we could grow huge. In doing so, it could capture the lion’s share of the muscular pain “market”. It was brilliant strategy – or was it?
By solely concentrating on a single indication, 40% of the business would be at risk (i.e., the non-muscle pain usage). Ad support would only be for muscle pain, leaving all other indications vulnerable and up for grabs.
The new campaign would, therefore, start with a significant share deficit – roughly 12% (40% non-muscle pain usage of the 30% share). It would have to make up the share loss by stealing from other muscle pain usage occasions – if that was even possible. The strategy was severely flawed.
My client immediately felt uneasy with the recommendation. She questioned the way in which the firm had asked its questions. The approach they used virtually guaranteed the results obtained and, in fact, it was designed to simply confirm what they had recommended.
We urged caution and requested resources to conduct a better-designed study. Management initially balked, saying “I don’t believe it”. But we prevailed, and showed that the pain reliever market is really a huge basket of symptoms far greater than muscular pain alone. There were many pain indications: headache, fever, menstrual pain, dental pain, fractures, flu/fever, swelling, fatigue, and more. Muscular pain was actually a smaller indication than headache or flu/fever. Our brand just didn’t have as much share for these indications.
With the executive team’s support, the brand was successfully re-positioned as an all-purpose pain reliever. Other pain reliever brands that went the “single indication” route ended up being much smaller businesses.
And we don’t hear people saying “I don’t believe it” anymore.
by Bob Walker | Sep 17, 2018 | Marketing and strategy, Marketing research, Survey research
On August 29, 2018 SurveyMonkey filed an initial registration statement with the SEC (symbol “SVMK”) to float an IPO; the offering is now expected in late September. A recent update to its IPO filing includes a first pass at pricing; it has printed a price range of $9-$11 which, at a midpoint valuation, is $1.29 billion (lower than originally estimated).
In 2017, SurveyMonkey had revenues of $219MM, up 5.5% from 2016, and appears to be on track for around $240MM in 2018. However, the company is losing money: the loss of $24MM in 2017 has already been exceeded in the first six months of 2018 ($27MM). The company attributes this to increased R&D spending, but this accounts for $15MM of that figure.
In the research space, there are other possible IPO candidates, e.g., Qualtrics (we expect an IPO eventually), Decipher (part of FocusVision), and Confirmit (already listed on the Oslo exchange). Of all of the SaaS offerings, SurveyMonkey has perhaps the most to gain as it contemplates expansion – or sets itself up to be acquired. The list of potential suitors could include social media, e.g., Facebook (Sheryl Sandberg owns 5% of SVMK) or Google, and on the research/data science side are ResearchNow/SSI, IBM (SPSS), or even Microsoft.
Yet the opposite might be true: SVMK notes that their large user base, offerings, extensive data set, and integrations provide opportunities to drive acquisition: remember Zoomerang?
The S1 statement is interesting from a trends standpoint, as SVMK makes the following observations (paraphrased) about the survey research industry:
- The nature of engagement between organizations and their key constituents is fundamentally changing by becoming more open, bi-directional and frequent. Internet-enabled business models, together with rapidly evolving societal changes have revolutionized constituent expectations for service, speed and experience. Organizations that ignore, misinterpret or react too slowly to feedback risk falling behind the competition.
- “Big data” alone is insufficient to optimize decision making. To make good decisions, organizations need to marry “big data” with “people powered data” so that organizations can see beyond basic trends and better understand issues affecting key constituents.
- Employees are increasingly empowered to make decisions, and decision making within organizations has become decentralized. Employees throughout organizations are directly collecting and analyzing feedback. Access to information enables more decisions to be made at more levels across the organization. This accelerates the operating speed of the organization and increases accountability for decision making at all levels. As this data set is aggregated, organizational leadership is also using these insights to improve organization-wide decision making.
- Technology adoption is changing: IT solutions are now shaped by decentralized use. As organizations let employees become more empowered, technology becomes accessible to more individuals with varying levels of skill. IT departments then must step in and impose enterprise-grade security, customized company branding, and integration with software applications.
SVMK bolsters its IPO case by noting that quality research requires design, analysis time, and expertise that many companies do not have. Thus, individuals with absolutely no research expertise can gather and analyze data like a pro. As a long-time marketing research consultant, I find this assertion to be silly. Believe what you want; an additional planned layer of AI technology is envisioned to add support to this naive conceptual model.
Of note, a study conducted by SVMK in 2017 showed that 45% of business users who utilize online survey software considered SurveyMonkey to be their survey platform of choice. This makes perfect sense to me: SurveyMonkey fits the needs of individuals and small teams who need answers to basic questions. The design tool and integrations are good, and the online reporting is solid (better than several enterprise platforms), and the mobile app is very good.
In the right hands, SurveyMonkey can work as well as enterprise platforms, giving SVMK much more runway to grow. Conversely, growth in enterprise platforms like Qualtrics is flattening, as more revenue must come from consulting services and thus stealing business from full-service research firms. And, unlike many enterprise platforms, SVMK has developed a huge stable of free integrations to expand its functionality, while other companies charge ridiculous amounts for the same thing.
There is no question that the impact of SurveyMonkey on the survey research industry has been vast: there are 60 million registered users, of which 16 million are active. While most accounts are non-revenue generating (i.e., free), there are still 600K paying customers across 300K organizations.
by Bob Walker | Aug 22, 2018 | Marketing research, Research design, Survey research
f you’re a researcher, you’ve no doubt heard about “concepts”. Concepts are ideas that can come from many places, such as R&D, in reaction to competitive activity, or as “blue sky” what-if explorations. Management consultant Peter Drucker was known for saying that companies have just two functions: marketing and innovation. If so, a concept is where these two functions intersect.
The fundamental purpose of concept testing is to help companies allocate scarce new product development resources in the most effective manner possible. While not an exact science, concept testing is the best way to evaluate the merits of an individual idea. So here are some simple guidelines to keep in mind when creating and testing concepts.
#1: Stick To A Standard Format
Using a standardized format helps minimize bias caused by differences in idea presentation, letting you compare across time. Use the same format to represent new ideas, flankers, line extensions, or repositioning of existing products.
#2: Avoid Subtle Differences
As a rule, subtle differences in concepts (i.e., “tweaking”) do not matter, yet brand managers will obsess over them. Our firm has tested hundreds of concepts and a mistake that is continually repeated is assuming that consumers either care about, or can react to, subtle differences in the wording or features and benefits. Small wording changes are meaningless and glossed over by the average consumer.
#3: Don’t Slam The Competition
Research consistently shows that consumers dislike brand comparisons, and especially those that attack a competitor directly. When creating a concept, it’s perfectly fine to focus on the benefits and positive story that your product or service offers, but avoid negative attacks on the competition.
#4: Keep It Pithy
In an age of ever-increasing distraction, consumers do not have the time or interest in reading an exhaustive concept description. Particularly in an online format, and even when using a double opt-in consumer panel as your sample, biometric data consistently shows that most respondents simply scan rather than read.
#5: The Use of Images
An image is an extremely powerful tool to support your concept or new product idea, and can be used for a multitude of purposes: to show product function, convey a persona, use occasions, set the tone, and emotion. However, the use of images in the context of a concept testing system for a company, where there is a need for comparing ideas across time/studies, is open for debate. Images can overpower factual details of a concept, and make subsequent comparisons more difficult. In early stage testing, images are best left out and introduced once the core idea has been identified (i.e., for positioning or advertising concept research).
To learn more, download our brief pdf on this topic here.
