Recap: Carlson On Metrics

Last week I attended and gave a talk at "Carlson On Metrics", organized by Professor Rajesh Chandy  and the staff at the Institute for Research in Marketing at the University of Minnesota's Carlson School of Management.

Prof. Chandy on the "Why?" for the conference:

• "Many options, few certainties"
• "Lots of data, little information"

Here are some notes on the talks (comments very welcome!):

"Long-Term Impact of Marketing Spending" -- Professor Dominique M. ("Mike") Hanssens, UCLA Anderson School of Management

• 93-94% of single marketing actions have no long-term lift
• Permanent impacts occur *almost exclusively* in emerging, evolving markets -- like HDTV's, social networks
• Cumulative long-term advertising effect = 2x short term effect.  Customers learn quickly, forget slowly; economic effects duration = 2 months
• Sales call resp = 5x ads, LT effect = 2x
• Sales promotions have no LT effects
• Sales promo impacts (Sunil Gupta 1988)-- 84% on brand choice, 2% on purchase quantity, 14% on category incidence; LT effects are 11%/ 23%/ 66%.  Meaning retailer is biggest beneficiary of LT effects of sales promos
• Can you get permanent lift from temporary action (I learned a new word: hysteresis)? Yes, when people are learning -- when the cement is still wet
• E-Curve vs. perceived utility -- former is steady decline, latter spikes up during initial E-Curve decline, then drops steeply v E-Curve, then recovery to E-Curve
• Example MSFT vs. Sony game, MSFT had one year availability lead, THAT was the time they should have marketed aggressively to build long-term brand value
• Cesar's takeaway -- really hard to build brand value with marketing spend in relatively stable, fixed markets.  SO marketing spend should focus on ST promotion.  Prof. Hanssen agrees with this conclusion in follow up Q&A
• The courageous exception: P&G shifted $260M from promo to advertising in 24 categories, 118 brands, lost 16% share but increased profits by $1B.  They can do it because they have tremendous distribution leverage, and reinvest some of the profits into product innovation that justifies higher premium (from fewer price promotions)
• Prof Hanssen's takeaways include conclusion that single marketing actions must pay for themselves -- can't count on LT effects, since whether or not these happen depend on timing AND a broader sustained and consistent application of marketing strategy and policy that individual initiatives can't typically count on.  Consumers are "cats, not cows" -- they require continuous prodding and they talk to each other, so it's hard to isolate price promo efforts.

"Evaluating the Business Impact of Brand Perceptions" -- James Henney, Wells Fargo

• "Jeff Bezos -- 'A brand is what people say about you when you are not in the room.'"
• Branding goal = "shift demand".  How well have we been able to identify this impact?
• Effects on customer acquisition?  Deepening existing relationships?
• Research
  • Key variables: Brand Characteristics ("Brand Performance Index")
    
    • Awareness -- have you heard of
    • Consideration -- would consider?
    • Preference -- prefer?
    • Distinctiveness
    • Relevance
    • Overall Impression
  • Execution
    
    • 6600 customers, apr-dec 06, 55 minute survey
  • trying to predict ownership and bal in checking and saving, measured jan 06 - feb 07
  • control for # HH's, # stores, competitive intensity
  • unit of analysis = zip code, 2k units covered
  • dependent variable = 3 month aggregate cust behavior (new HH, new check acct, new sav acct) before and after brand measurement
  • WLS TS regression
  • Findings
    • R^2 = .86 on # new HH, .91 on # new check, .92 on # new sav
  • stat significant
  • but 2/3 of credit can be attributed to "lagged vars", 10-25% to store distribution, 15-20% attributed to BRAND SCORES
  • So, how much increase in (new, deepening) business would be attributable to 10% increase in brand scores?  About 3-5% in first and second , flattens year three. Don't know how much it would *cost* to do this.
  • Enhancements to come -- *which* of the brand factors matter most?

"Stronger Relationships, Better Results: New Metrics of Loyalty and Engagement" -- Linda Vytlacil, Carlson Marketing

• Revenue is by far the most important driver of marketing investment decisions, followed by ROI and Profitability, Brand Equity, and cost
  • Much less consideration of competitive response, customer lifetime value, etc.
• Partnership between Peppers & Rodgers Group and Carlson Marketing to assess the impact of relationship strength, and manage it practically
• Proposing Relationship Strength metric -- "RSx" -- as an intermediate aggregate of a number of "antecedents"
• Interesting analysis of different effects, industry by industry (fiserv, retail, auto, airline, (telecom, high tech), of antecedents on Relationship Strength "RSx" metric
  • auto industry scores strongest overall and for mutuality and alignment (emotional factors cited), fiserv for trust, commitment (given higher switching costs)
• Strongest RSX to outcome linkage is on recommendation (validates Net Promoter Score framework)

"Marketing Effectiveness: Beyond ROI" -- Gordon Wyner, Millward Brown

Statistics from his firm's experience I hadn't seen before:

• Optimizing marketing mix yields 10-20% improvements
• Experimental testing increases yields by 2x

"Leveraging Loyalty Card Data to Measure ROI" -- Adrian Sosa, CVS Caremark

Adrian (a fellow Tuck grad) is part of the team that runs ExtraCare, the largest loyalty program in the US -- one out of every six people in this country belong to it.  The program has three components:  circular-based ExtraBucks, clip-free-couponing, and purchase-based targeted offers (based on personas and customer life-time value, driving 140 variants).  Adrian described the RFM-based approach behind the program (very little data augmentation / appending in their world), and the home-grown tools they have built to run it.  Salient feature -- the sheer scale of the program (10 billion database rows updated daily) allows for statistically rigorous tests of offers vs. a variety of controls. 

The program works:  for example, Adrian presented data showing a 0-4% revenue uplift associated with a 0.2-1.0% redemption rate (on de-seasonalized sales) for their ExtraBucks incentive, with the majority of data points concentrated at 1-2% uplift for 0.2-0.4% redemption.  Adrian says, "ExtraCare is *the* biggest differentiatior for CVS...", a perspective echoed by his boss the CMO, Rob Price. (Shameless plug: according to Adrian, Rob's a Monitor Group alum.)

More specifically, Adrian noted also that email offers, no matter how well-targeted and personalized, don't perform as well as POS/kiosk offers (my take is that this is likely characteristic of the category, generally composed of lower-consideration products).

(Related nugget Adrian shared that I didn't know: 75% of CVS sales are pharmacy, only 25% are "front of store", though profits are more like 50-50; the opportunity to grab more of the profit associated with all of these prescription drug sales would explain why it was so attractive for CVS to acquire PBM Caremark.)

(Also related, I had a chance to chat with Adrian about a "physical hyperlink" strategy for promotions, using SMS or possibly (in the future) QR codes; Adrian mentioned they've been experimenting with it and see it as promising but that to date samples have been small and results more sporadic.  My own thinking has been that since a mobile-device-based approach to tracking and couponing is less expensive than rolling out kiosks and more a more granular and accurate method than PRISM-style store traffic tracking, it's the inevitable way forward in retail, but I guess today's mobile-savvy teens aren't yet at an age to constitute the vanguard of incentive-motivated shoppers.)

"Customer Value in a Networked Society" -- Prof. Sunil Gupta, Harvard Business School

Prof. Gupta has taken a long-term interest in customer value and applied it to trying to understand better the relationship between customer value and enterprise value in social media.  In his research, Prof. Gupta has developed a model that better explains (vs. customer value alone) the relationship between the balance of "buyers and sellers" in markets like eBay to the value of such firms.  In short, if you multiply the discounted customer profitability (customer lifetime value, or CLV) of firms like Capital One, and E*TRADE by their respective number of customers, you get a pretty close approximation of their market caps.  But this doesn't work for properties like eBay, Monster, and Amazon, which have created user-to-user " markets as most or all of their businesses. 

This is really interesting and relevant research, because a "revenue per unique user" is a common valuation metric for today's Web 2.0 property M&A market.  In the Q&A following his talk, we explored the potential for large media firms to use this construct to properly balance advertisers against readers and subscribers for maximum enterprise value.  This should be of interest not only for web pure plays but also for large media conglomerates looking to reinvent themselves as  "online community" franchises.

"Linking Consumer Data to Business Metrics That Matter" -- David Krajicek, GfK

David's talk focused on the benefits of a more integrated approach to marketing data analysis, and on the mechanics of making that a reality.  More specifically, his model seeks to tie the impact of individual marketing investments through intermediate marketing-related operational metrics, to financially-based return metrics.  Of particular interest to me was a case study about MarCom strategy for a wireless carrier, where the model was used to optimize the mix of "reliability", "tech leadership", "value", and "non-theme-based awareness-building" imagery investments.   The growing experience of firms like GfK, combined with the increasingly powerful multi-channel analytic capabilities of firms like VisualIQ (whom I've gotten to know recently) together are accelerating the half-life of the "wasted half" of ad spending.

"Metrics of Pricing" -- Prof. Mark Bergen, Carlson School/ UM

Pricing as a marketing lever is often thought to be "costless".  It turns out to be anything but, for a variety of reasons, ranging from the sheer logistical complexity of changing listed prices, to the need to coordinate adjustments with other parts of the firm and with legal requirements, to the challenges of communicating the logic of price changes to employees and through them to customers.  Implementing pricing changes artfully, in harmony with the rest of a firm's marketing mix, turns out to be as much an organizational challenge as a logical puzzle that if mastered can be a real strategic advantage. 

"Budgeting and Allocation Across Corporate and Brand Advertising" -- Prof. Prasad Naik, UC Davis Graduate School of Management

Using Ford as an example, Prof. Naik demonstrated how by understanding the "brand sales elasticity of corporate advertising", and by managing brand advertising investments as a portfolio under a corporate umbrella, it's possible to make 1+1 >2.  In the Q&A, Jim Schroer (CEO of Carlson Marketing and former CMO of Ford) noted that he'd wished he'd had this analysis when he was at Ford, but noted that a lot also depends on the corporate campaign messaging.  For example, he observed that "Quality is Job 1" was a very complementary theme, a circumstance that might not always exist.  Others noted that much depends on having an established corporate brand that can help, rather than one that must first be built.

"Multi-Channel Marketing Optimization" -- yours truly

Here are the results of an audience quiz I did:

• about half of the relevant participants (i.e., those from big firms) report that they consistently and systematically optimize mix across channels; roughly a quarter say they do this somewhat, campaign by campaign; the balance indicated they do so rarely (much higher than I expected)
• about 2/3 indicated they have multi-channel marketing dashboards and data warehouses encompassing ~80% of available channels (again, much higher than I'd thought)
• but, only roughly one in five indicated they had active cross-channel teams, with related metrics and compensation

So, as with other things, our technological reach seems to exceed our organizational grasp.

"Measuring Tasks and Compensating Channel Partners: Guidelines and Consequences" -- Prof. George John, Carlson School/ UM

I never knew this, but apparently when IBM engaged Microsoft to write MS/DOS for the IBM PC, they paid them according to KLOC, or per thousand lines of code.  (MSFT skeptics will of course say that this explains so much.)  Here's a YouTube video Prof. John showed (excerpted from the PBS series "The Triumph of the Nerds" -- start watching at 4 minutes in) that really brought this issue to life.  A better way might have been to compensate Microsoft based on a base of time and materials, plus a grid-based  incentive, where the coordinates would be a function of functionality and performance.

My take-away from Prof. John's talk:  When conditions are "noisy" -- that is to say, a low correlation exists correlation between outcomes as measured by channel metrics, and the effort that a partner puts in -- you need a more sophisticated, multi-dimensional compensation scheme to properly align interests.

Panel: "Metrics of Growth" -- Greg Michaels, Kraft; Jeff Hunter, General Mills; Tom Sullivan, United Health; (moderated by Prof. Chandy)

Greg Michaels presented an in-depth look at how Kraft measures and reports Total Brand Value, as inspired by Jim Kilts' tenure there (and further described in his book Doing What Matters).  He also described how Kraft has worked with Canback-Dangel Predictive Analytic Integrators to blend a variety of inputs and models to predict demand for new products in emerging markets where pre-existing data is very limited.  For example, we looked at how Kraft assesses likely demand for "cheese for China"; turns out disposable income drives 50-100% more "elasticity of cheese demand" than any other factor.  Who knew?

(Unfortunately I wasn't able to stay for the discussions on Friday after lunch.  So, much appreciated if any readers can link to accounts of these sessions in comments!)

I really liked this conference:

• right content, right time -- with apologies to LOTR fans, "The time of digital evangelists is over, and the time of business people is at hand.  But we must have metrics to order all things as we will..."
• balanced mix of perspectives-- top marketing research/ analytics executives from some of the world's leading brands (Kraft, Unilever, CVS, Wells Fargo, American Airlines, Target, Astra Zenica, Seagate, 3M, and many others), combined with leading academics in the field, leavened with some trusted advisors with deep roots in the business and the research and data sets to show for them.
• intimate -- about 115 people
• run flawlessly -- even with last minute additions by folks like me.  (Thanks to Rebecca Monro and Letta Wren Christianson!)
• great location -- late-May Minneapolis was glorious; I enjoyed a run on the banks of the Mississippi, passing some interesting sights; and, if you haven't been downtown there lately (as I hadn't) you'd be surprised at how vibrant the scene is.  Prof. Chandy hosted two wonderful dinners for the speakers, including one at the tres hip Chambers Hotel, where we pondered art like this :-)

I look forward to tracking what's coming out of the Institute, and to visiting again soon!