About

I'm a partner in the advanced analytics group at Bain & Company, the global management consulting firm. My primary focus is on marketing analytics (bio). I've been writing here (views my own) about marketing, technology, e-business, and analytics since 2003 (blog name explained).

Email or follow me:

-->

92 posts categorized "Analytics"

September 11, 2015

@AMABoston #Marketing #Attribution 9/17 Panel Discussion Guide - Suggestions Welcome!

On September 17, I'm moderating a panel on marketing attribution analysis at a meeting of the American Marketing Association's Boston Chapter.

Here are some questions we've put together to guide the discussion.  What's on your mind?

  1. Let’s start with a plain English question: How do you decide where to spend your marketing dollars?

  2. What assumptions do you make about the relative effectiveness and efficiency of different options?

  3. What are these assumptions based on?

  4. So, for me, attribution is a fancy word for giving different marketing investment options -- channels, partners within those channels, creative alternatives -- proper, or at least more proper credit for their impact on a metric of interest, like a lead or a sale.  How would you define it, or improve on this?

  5. Earlier you told us about how you do it today, but I’m also interested in your attribution journey.  Where was the organization’s level of attribution sophistication when you got into your latest role?

  6. What attribution adjustments did you make first, and next?

  7. What was the logic for starting there? How much was it the potential value -- the magnitude of the spend, for example -- and how much was it about feasibility - you could reasonably expect to get the data?

  8. How much of the adjustment was based on “top down” statistical modeling of overall spend and impression data, and how much of it was bottom up based on user-level proxies like cookie data?

  9. How long did it take to get up and to an interesting insight you could put to work?

  10. What worked, and didn’t?

  11. What surprised you?

  12. How did you balance building the capability and getting results?

  13. What kind of results have you gotten so far?

  14. What’s next on your roadmap?

  15. How far do you think you can push attribution analysis before you hit diminishing or negative ROI?

  16. What’s the limiter? Is it data quality? Integration challenges like tracking a user across channels? Is it your ability to put insights to work?

  17. Let’s take each of those:

    1. We hear a lot about how cookies are dead, and then the death of the cookie has been greatly exaggerated -- how far can you trust cookie based data for attribution purposes?

    2. Now that the big platforms -- Facebook, Google -- can track us everywhere, and use that to do cross-channel advertising programs, do you try to beat them, or join them? If you try to beat them, how far can you take cross-channel tracking? When and how does privacy come into play -- what are your bright lines for this?

    3. What parts of your attribution insights are then integrated with your marketing automation platforms? What’s still manual?

  18. What does managing based on  more sophisticated attribution look like? Who does it? What kind of interfaces and reports does this person or team use?  How often do they review and act on the results?

  19. What kind of people and organizational challenges have more sophisticated attribution approaches created? How have you solved them?

  20. What advice would you offer to someone with a marketing budget of $1M? $10M? $100M? 

  21. What’s the practical technical frontier of what people are doing?

  22. How do you keep up?  What resources would you suggest for someone interested in learning more and staying up to date?

August 17, 2015

Ten Things I Think*: Thoughts on The #Amazon Workplace in @nytimes @jodikantor @DavidStreitfeld

On Saturday the New York Times published a piece by Jodi Kantor and David Streitfeld titled, "Inside Amazon: Wrestling Big Ideas in a Bruising Workplace". Here's the link:

http://www.nytimes.com/2015/08/16/technology/inside-amazon-wrestling-big-ideas-in-a-bruising-workplace.html

The article provoked a large number of comments and showed up in several of the social media feeds I follow. Plus, my business is helping organizations build their capabilities to use data to drive results.  

Let's stipulate that the article presented facts accurately and in balance. Here's ten things I think (*Thanks to @SI_PeterKing). 

1. What a great litmus test for values. But don't listen to what people say, watch what they do.

You live your values every time you shop there, or anywhere else. I'm reminded of the mission statement of ArsDigita, where I once worked:

...Beyond that, we don't worry about corporate culture. We have a certain set of customers. We have a certain set of people. We have a certain set of tools. Discussions or theories won't change any of those things. If any ArsDigita member wants to change ArsDigita, he or she need only add to the customers, add to the people, or add to the tools.

So, shop at Amazon, or don't, or, since life's complicated, change the mix of where you shop, according to some rules that make sense to you. Like, "I'll start by looking for competitively priced and appropriately convenient and accountable alternatives, whose operating and employment practices are more consistent with my beliefs about how to treat people. But if I can't find one, I'll still shop at Amazon."

Or, if you work there, and the current culture doesn't work for you, either accept the fact that it doesn't fit and leave, or work to change it as far as you can.

IMG_1799

 

2. Everything's relative.

The article calls out Amazon, but doesn't explicitly contrast it with any other retailers, and examine the trade-off in performance with these retailers: prices for consumers, returns for shareholders -- which of course include many pension funds holding the savings of a wide range of investors. (Since Amazon competes in more categories than retail, and it's an icon of the global economy, it's worth extending this comparison of course, but you have to start someplace.)

Is Amazon an anomalous white-collar sweatshop? Why stop at researching the question? Here's an idea for a motivated someone: create a "Shopper's Bill of Expectations" service.  The service would articulate and promote a set of values / standards / expectations its members would like to see: for example, paternity leave, compliance with environmental regulations and trends, living wage, working conditions, etc. Then it would invite members to subscribe to an auditing service, like Consumer Reports, and invite retailers who agree to comply with the service's expectations to enroll in it. Finally, it would provide some sort of tracking mechanism, manual or automated, so it and its enrolled retailers and members can see how much dollar volume the service sends from members to retailers. There's a huge data play in here, too! I have no objections to this service leveraging that in ways that are transparent and which don't unduly compromise or pressure its core proposition. 

Naïve? Seattle's embraced Fair Trade for coffee-growing campesinos, why not for highly-compensated ecommerce executives? (Uh, well...) 

3. This isn't me.  

Whether I expect emails and texts to be answered at midnight depends on the circumstances -- and they have to be pretty dire.  If someone takes a planned vacation which business conditions support, and thoroughly plans coverage, and says he or she will be out of contact, I respect that choice. If someone has a short-term or longer-term health crisis, I do what I can to support that person as far as practically possible, and then beyond as far as I can.  I don't believe in stack-ranking and firing the lowest decile; I believe in setting and continuously raising performance expectations, being direct and thorough and fair in evaluating people against them, and encouraging anyone who doesn't consistently clear those goals to find something else, with compassion, and endorsement for the stronger things they do. I believe in courtesy and kindness. I believe in teamwork, not beggar-thy-neighbor advancement.

4. This is me.  

I'm objective about performance. I prefer a focus on ends, not means, as long as means are pursued ethically, which = Golden Rule to me. I believe in coaching and development, but I expect hard work and demonstrated progress and positivity and enthusiasm in return. I expect to care, and I expect my colleagues to care as well and as much. And, if I'm asked to be up late busting my behind to help someone under pressure, I do expect him or her to be responsive to late-night emails and texts. I don't care for passive-aggressive cultures whose members play nice superficially but work at cross-purposes behind the scenes; in this regard Amazon's directness and openness to public disagreement seems much healthier.

5. Context doesn't justify, but it matters to the assessment.

Amazon is a product of a super-thin-margin industry. It was founded by a man with the characteristics to succeed in that unambiguous, unprotected environment, and it has thrived by attracting managers with similar profiles. If and when you contrast a culture like Amazon's with some other avatar of corporate humanitarianism, it may be worth a look at the glass-house bulwarks supporting the profitability that enable those practices. These may take the forms of de facto technical standards the firm has established, or a monopoly in its market(s), or legal or regulatory actions -- for example, easy monetary policies -- or hard-won, carefully-cultivated brand strength. Assume the bulwarks weren't there: how would things change? Many successful firms would do well to ask this of themselves.

6. Data-driven, to a fault?

If what gets measured gets managed, we also manage as we measure. I see a spectrum: we ignore data at one end, we are slaves to it at the other end.  Both extremes are dysfunctional. Ignorance is of course not bliss, but in very few cases do we have enough clean, ungamed data to put our faith in it exclusively.  Digital commerce may be one such exception, but employee behavior -- measured as described in the article, and here -- is not.

In most cases data are "Platonic Shadows" for what's happening in the business. To be useful they need to be looked at holistically, and across as much time as is available to distinguish signal from noise. I believe in watching for emergent patterns from machine learning, but also in humans having their heads in the game with hypotheses and explicit sensitivity to biases of many kinds.

7. Development requires opportunity and demands responsibility.

It may be dysfunctional at the margins, but I admire the empowerment Amazon offers its managers to get things done, and its expectation that they will get things done.  There's nothing sadder than well-educated and qualified managers who feel blocked and just go through the motions for a paycheck. Well, actually, there is: employees with extremely limited choices who are ethically and illegally exploited.

I appreciate Amazon's bias for action, and for matching analysis to the stakes and uncertainty associated with a decision.

8. Is this a cult of personality?

What happens beyond Jeff Bezos? Paging John Galt!

9. How do you compete?

Amazon competes on utility and service, tightly defined and realized based on the firm's extreme degree of customer focus. Their managerial ranks thus reflect this.  If you don't want to join them, or can't, or want to leave, then beat them.  This means playing a different game. For example, Amazon doesn't create brands so much as it amplifies them.  Likely you will need to create a brand along an emotional dimension that Amazon under-serves, then partner with Amazon in ways that extend but don't erode it. What opportunities to speak to or reflect someone's actual or desired identity can you reinforce through an online-retail service? Maybe some sort of emo-oriented Edgio blending content and commerce in curatively creative ways?

10. On reflection...

There are older and fouler things than Amazon in the deep places of the world. But Amabots and Amholes at Amazon and beyond, remember: In the end, the love you take is equal to the love you make. 

 

 

 

July 07, 2015

My Q&A With @chiefmartec's Scott Brinker

One Marketing Analytics Myth? The Perfect KPI... published July 6, 2015

February 20, 2015

My latest article in CMO.com: "Analytics Is Too Important To Be Left To Analysts"

"Analytics Is Too Important To Be Left To Analysts", published February 20, 2015

October 28, 2014

Driving Social Engagement With Sentiment Analysis: Text Analytics Summit West

McGraw Hill VP of R&D and Analytics Al Essa kindly invited me to join him in delivering this workshop in San Francisco on November 3.  Hope to see you there!

October 07, 2014

My Interview in EContentmag.com: "Which Analytics Tell The Best Story?"

http://bit.ly/watbs-econtmag, published October 3

July 21, 2014

"Lionel Messi Is Impossible." How About Your Salespeople? @skepticalsports #Analytics

Warning: due to author sloth, this post is dated.  Most of you have likely moved on with your lives.  Me, my Heart Is Full for los albicelestes, despite a disappointing outcome, but I still have a lingering case of futbolytics to get over.

Fivethirtyeight.com contributor Benjamin Morris had a fascinating article a couple of weeks ago that examined Argentine footballer Lionel Messi's play in recent years.  Until this year's World Cup of course, Messi had come under some criticism for, against the backdrop of his ethereal play for Barcelona, under-delivering for Argentina in recent years.  The article -- warning, 4600+ words long, with charts and videos and hyperlinks and footnotes, as in "Ask not, Bill Simmons, for whom the bell tolls..." -- explains persuasively that Messi is an outlier among outliers, even in his performances for the Argentine national team.  

The analysis of his shooting is fascinating enough.  But what really caught my attention, was the analysis of his passing and other influences on the game. In particular, here's a fascinating chart from Morris' article that makes the point neatly.

My friends at the multi-channel marketing attribution analytics firm Visual IQ are fond of a soccer metaphor to explain what they do.  "Giving all the conversion credit to the last-touched marketing channel is like giving Mario Götze all the credit for his goal in the 113th minute of the World Cup final," they say (with a zesty cruelty that borders on the sociopathic).

In the sales world, it's a holy grail to get to this kind of dynamic, or even an understanding of where on Morris' chart members of your team would be. It can be a thorny path to get there though, because unlike in Messi's world, a helping hand in sales can be harder to observe, and even if you try to measure it, often it can be (will be) gamed mercilessly and unhelpfully.

One way you can track this sort of thing is through online and offline knowledge sharing by members of your team.  Winning proposals, presentations, good answers to FAQs, and then views of these by others can all be tracked in relatively painless, game-free ways. A number of years ago when I worked at ArsDigita, we worked with Siemens to build ShareNet, a global sales and marketing knowledge management system that for many years was a poster child for applications of its kind (see here for the HBR case study).   The secret behind Siemens' success with ShareNet was the flexibility with which it could adapt what was captured, and how, to make it easy for people to contribute and consume.  Today, fortunately, the tools and costs for building capabilities like this are far more accessible. And now as attribution analysis moves closer to the center of the marketing analytics agenda, we have the opportunity to put the resulting data to work in a way that moves the dominant motivation for this kind of behavior beyond altruism to proper credit.

So if you'd like to improve your organization's gol-orientation, perhaps it's time to compile and publish your own assist chart?

May 29, 2014

Mary Meeker's @KPCB #InternetTrends Report: Critiquing The "Share of Time, Share of Money" Analysis

Mary Meeker's annual Internet Trends report is out.  It's a very helpful survey and synthesis of what's going on, as ever, all 164 pages of it. But for the past few years it's contained a bit of analysis that's bugged me.

Page 15 of the report (embedded below) is titled "Remain Optimistic About Mobile Ad Spend Growth... Print Remains Way Over-Indexed."  The main chart on the page compares the percentage of time people spend in different media with the percentage of advertising budgets that are spent in those media.  The assumption is that percentage of time and percentage of budget should roughly be equal for each medium.  Thus Meeker concludes that if -- as is the case for mobile -- the percentage of user time spent is greater than budget going there, then more ad dollars (as a percent of total) will flow to that medium, and vice versa (hence her point about print).

I can think of  demand-side, supply-side, and market maturity reasons that this equivalency thesis would break down, which also suggest directions for improving the analysis.  

On the demand side, different media may have different mixes of people, with different demographic characteristics.  For financial services advertisers, print users skew older -- and thus have more money, on average, making the potential value to advertisers of each minute of time spent by the average user there more valuable.  Different media may also have different advertising engagement power.  For example, in mobile, in either highly task-focused use cases or in distracted, skimming/ snacking ones, ads may be either invisible or intrusive, diminishing their relative impact (either in terms of direct interaction or view-through stimulation). By contrast, deeper lean-back-style engagement with TV, with more room for an ad to maneuver, might if the ad is good make a bigger impression. I wonder if there's also a reach premium at work.  Advertisers like to find the most efficient medium, but they also need to reach a large enough number of folks to execute campaigns effectively.  TV and print are more reach-oriented media, in general.

On the supply side, different media have different power distributions of the content they can offer, and different barriers to entry that can affect pricing.  On TV and in print, prime ad spots are more limited, so simple supply and demand dynamics drive up prices for the best spots beyond what the equivalency idea might suggest.  

In favor of Meeker's thesis, though representing another short term brake on it, is yet another factor she doesn't speak to directly. This is the relative maturity of the markets and buying processes for different media, and the experience of the participants in those markets.  A more mature, well-trafficked market, with well-understood dynamics, and lots of liquidity (think the ability for agencies and media brokers to resell time in TV's spot markets, for example), will, at the margin, attract and retain dollars, in particular while the true value of different media remain elusive. (This of course is one reason why attribution analysis is so hot, as evidenced by Google's and AOL Platform's recent acquisitions in this space.)  I say in favor, because as mobile ad markets mature over time, this disadvantage will erode.

So for advertisers, agency and media execs, entrepreneurs, and investors looking to play the arbitrage game at the edges of Meeker's observation, the question is, what adjustment factors for demand, supply, and market maturity would you apply this year and next?  It's not an idle question: tons of advertisers' media plans and publishers' business plans ride on these assumptions about how much money is going to come to or go away from them, and Meeker's report is an influential input into these plans in many cases.

A tactical limitation of Meeker's analysis is that while she suggests the overall potential shift in relative allocation of ad dollars (her slide suggests a "~$30B+" digital advertising growth opportunity in the USA alone - up from $20B last year*), she doesn't suggest a timescale and trendline for the pace with which we'll get there. One way to come at this is to look at the last 3-4 annual presentations she's made, and see how the relationships she's observed have changed over time.  Interestingly, in her 2013 report using 2012 data, on page 5, 12% of time is spent on mobile devices, and 3% of ad dollars are going there, for a 4x difference in percentages. In the 2014 report using 2013 data, 20% of time is spent on mobile, and 5% of media dollars are going there -- again, a 4x relationship.  

So, if the equivalency zeitgeist is at work, for the moment it may be stuck in a phone booth. But in the end I'm reminded of the futurist Roy Amara's saying: "We tend to overestimate the effect of a technology in the short term and underestimate its effect in the long term."  Plus let's not forget new technologies (Glass, Occulus Rift, both portable and  large/immersive) that will further jumble relevant media categories in years to come.

(*Emarketer seems to think we'll hit the $30B mobile advertising run rate sometime during 2016-2017.)

 

April 16, 2014

My New Book: #Marketing and #Sales #Analytics

I've written a second book.  It's called Marketing and Sales Analytics: Proven Techniques and Powerful Applications From Industry Leaders (so named for SEO purposes).  Pearson is publishing it (special thanks to Judah Phillips, author of Building A Digital Analytics Organization, for introducing me to Jeanne Glasser at Pearson).  The ebook version will be available on May 23, and the print version will come out June 23.

The book examines how to focus, build, and manage analytics capabilities related to sales and marketing.  It's aimed at C-level executives who are trying to take advantage of these capabilities, as well as other senior executives directly responsible for building and running these groups. It synthesizes interviews with 15 senior executives at a variety of firms across a number of industries, including Abbott, La-Z-Boy, HSN, Condé Nast, Harrah's, Aetna, The Hartford, Bed Bath & Beyond, Paramount Pictures, Wayfair, Harvard University, TIAA-CREF, Talbots, and Lenovo. My friend and former boss Bob Lord, author of Converge was kind enough to write the foreword.

I'm in the final editing stages. More to follow soon, including content, excerpts, nice things people have said about it, slideshows, articles, lunch talk...

January 17, 2014

Culturelytics

I'm working on a book. It will be titled Marketing and Sales Analytics: Powerful Lessons from Leading Practitioners. My first book, Pragmalytics, described some lessons I'd learned; this book extends those lessons with interviews with more than a dozen senior executives grappling with building and applying analytics capabilities in their companies. Pearson's agreed to publish it, and it will be out this spring. Right now I'm in the middle of the agony of writing it. Thank you Stephen Pressfield (and thanks to my wife Nan for introducing us).

A common denominator in the conversations I've been having is the importance of culture. Culture makes building an analytics capability possible. In some cases, pressure for culture change comes outside-in: external conditions become so dire that a firm must embrace data-driven objectivity. In others, the pressure comes top-down: senior leadership embodies it, leads by example, and is willing to re-staff the firm in its image. But what do you do when the wolf's not quite at the door, or when it makes more sense (hopefully, your situation) to try to build the capability largely within the team you have than to make wholesale changes?

There are a lot of models for understanding culture and how to change it. Here's a caveman version (informed by behavioral psychology principles, and small enough to remember). Culture is a collection of values -- beliefs -- about what works, and doesn't: what behaviors lead to good outcomes for customers, shareholders, and employees; and, what behaviors are either ignored or punished.

Photo (16)

Values, in turn, are developed through chances individuals have to try target behaviors, the consequences of those experiences, and how effectively those chances and their consequences are communicated to other people working in the organization.

Photo (15)

Chances are to  culture change as reps (repetitions) are to sports. If you want to drive change, to get better, you need more of them. Remember that not all reps come in games. Test programs can support culture change the same way practices work for teams. Also, courage is a muscle: to bench press 500 pounds once, start with one pushup, then ten, and so on. If you want your marketing team to get comfortable conceiving and executing bigger and bolder bets, start by carving out, frequently, many small test cells in your programs. Then, add weight: define and bound dimensions and ranges for experimentation within those cells that don't just have limits, but also minimums for departure from the norm. If you can't agree on exactly what part of your marketing mix needs the most attention, don't study it forever. A few pushups won't hurt, even if it's your belly that needs the attention. A habit is easier to re-focus than it is to start.

Consequences need to be both visible and meaningful. Visible means good feedback loops to understand the outcome of the chance taken. Meaningful can run to more pay and promotion of course, but also to opportunity and recognition. And don't forget: a sense of impact and accomplishment -- of making a difference -- can be the most powerful reinforcer of all. For this reason, a high density of chances with short, visible feedback loops becomes really important to your change strategy.

Communication magnifies and sustains the impact of chances taken and their consequences. If you speak up at a sales meeting, the client says Good Point, and I later praise you for that, the culture change impact is X. If I then relate that story to everyone at the next sales team meeting, the impact is X * 10 others there. If we write down that behavior in the firm's sales training program as a good model to follow, the impact is X * 100 others who will go through that program.

Summing up, here's a simple set of questions to ask for managing culture change:

  • What specific values does our culture consist of?
  • How strongly held are these values: how well-reinforced have they been by chances, consequences, and communication?
  • What values do I need to keep / change / drop / add?
  • In light of the pre-existing value topology -- fancy way of saying, the values already out there and their relative strength -- what specific chances, consequences, communication program will I need to effect the necessary keeps / changes / drops / adds to the value set?
  • How can my marketing and sales programs incorporate a greater number of formal and informal tests? How quickly and frequently can we execute them?
  • What dimensions (for example, pricing, visual design, messaging style and content, etc.) and "min-max" ranges on those dimensions should I set? 
  • How clearly and quickly can we see the results of these tests?
  • What pay, promotion, opportunity, and recognition implications can I associate with each test?
  • What mechanisms are available / should I use to communicate tests and results?

Ask these questions daily, tote up the score -- chances taken, consequences realized, communications executed -- weekly or monthly. Track the trend, slice the numbers by the behaviors and people you're trying to influence, and the consequences and communications that apply. Don't forget to keep culture change in context: frame it with the business results culture is supposed to serve. Re-focus, then wash, rinse, repeat.  Very soon you'll have a clear view of and strong grip on culture change in your organization.