Data and Disruption

In the June 23 edition of The New Yorker, in an article titled "The Disruption Machine", Harvard history professor Jill Lepore critiques the theories of Harvard Business School professor Clayton Christensen.  Her article is wonderfully written of course, but curiously sharp in tone (or is it me?). Re-reading it, I have the impression she's sick of the Manifest Destiny - like over-simplification and over-application of Christensen's theories:

Most big ideas have loud critics. Not disruption. Disruptive innovation as the explanation for how change happens has been subject to little serious criticism, partly because it’s headlong, while critical inquiry is unhurried; partly because disrupters ridicule doubters by charging them with fogyism, as if to criticize a theory of change were identical to decrying change; and partly because, in its modern usage, innovation is the idea of progress jammed into a criticism-proof jack-in-the-box.

But unfortunately the baby appears to go out with the bathwater. Lepore suggests -- accuses -- Christensen of cherry-picking his data, and describes a dynamic in business scholarship (as distinguished from practices in other fields) that over-promotes findings. In taking aim, fairly or not, at Christensen (who beyond his reputation as a scholar is even more respected for his integrity), Lepore goes for the jugular of the academic-industrial complex. 

Professor Christensen replied at length to Lepore's piece in an interview he gave last Friday to Bloomberg Businessweek reporter Drake Bennett, taking significant issues with her points -- maybe too personally?

Well, in the first two or three pages, it seems that her motivation is to try to rein in this almost random use of the word “disruption.” The word is used to justify whatever anybody—an entrepreneur or a college student—wants to do. And as I read that, I was delighted that somebody with her standing would join me in trying to bring discipline and understanding around a very useful theory. I’ve been trying to do it for 20 years.

And then in a stunning reversal, she starts instead to try to discredit Clay Christensen, in a really mean way. And mean is fine, but in order to discredit me, Jill had to break all of the rules of scholarship that she accused me of breaking—in just egregious ways, truly egregious ways. In fact, every one—every one—of those points that she attempted to make [about The Innovator’s Dilemma] has been addressed in a subsequent book or article. Every one! And if she was truly a scholar as she pretends, she would have read [those]. I hope you can understand why I am mad that a woman of her stature could perform such a criminal act of dishonesty—at Harvard, of all places.

Why am I interested, and why should you care?

Christensen's research into innovation, and his books like The Innovator's Dilemma and its successors have pretty much dominated the landscape on the subject for nearly the last twenty years. If you're a senior executive at a big established firm, he's made you paranoid.  If you're an entrepreneur at a small, ambitious firm, you're telling your investors, customers, and employees you're one of the "disruptive innovations"  making those executives paranoid. There's even a slim chance you might be right! Lepore is a Pulitzer Prize and National Book Award winner (as well as a department chair at Harvard).  Scholars of her prominence usually don't take on other scholars, particularly outside their field, without good reason or care. Christensen's reply suggests she had neither, and prior critiques of her work suggest this might be the latest in a pattern of occasional trip-ups. In any event, the critique's attracted some attention. If you follow either of these folks for professional reasons (or simply are drawn to MMA-style clashes between academic titans) you might be interested to track this story too.

I'm interested because the controversy highlights a pattern I often see, and as I describe in my new book Marketing and Sales Analytics, I use a variety of techniques to compensate for.

The pattern is to treat ideas and supporting evidence like Christensen's as the last word on a subject.  After all, he teaches at Harvard Business School, right? But even Christensen says (in his Bloomberg Businessweek interview) that models and supporting stories like his are at best just partly predictive  points of departure for any specific "truth" I need to be tracking, and hedging against.

The psychologist Daniel Kahneman, who wrote the best-seller Thinking, Fast and Slow about his research into cognitive bias, talks about the power of stories, and in particular about our susceptibility to them and to other biases when we're tired, frazzled, juggling a thousand things. I think Christensen's theory of disruptive innovations invites criticisms like Lepore's because it mirrors,  purposefully or not, an especially powerful archetypal story, in this case of the hero who battles long odds but ultimately triumphs grandly.  

Recent history and scholarship has reinforced the power of this particular theme.  The Internet revolution has created a number of poster children for it (I'm reminded of Paul Simon's line "Every generation thows a hero up the pop charts", and in our generation we've had several), and created conditions for accelerating the pace of disruption. Nassim Taleb's scholarship, in particular his own best-selling book The Black Swan, has sensitized us to the existence of outlier possibilities our models often miss (and even explicitly discard to improve fit).

So the combination of a story model we're especially tuned for, and recent examples that magnify its power, position it uniquely in our consciousness. The implication for executives is to be mindful of the bias this creates, and to have appropriate mechanisms in place for managing it.

In my book I describe a portfolio-driven approach to managing analytic efforts that can help with this.  In brief, the idea is to manage analytic projects not just as a list of questions to be checked off as you answer them, but as a venture capitalist's portfolio of investments that need to generate target returns appropriate to their size and riskiness.  In the governance of such portfolios we set up in our work with clients, we manage to a rule we call "3-2-1".  In any given quarter, we aim to have the collection of analytic initiatives we're pursuing yield (notionally) three "news you can use" insights, two experiments based on those insights, and one outcome we're putting into production at scale to help pay the freight for the overall investment in the portfolio (including us).  The portfolios are constructed to reflect priorities across a grid that itself is a collection of different business opportunities ("Need X for Customer Y") we're targeting and different purchase funnel or customer journey stages for these opportunities.  This grid helps you judge how concentrated you are on existing versus new opportunities, and on whether your investments are appropriately focused on bottlenecks in the relevant funnels or journeys.

The main point of using a grid like this, as opposed to just a list of individual projects, is that it forces you to think backwards in a data-driven way from the customer-defined strategies, goals, and objective performance of the business, and about whether you've got sufficient attention on the hot spots thereof. In the parallel world of innovation, I believe the opportunity for subscribers to Christensen's ideas is to neither accept nor reject "disruption dogma", but to ask themselves whether they've got sufficient "3-2-1" style attention and results on the hot spots in their business.  "What customers are important to us?"  "What needs are important to them?" "How well are we serving those needs?" "What's / who's out there trying to serve these same folks and their needs?" "What kind of progress are they making, and what's contributing or impeding that progress?" "What research, analysis, or testing should we be doing to stay close to potentially meaningful threats?" "Are the hedges these efforts represent probabilistically - in likely magnitude and yield - a good match for the risk the potential competition out there represents to our business?"

A "Clash Of The Titans" around business theories, while good sport for some, can cause a lot of anxiety for many others trying to build and run their businesses. Framing your approach specifically and objectively using the techniques for managing analytic efforts can help you get past these concerns in a practical, tailored - and maybe even heroic - way.