Octavianworld

Controlling for Impression Volatility in Digital Ad Spend Tests @DataXu

I've recently been involved in evaluating the results of a matched market test that looked at the impact of changes in digital advertising spend by comparing test vs. control markets, and by comparing differential lift in these markets over prior periods (e.g., year on year).  One of the challenges involved in such tests is significant "impression volatility" across time periods -- basically, each dollar can buy you very different volumes of impressions from year to year.  

You can unpack this volatility into at least three components:  

• changes in overall macro-economic conditions that drive target audiences' attention,
• changes in the buying approach you took / networks you bought through, due to network-specific structural (like what publishers are included) and supply-demand drivers (like the relative effectiveness of the network's targeting approach)
• changes in "buy-specific" parameters (like audiences and palcements sought).  

Let's assume that you handle the first with your test / control market structure.  Let's also assume that the third is to be held constant as much as possible, for the purposes of the test (that is, buying the same properties / audiences, and using the same ad positions / placements for the tests).   So my question was, how much volatility does the second factor contribute, and what can be done to control for that in a test?

Surfing around I came on DataXu's March 2011 Market Pulse study.  DataXu is a service that allows you to buy across networks more efficiently in real time, sort of like what Kayak would be to travel if it were a fully automated agent and you flew every day.  The firm noted a year-on-year drop in average daily CPM volatility from 102% to 42% from May 2010 to February 2011 (meaning I think the average day to day change in price across all networks in each of the two months compared).  They attributed this to "dramatically increased volume of impressions bought and sold as well as maturation of trading systems".  Notwithstanding, the study still pointed to a 342% difference in average indexed CPMs across networks during February 2011.  

A number this big naturally piqued my interest, and so I read into the report to understand it better.  The top of page 2 of the report summary presents a nice graph that shows average monthly indexed CPMs across 11 networks, and indeed shows the difference between the highest-priced and the lowest-priced network to be 342%.  Applying "Olympic scoring" (tossing out highest- and lowest-priced exchanges) cuts that difference to about 180%, or roughly by half -- still a significant discrepancy of course.  Looking further, one standard deviation in the whole sample (including the top and bottom values) is about 44%.  Again, though perhaps a bit less dramatic for marketers' tastes, still lots.

(It's hard to know how "equivalent" the buys compared were, in terms of volumes, contextual consistency, and audience consistency, since the summary doesn't address these.  But let's assume they were, roughly.)

So what? If your (display) ad buys are not so property-specific / audience-targeted that run-of-network buys in contextual or audience categories are OK, future tests might channel buys through services like DataXu and declare the buys "fully-price-optimized" across the periods and markets compared, allowing you to ignore +/- ~50% "impression volatility" swings, assuming the Feb 2011 spreads hold.

However, if what you're buying is very specific -- and only available through direct purchase, or one or two specialized networks at most -- then you ignore factor 2, trust the laws of supply and demand, and assume that you've bought essentially the same "attention" regardless of the difference in impressions.

I've asked some knowledgeable friends to suggest some perspectives on this, and will pass along their ideas.  Other feedback welcome, especially from digital advertising / testing pros!  Oh and if you're really interested, check out the DataXu TC50 2009 pitch video.

#Watson Wins #Jeopardy! The Singularity Is Near(er): Are You Ready for Computational Engine Optimization ("CEO")?

Saw the news (though missed the show) that IBM's Watson won on Jeopardy.  Interesting to see this and other articles call out Watson's "stumble" -- as though they expected perfection, which is a milestone in itself.  Here's a great explanation of "what went wrong". 

There are two notable things to me about this development / achievement.

The first is to ask whether this puts us ahead or behind Ray Kurzweil's schedule for 2019 (as predicted in 1999).  (Really worth reading his predictions, since we're within shouting distance!  What would you "keep / change / drop / add"?)  

The second is a little closer in.   Given the pace of this development, what does it mean for us as humans / users / consumers / citizens on the one hand, and as marketers / investors, etc. on the other -- from "now" to, say, "two years out"?

Imagine for example that in two years, IBM provides access to a more generalized form of Watson as a cloud-based API.   What might you, as a person or as a business or other organization, do with a service that can understand speech, parse meanings, and optimize spending and investment recommendations based on how sure it is of the answer?

Cesar: "Watson, our lease is up soon, can you suggest some available space options nearby that would make sense for a business like Force Five Partners?"

Watson: "Cesar, here are five choices, with suggestions for what you should be paying for each, based on what I can find out right now..."

A stretch?  Apple's integrated Wolfram Alpha - based support into the Siri app for the iPhone now.  Try asking Siri, out loud: "What is the market capitalization of Goldman Sachs, divided by the US population?"   Answer back to me, in three seconds (iPhone 3GS / AT&T):

(Cross-check: Wolfram Alpha direct. Or,  GS on Google Finance / US Population on Google Search.)

 (FWIW, this hits 3 of 4 criteria in a prediction framework I suggested nearly six years ago.)

Wow.  We had barely figured out SEO, when we got slammed with SNO -- Social Network Optimization (as well as the frozen kind)!  Now we have to figure out Computational Engine Optimization?  (Confusingly, natch, "CEO" -- you read it here first!)  How do I optimize for "What inexpensive steakhouses are nearby?"   How do we even think about that?  

(Possible direction: Semantic Web Optimization -- "SWO", of course.  Make sure you are well tagged-for, and indexed-by, the data stores and services where the terms "inexpensive", "steakhouse", and "nearby" would be judged.  Or, in plain English: if Wolfram Alpha looks to Yelp to help answer this question, make sure your restaurant's entry there is labeled as a steakhouse, has an accurate address, and is accurately price-rated as "$".  Whatever gaming ensues,  just don't blame IBM / Apple / Wolfram /(Google too) for going for the mega-cheddar.)

It's trite to say that change is accelerating as technology develops.  ("We're only in the second inning!")  Some dismiss this (as Arthur C. Clarke said, we always overestimate the impact of technology in the short term, but underestimate it in the long term).  But, if you doubt, this chart is worth a look.  And then think about the degree to which "social" and "mobile" are now reinforcing, amplifying, and accelerating each other...

(Insert shameless commercial:) What are you doing to help your organization keep up? 

 

Lifetime Learning

A lovely Saturday:

A perfect day for some refreshment:

Studying http://philip.greenspun.com/teaching/rdbms-iap-2011

Why?  (And, why now?)  Relational databases and SQL have been around for forty years.  Yet, no reasonable business person would disagree that:

1. it's useful to know how to use spreadsheet software, both to DIY and manage others who do;

2. there's much more information out there today;

3. harnessing this information is not only advantageous but essential;

4. more powerful tools like database management systems are necessary for this.

Therefore, business people should know a little bit about these more powerful tools, to continue to be considered reasonable.

#Google Search and The Limits of #Location

I broke my own rule earlier today and twitched (that's tweeted+*itched -- you read it here first) an impulsive complaint about how Google does not allow you to opt out of having it consider your location as a relevance factor in the search results it offers you:

I don't take it back.  But, I do think I owe a constructive suggestion for how this could be done, in a way that doesn't compromise the business logic I infer behind this regrettable choice.  Plus, I'll lay out what I infer this logic to be, and the drivers for it, in the hope that someone can improve my understanding.  Finally, I'll lay out some possible options for SEO in an ever-more-local digital business context.

OK, first, here's the problem.  In one client situation I'm involved with, we're designing an online strategy with SEO as a central objective.  There are a number of themes we're trying to optimize for.  One way you improve SEO is to identify the folks who rank / index highly on terms you care about, and cultivate a mutually valuable relationship in which they eventually may link to relevant content you have on a target theme.  To get a clean look at who indexes well on a particular theme and related terms, you can de-personalize your search.  You do this with a little url surgery:

Start with the search query:

http://www.google.com/search?q=[theme]

Then graft on a little string to depersonalize the query:

http://www.google.com/search?q=[theme]&pws=0

Now, when I did this, I noticed that Google was still showing me local results.  These usually seem less intrusive.  But now, like some invasive weed, they'd choked off my results, ranging all the way to the third position and clogging up most of the rest of the first page, for a relatively innocuous term ("law"; lots of local law firms, I guess).  

Then I realized that "&pws=0" tells Google to stop rummaging around in the cookies it's set on my browser, plus other information in my http requests, and won't help me prevent Google guessing / using my location, since that's based on the location of the ISP's router between my computer and the Google cloud.

 Annoyed, I poked around to see what else I could do about it.  Midway down the left-hand margin of the search results page, I noticed this:

 

So naturally, my first thought was to specify "none", or "null", to see if I could turn this off.  No joy. 

Next, some homework to see if there's some way to configure my way out of this.  That led me to Rishi's post (see the third answer, dated 12/2/2010, to the question).  

Unbelieving that an organization with as fantastic a UI aesthetic -- that is to say, functional / usable in the extreme -- as Google would do this, I probed further. 

First stop: Web Search Help.  The critical part:

Q. Can I turn off location-based customization?

A. The customization of search results based on location is an important component of a consistent, high-quality search experience. Therefore, we haven't provided a way to turn off location customization, although we've made it easy for you to set your own location or to customize using a general location as broad as the country that matches your local domain...

Ah, so, "It's a feature, not a bug." :-)

...If you find that your results for a particular search are more local than what you're looking for, you can set your location to a broader geographical area (such as a country instead of a city, zip code, or street address). Please note that this will greatly reduce the amount of locally relevant results that you’ll see. [emphasis mine]

 Exactly!  So I tried to game the system:

Drat!  Foiled again.  Ironic, this "Location not recognized" -- from the people who bring us Google Earth!

Surely, I thought, some careful consideration must have gone into turning the Greatest Tool The World Has Ever Known into the local Yellow Pages.  So, I checked the Google blog.  A quick search there for "location", and presto, this. Note that at this point, February 26, 2010, it was still something you could add.  

Later, on October 18, 2010 -- where I have I been? -- this, which effectively makes "search nearby" non-optional:

We’ve always focused on offering people the most relevant results. Location is one important factor we’ve used for many years to customize the information that you find. For example, if you’re searching for great restaurants, you probably want to find ones near you, so we use location information to show you places nearby.

Today we’re moving your location setting to the left-hand panel of the results page to make it easier for you to see and control your preferences. With this new display you’re still getting the same locally relevant results as before, but now it’s much easier for you to see your location setting and make changes to it.

(BTW, is it just me, or is every Google product manager a farmer's-market-shopping, restaurant-hopping foodie?  Just sayin'... but I seriously wonder how much designers' own demographic biases end up influencing assumptions about users' needs and product execution.)

Now, why would Google care so much about "local" all of a sudden?  Is it because Marissa Mayer now carries a torch for location (and Foursquare especially)?  Maybe.  But it's also a pretty good bet that it's at least partly about the Benjamins.  From the February Google post, a link to a helpful post on SocialBeat, with some interesting snippets: 

"Location may get a central place in Google’s web search redesign"

Google has factored location into search results for awhile without explicitly telling the user that the company knows their whereabouts. It recently launched ‘Nearby’ search in February, returning results from local venues overlaid on top of a map.

Other companies also use your IP address to send you location-specific content. Facebook has long served location-sensitive advertising on its website while Twitter recently launched a feature letting users geotag where they are directly from the site. [emphasis mine]

Facebook's stolen a march on Google in the social realm (everywhere but Orkut-crazed Brazil; go figure).  Twitter's done the same to Google on the real-time front.  Now, Groupon's pay-only-for-real-sales-and-then-only-if-the-volumes-justify-the-discount threatens the down-market end of Google's pay-per-click business with a better mousetrap, from the small biz perspective.  (BTW, that's why Groupon's worth $6 billion all of a sudden.)  All of these have increasingly (and in Groupon's case, dominantly) local angles  where the value to both advertiser and publisher (Facebook / Twitter / Groupon) are presumably highest.

Ergo, Google gets more local.  But that's just playing defense, and Eric, Sergey, Larry, and Marissa are too smart (and, with $33 billion in cash on hand, too rich) to do just that.

Enter Android.  Hmm.  Just passed Apple's iOS and now is running the table in the mobile operating system market share game.  Why wouldn't I tune my search engine to emphasize local search results, if more and more of the searches are coming from mobile devices, and especially ones running my OS?  Yes, it's an open system, but surely dominating it at multiple layers means I can squeeze out more "rent", as the economists say?

The transcript of Google's Q3 earnings call is worth a read.

Now, back to my little problem.  What could Google do that would still serve its objective of global domination through local search optimization, while satisfying my nerdy need for "de-localized" results?  The answer's already outlined above -- just let me type in "world", and recognize it for the pathetic niche plea that it is.  Most folks will never do this, and this blog's not a bully-enough pulpit to change that. Yet.

The bigger question, though, is how to do SEO in a world where it's all location, location, location, or as SEOmoz writes

"Is Every Query Local Now?" 

Location-based results raise political debates, such as "this candidate is great" showing up as the result in one location while "this candidate is evil" in another.  Location-based queries may increase this debate.  I need only type in a candidate's name and Instant will tell me what is the prevailing opinion in my area.  I may not know if that area is the size of a city block or the entire world, but if I am easily influenced then the effect of the popular opinion has taken one step closer (from search result to search query) to the root of thought.   The philosphers among you can debate whether or not the words change the very nature of ideas.

Heavy.

OK, never leave without a recommendation.  Here are two:

First, consider that for any given theme, some keywords might be more "local" than others.  Under the theme "Law", the keyword "law" will dredge up a bunch of local law firms.  But another keyword, say "legal theory", is less likely to have that effect (until discussing that topic in local indie coffee shops becomes popular, anyway).  So you might explore re-optimizing for these less-local alternatives.  (Here's an idea: some enterprising young SEO expert might build a web service that would, for any "richly local" keyword, suggest less-local alternatives from a crowd-sourced database compiled by angry folks like me.  Sort of a "de-localization thesaurus".  Then, eventually, sell it to a big ad agency holding company.)

Second, as location kudzu crawls its way up Google's search results, there's another phenomenon happening in parallel.  These days, for virtually any major topic, the Wikipedia entry for it sits at or near the top of Google's results.  So, if as with politics, now too search and SEO are local, and much harder therefore to play, why not shift your optimization efforts to the place that the odds-on top Google result will take you, if theme leadership is a strategic objective?

 

PS Google I still love you.  Especially because you know where I am. 

 

Analytics Commons Post in Google Analytics Blog Today @analyticscommns @linchen @perryhewitt #analytics

Our Analytics Commons project (which I previously wrote about here) got written up in a post on the Google Analytics blog today. ( Thanks to Nick Mihailovski at Google, and to Perry Hewitt at Harvard!  And of course to my partners Lin and Kehan at New Circle Consulting!)

Fly-By-Wire Marketing, Part II: The Limits Of Real Time Personalization

A few months ago I posted on what I called "Fly-By-Wire Marketing", or the emergence of the automation of marketing decisions -- and sometimes the automation of the development of rules for guiding those decisions.

More recently Brian Stein introduced me to Hunch, the new recommendation service founded by Caterina Fake of Flickr fame.  (Here's their description of how it works.  Here's my profile, I'm just getting going.)  When you register, you answer questions to help the system get to know you.  When you ask for a recommendation on a topic, the system not only considers what others have recommended under different conditions, but also what you've told it about you, and how you compare with others who have sought advice on the subject.

It's an ambitious service, both in terms of its potential business value (as an affiliate on steroids), but also in terms of its technical approach to "real time personalization".  Via Sim Simeonov's blog, I read this GigaOm post by Tom Pinckney, a Hunch co-founder and their VP of Engineering.  Sim's comment sparked an interesting comment thread on Tom's post.  They're useful to read to get a feel for the balance between pre-computation and on-the-fly computation, as well as the advantages of and limits to large pre-existing data sets about user preferences and behavior, that go into these services today.

One thing neither post mentions is that there may be diminishing returns to increasingly powerful recommendation logic if the set of things from which a recommendation can ultimately be selected is limited at a generic level.  For example, take a look at Hunch's recommendations for housewarming gifts.  The results more or less break down into wine, plants, media, and housewares.  Beyond this level, I'm not sure the answer is improved by "the wisdom of Hunch's crowd" or "Hunch's wisdom about me", as much as my specific wisdom about the person for whom I'm getting the gift, or maybe by what's available at a good price. (Perhaps this particular Hunch "topic" could be further improved by crossing recommendations against the intended beneficiary's Amazon wish list?)

My point isn't that Hunch isn't an interesting or potentially useful service.  Rather, as I argued several months ago,

The [next] question you ask yourself is, "How far down this road does it makes sense for me to go, by when?"  Up until recently, I thought about this with the fairly simplistic idea that there are single curves that describe exponentially decreasing returns and exponentially increasing complexity.  The reality is that there are different relationships between complexity and returns at different points -- what my old boss George Bennett used to call "step-function" change.

For me, the practical question-within-a-question this raises is, for each of these "step-functions", is there an version of the algorithm that's only 20% as complex, that gets me 80% of the benefit?  My experience has been that the answer is usually "yes".  But even if that weren't the case, my approach in jumping into the uncharted territory of a "step-function" change in process, with new supporting technology and people roles, would be to start simple and see where that goes.

At minimum, given the "step-function" economics demonstrated by the Demand Medias of the world, I think senior marketing executives should be asking themselves, "What does the next 'step-function' look like?", and "What's the simplest version of it we should be exploring?" (Naturally, marketing efforts in different channels might proceed down this road at different paces, depending on a variety of factors, including the volume of business through that channel, the maturity of the technology involved, and the quality of the available data...)

Hunch is an interesting specific example of the increasingly broad RTP trend.  The NYT had an interesting article on real time bidding for display ads yesterday, for example.  The deeper issue in the trend I find interesting is the shift in power and profit toward specialized third parties who develop the capability to match the right cookie to the right ad unit (or, for humans, the right user to the right advertiser), and away from publishers with audiences.  In the case of Hunch, they're one and the same, but they're the exception.  How much of the increased value advertisers are willing to pay for better targeting goes to the specialized provider with the algorithm and the computing power, versus the publisher with the audience and the data about its members' behavior?  And for that matter, how can advertisers better optimize their investments across the continuum of targeting granularity?  Given the dollars now flooding into digital marketing, these questions aren't trivial.

#Adobe: Duct Tape for the "Splinternet"

(Previously titled: "Adobe: Up In The Air")

As folks line up for the iPad, SXSW rages, and the Splinternet splinters, if you own a smartphone or plan to own one, or a tablet, or if you're about to commission an app for one of these platforms, this post is for you.

A couple of years ago, Adobe seemed to have positioned itself smartly for global domination.  The simple logic:

• Online experiences becoming richer
• Adobe makes tools for rich experiences (Flash, Flex, Air)
• Ergo, Adobe becomes richer

Or for you Mondrian fans, the visual version of Adobe's "All Mine!"

Oh that it were that simple.  So, Apple, also vaguely interested in rich immersive experiences as its path out of the hip hardware niche toward intergalactic domination, plays the digital Soup Nazi: "No Flash support for you!"  Again, for the Visualistas:

The nerve!  As if that weren't bad enough, there are those pesky evolving standards to stay ahead of.  HTML 5 now rides into town to save the Internet garden from the weedy assault of proprietary browser plugins (Flash, Gears, Silverlight) for supporting rich experiences (read as: need more client-side processing and storage than HTML 4 + browsers could offer).  Like any abstraction, it has performance compromises.  But, with powerful friends behind it with a shared interest in taking down the de facto rich experience standard -- Flash is on basically every non-mobile browser out there -- HTML 5 will get better, if like any standard, slowly.  The picture:

For you conspiracy theorists, a Smoking Gun:

link

Now, those Adobe folks are pretty smart too, and they aren't sitting still.  Basically, their strategy amounts to two things:

• "Duct tape for the Splinternet", aka "Son Of Java" -- the ability to develop your app in their tools, and then compile them for whatever platform you'd like to publish to, including Apple's.  Remember "Write once, run anywhere?"  Of course you give a few things up -- some features, some performance.  But if you're a publisher, pretty tempting! 
• Do what Microsoft did a few years ago -- "Embrace and Extend".  Basically, agree to play nicely with the non-threatening parts of the HTML 5 spec while continuing to extend the feature set and performance of Flash so it's preferred on the margin as the environment for the coolest rich experiences. For example, one way -- now that Adobe owns Omniture -- to extend the feature set might be to embed analytic tracking into the application layer.

Here's a good interview Rob Scoble did with the Adobe guys where they explain all this in 22 minutes.  Here's my graphic translation of the interview:

 
A while ago I wrote a post on strategy in the software business that forms the frame for how I try to understand what's happening.  I think it still makes sense, but I'm eager to hear suggestions for improving it!

So what?  What does this mean for the publishers who are trying to figure out how to respond to the Splinternet?  I think it makes sense, as always, to start with The User.  Is what you are trying to do for him or her sufficiently exotic (and rewardably so) that you need the unique capabilities of each smartphone's / tablet's native OS / SDK?  Or is the idea sufficiently "genius" that you don't need to tart it up with whizziness, and can accept certain limitations in exchange for "Write once, run anywhere?"

I'd predict that Adobe will make common cause with some hardware manufacturer(s) -- HP, anyone?  It will be interesting to see what Adobe's willing to trade off for that support.

Where's Microsoft in all this?

Filtering The Collective Preconscious: Darwin Ecosystem

More and more, people agree that filtering the flood of information that's coming at us is supplanting publishing, finding, and connecting as the problem of the Information Age.  Today, the state of the art for doing this includes several approaches:

• Professional filters: we follow people whose jobs are to cover an area.  Tom Friedman covers international issues, Walt Mossberg covers personal technology.
• Technical filters: we use services like Google Alerts to tell us when there's something new on a topic we're interested in
• Social filters: we use services like Digg, Reddit, and Stumbleupon to point us to popular things
• Tribal filters: we use Facebook, Twitter, LinkedIn, and (Google hopes) Buzz to get pointed to things folks we know and trust think are important

In addition to what gets through, there's how it's presented.  RSS readers for example offer a huge productivity boost to anyone trying to keep up with more than a few sources of information.  However, once you get several hundreds items in your RSS reader, unsorted by anything other than "last in", it's back to information overload.  To solve this, innovative services like Newsmap provide multi-dimensional visual displays to try to push your information awareness productivity even further.  But so far, they've seen only modest adoption.

One limitation of today's filtering and productivity tools is that they pick items up either too early, before it's clear they represent something meaningful, or too late, once the advantages of recognizing a trend have passed.

Yesterday, I visited the team behind a new service called Darwin Ecosystem (video showing it off here) that takes a different and potentially more powerful and useful approach to helping you "filter the collective preconscious"  -- that is, to identify emergent signals in the vast noise of the Internet (or any other body of information you might point to -- say, for example, customer service call logs).  Co-founder and CEO Thierry Hubert is a veteran of the knowledge management world going back to senior technical roles at Lotus and IBM; his partner Frederic Deriot shares similar experiences; and, my friend Bill Ives -- formerly head of Accenture's KM client practice -- is also involved as VP Marketing.

Briefly -- look at the demo, it does a better job of explaining itself -- the service presents a tag cloud of topics that it thinks represent emergent themes to pay attention to in the "corpus" filled by sources you point it to (in the demo, sources run to hundreds of news sources and social media).  The bigger the font, the more important the theme.  Hover your mouse over a theme, and it highlights other related themes to put them all into a collective context.  The service also provides a dynamic view of what's hot / not with a stock-ticker-style ribbon running at the top of the page.  You can view the cloud of emergent themes either in an "unfiltered view", or more usefully, filtered with "attractor" keywords you can specify.

This interface, while interesting, will likely not be the eventual "user/use-case" packaging of the service.  I can see this as a built-in "front page" for an RSS reader, for example, or, minus the tag cloud, as the basis for a more conventional looking email alert service.

The service is based on the math behind Chaos Theory.  This is the math that helps us understand how the proverbial beating of a butterfly's wings in China might become a massive storm.  (Math nerds will appreciate the Lorenz-attractor-plot-as-butterfly-wings logo.)  The service uses this math to tell you not only what individual topics are gaining or losing momentum, but also to highlight relationships between and among different topics to put them into context  -- like why "underwear" and "bomber" might be related. 

Now in beta, with a few large organizations (including large media firms) as early adopters, the service has had some early wins that demonstrate its potential.  It told users, for example, that Lou Dobbs might be on his way out at CNN a week before his departure was reported in the mainstream press.  It also picked up news of UCLA's planned tuition hikes 48 hours in advance of this getting reported in popular mainstream or social media.

It strikes me that a service like Darwin is complementary to that of  Crimson Hexagon, a sentiment analysis firm based on Prof. Gary King's work at Harvard (here's the software that came out of that work), with a variety of marketing, media, and customer support applications.  Darwin helps tell you what to pay attention to -- suggests emergent themes and their context; Crimson Hexagon can then tell you how people feel about these issues in a nuanced way, beyond simple positive / negative buzz.

The current business model has Darwin pursuing enterprise licensing deals with major firms, but depending on partners that emerge, that may not be the last stop on the adoption / monetization express.  For example, it seems to me that a user's interaction with a tool like Darwin represents highly intentional behavior that would be useful data for ad / offer targeting, or personalization of content generally.  This potential use as a marketing analytics input makes it especially interesting to me.

Bottom line: if you are responsible for syndicating and helping users usefully navigate a highly dynamic information set collected through a multitude of sources -- say, a news organization, a university, a large consumer products or services firm -- and are evaluating monitoring technologies, Darwin is worth a look.

Beyond #A/B and #MVT #Testing: Optimizing Experiences

I caught up recently with Jeff Eckman and Geordie McClelland of BGC to learn more about their service for deploying, testing, and optimizing direct-response page flows.  The easy-to-use service (I saw a live demo), which is based on ion Interactive Pres & CTO Scott Brinker's LiveBall platform, combines a template-based authoring tool, CMS platform, and testing engine.  A marketing organization creates a subdomain on its web domain, and then works with BGC to develop different landing "experiences" (sequences of pages arrayed along a logical path) on it, instead of a single landing page or static, product-focused microsites.  Then the system tracks conversion rates along and through these experiences and, with full transparency to the marketers, shifts a greater share of requests to the winner.  Meanwhile, the marketers develop new page elements, pages, and paths based on what the more granular feedback about those suggests might make sense.

Contrary to popular wisdom about how fewer clicks drive increased conversion rates, BGC has found that an experience that asks for an "optimum" amount of information in an "optimum" series of screens delivers far higher results: on average 300% higher and as much as 3000+% higher in the 3+ years this practice has been in use.  I asked if their experience across multiple clients had yet suggested an overall rule of thumb, such as, "For products in this price range, three pages and three questions to each page work best", to help jump-start the creative process and save on initial creative costs, but Jeff and Geordie hadn't seen one emerge yet, and were somewhat skeptical that it could / would be worth it to pursue.  I struggled with that answer for a bit, but it's occurred to me that maybe I need to break out of my heuristic-seeking box and accept that in 2010 and beyond, the Process is the Answer (see my earlier post "Personalization Is A Process").  Jeff and Geordie are putting their money where their mouths are on this point:  The service can be offered on an easy-to-try-and-buy CPA basis (after a very modest setup fee).

We discussed technical directions that could broaden the span of the experiences they can engineer and test.  For example, you might imagine that an experience might start within a Facebook application for a couple of screens, and then make the jump to your site for the rest of the process and, hopefully, a conversion.  While they haven't done one of these yet, since a url is a url and its contents can be rendered ecumenically, this should be straightforward.

I haven't seen all the direct and indirect competition yet, but BGC's solution seems fairly unique and easy to use.  In theory you could cobble a GA-based solution together that combines their funnel analysis with their Web Optimizer A/B testing tool, but that would be clunky and likely brittle, perhaps a false economy.  And, not too long ago I wrote about Sitespect's url tunneling capability, but it strikes me that while somewhat similar, the two are currently focused on different parts of the "attract-engage-convert" process.

Bottom line: have a look, especially if you use form-based landing pages to qualify and convert leads for higher-price-point products and services.

What's NYT.com Worth To You, Part II

OK, with the response curve for my survey tailing off, I'm calling it.  Here, dear readers, is what you said (click on the image to enlarge it):

(First, stats: with ~40 responses -- there are fewer points because of some duplicate answers -- you can be 95% sure that answers from the rest of the ~20M people that read the NYT online would be +/- 16% from what's here.)

90% of respondents would pay at least $1/month, and several would pay as much as $10/month. And, folks are ready to start paying after only ~2 articles a day.  Pretty interesting!  More latent value than I would have guessed.  At the same time, it's also interesting to note that no one went as high as the $14 / month Amazon wants to deliver the Times on the Kindle. (I wonder how many Kindle NYT subs are also paper subs getting the Kindle as a freebie tossed in?)

Only a very few online publishers aiming at "the general public" will be able to charge for content on the web as we have known it, or through other newer channels.  Aside from highly-focused publishers whose readers can charge subscriptions to expense accounts, the rest of the world will scrape by on pennies from AdSense et al. 

But, you say, what about the Apple Tablet (announcement tomorrow! details yesterday), and certain publishers' plans for it?  I see several issues:

• First, there's the wrestling match to be had over who controls the customer relationship in Tabletmediaworld. 
• Second, I expect the rich, chocolatey content (see also this description of what's going in R&D at the Times) planned for this platform and others like it to be more expensive to produce than what we see on the web today, both because a) a greater proportion of it will be interactive (must be, to be worth paying for), but also because b) producing for multiple proprietary platforms will also drive costs up (see for example today's good article in Ad Age by Josh Bernoff on the "Splinternet"). 
• Third, driving content behind pay walls lowers traffic, and advertising dollars with it, raising the break-even point for subscription-based business models. 
• Fourth, last time I checked, the economy isn't so great. 

The most creative argument I've seen "for" so far is that pushing today's print readers/ subscribers to tablets will save so much in printing costs that it's almost worth giving readers tablets (well, Kindles anyway) for free -- yet another edition of the razor-and-blade strategy, in "green" wrapping perhaps.

The future of paid content is in filtering information and increasing its utility.  Media firms that deliver superior filtering and utility at fair prices will survive and thrive.  Among its innovations in visual displays of information (which though creative, I'd guess have a limited monetization impact) is evidence that the Times agrees with this, at least in part (from the article on Times R&D linked to above):

When Bilton swipes his Times key card, the screen pulls up a personalized version of the paper, his interests highlighted. He clicks a button, opens the kiosk door, and inside I see an ordinary office printer, which releases a physical printout with just the articles he wants. As it prints, a second copy is sent to his phone.

The futuristic kiosk may be a plaything, but it captures the essence of R&D’s vision, in which the New York Times is less a newspaper and more an informative virus—hopping from host to host, personalizing itself to any environment.

Aside from my curiosity about the answers to the survey questions themselves, I had another reason for doing this survey.  All the articles I saw on the Times' announcement that it would start charging had the usual free-text commenting going.  Sprinkled through the comments were occasional suggestions from readers about what they might pay, but it was virtually impossible to take any sort of quantified pulse on this issue in this format.  Following "structured collaboration" principles, I took five minutes to throw up the survey to make it easy to contribute and consume answers.  Hopefully I've made it easier for readers to filter / process the Times' announcement, and made the analysis useful as well -- for example, feel free to stick the chart in your business plan for a subscription-based online content business ;-)  If anyone can point me to other, larger, more rigorous surveys on the topic, I'd be much obliged.

The broader utility of structuring the data capture this way is perhaps greatest to media firms themselves:  indirectly for ad and content targeting value, and perhaps because once you have lots of simple databases like this, it becomes possible to weave more complex queries across them, and out of these queries, some interesting, original editorial possibilities.

Briefly considered, then rejected for its avarice and stupidity: personalized pricing offers to subscribe to the NYT online based on how you respond to the survey :-)

Postscript: via my friend Thomas Macauley, NY (Long Island) Newsday is up to 35 paid online subs.