Filtering The Collective Preconscious: Darwin Ecosystem

More and more, people agree that filtering the flood of information that's coming at us is supplanting publishing, finding, and connecting as the problem of the Information Age.  Today, the state of the art for doing this includes several approaches:

• Professional filters: we follow people whose jobs are to cover an area.  Tom Friedman covers international issues, Walt Mossberg covers personal technology.
• Technical filters: we use services like Google Alerts to tell us when there's something new on a topic we're interested in
• Social filters: we use services like Digg, Reddit, and Stumbleupon to point us to popular things
• Tribal filters: we use Facebook, Twitter, LinkedIn, and (Google hopes) Buzz to get pointed to things folks we know and trust think are important

In addition to what gets through, there's how it's presented.  RSS readers for example offer a huge productivity boost to anyone trying to keep up with more than a few sources of information.  However, once you get several hundreds items in your RSS reader, unsorted by anything other than "last in", it's back to information overload.  To solve this, innovative services like Newsmap provide multi-dimensional visual displays to try to push your information awareness productivity even further.  But so far, they've seen only modest adoption.

One limitation of today's filtering and productivity tools is that they pick items up either too early, before it's clear they represent something meaningful, or too late, once the advantages of recognizing a trend have passed.

Yesterday, I visited the team behind a new service called Darwin Ecosystem  that takes a different and potentially more powerful and useful approach to helping you "filter the collective preconscious"  -- that is, to identify emergent signals in the vast noise of the Internet (or any other body of information you might point to -- say, for example, customer service call logs).  Co-founder and CEO Thierry Hubert is a veteran of the knowledge management world going back to senior technical roles at Lotus and IBM; his partner Frederic Deriot shares similar experiences; and, my friend Bill Ives -- formerly head of Accenture's KM client practice -- is also involved as VP Marketing.

Briefly, the service presents a tag cloud of topics that it thinks represent emergent themes to pay attention to in the "corpus" filled by sources you point it to (in the demo, sources run to hundreds of news sources and social media).  The bigger the font, the more important the theme.  Hover your mouse over a theme, and it highlights other related themes to put them all into a collective context.  The service also provides a dynamic view of what's hot / not with a stock-ticker-style ribbon running at the top of the page.  You can view the cloud of emergent themes either in an "unfiltered view", or more usefully, filtered with "attractor" keywords you can specify.

This interface, while interesting, will likely not be the eventual "user/use-case" packaging of the service.  I can see this as a built-in "front page" for an RSS reader, for example, or, minus the tag cloud, as the basis for a more conventional looking email alert service.

The service is based on the math behind Chaos Theory.  This is the math that helps us understand how the proverbial beating of a butterfly's wings in China might become a massive storm.  (Math nerds will appreciate the Lorenz-attractor-plot-as-butterfly-wings logo.)  The service uses this math to tell you not only what individual topics are gaining or losing momentum, but also to highlight relationships between and among different topics to put them into context  -- like why "underwear" and "bomber" might be related. 

Now in beta, with a few large organizations (including large media firms) as early adopters, the service has had some early wins that demonstrate its potential.  It told users, for example, that Lou Dobbs might be on his way out at CNN a week before his departure was reported in the mainstream press.  It also picked up news of UCLA's planned tuition hikes 48 hours in advance of this getting reported in popular mainstream or social media.

It strikes me that a service like Darwin is complementary to that of  Crimson Hexagon, a sentiment analysis firm based on Prof. Gary King's work at Harvard (here's the software that came out of that work), with a variety of marketing, media, and customer support applications.  Darwin helps tell you what to pay attention to -- suggests emergent themes and their context; Crimson Hexagon can then tell you how people feel about these issues in a nuanced way, beyond simple positive / negative buzz.

The current business model has Darwin pursuing enterprise licensing deals with major firms, but depending on partners that emerge, that may not be the last stop on the adoption / monetization express.  For example, it seems to me that a user's interaction with a tool like Darwin represents highly intentional behavior that would be useful data for ad / offer targeting, or personalization of content generally.  This potential use as a marketing analytics input makes it especially interesting to me.

Bottom line: if you are responsible for syndicating and helping users usefully navigate a highly dynamic information set collected through a multitude of sources -- say, a news organization, a university, a large consumer products or services firm -- and are evaluating monitoring technologies, Darwin is worth a look.

Comments

[Bill Ives]

Cesar - Thanks for your kind words and advice. Both are much appreciated. Bill