Swarm-based Web Coolhunting with Condor
The Web has become a mirror of the real world. Reading and interpreting the collective mind allows discovering trends for weeks into the future. Our coolhunting approach gathers open-source information, analyzes and filters it to identify trends and leaders, and provides interfaces supporting various visualizations and interactive data analysis using our software tool CONDOR. CONDOR was developed and tested over the last ten years at the MIT Center for Collective Intelligence, Dartmouth College, University of Cologne, and the University of Applied Sciences Northwestern Switzerland. CONDOR applies social network analysis to discover Collaborative Innovation Networks (COINs) (Gloor 2006). To put it in other words, CONDOR is discovering trends through the identification of trendsetters by measuring their social network position, and filtering their discussion by contents, creating subnetworks of people talking about the same topics. At the same time CONDOR is combining these efforts with automatic sentiment analysis.
Our coolhunting approach combines 4 dimensions: people (actors),
contents (facts), sentiment, and time series analysis. Coolhunting
calculates predictive models by combining five unique components:
(1) CONDOR finds trends by finding the trendsetters. The key
differentiator of our approach is the use of social networking
techniques to determine those actors that are most influential. Our
approach weighs every instance of a search term with the social
networking position of the actor using the search term.
(2) CONDOR uses implicit data to interpret “honest signals.” One
of the core concepts of CONDOR’s analysis philosophy is that the
metadata surrounding the explicit data associated with a posting
(e.g. author, date, text) is as important as the data that expresses
the hidden intent of the author, frequently without him/her actually
being aware of it. For example, Wikipedia not only provides content,
but these articles are linked together, allowing users to easily
view these explicit connections. Wikipedia has a great amount of
hidden, latent value in the metadata surrounding these content
pages: the authors (and what else they’ve authored), the course of
activity (edits/time), the location of links in the content, the
networks of links and authorship. The analysis of this metadata
contributes greatly to the quality of the early discovery of events
through the many types of context it provides.
(3) CONDOR segments the Web data into three information spheres:
Crowd, Experts, and Swarms. It predicts societal trends based on Web
buzz time series captured through identifying and weighing the
appropriate data source in the three different information spheres.
The crowd’s output is collected by analyzing the Web through the
Google/Bing/Yahoo search indices, the experts’ output is collected
through Wikipedia and scientific records such as PubMed, and the
swarm is found in online forums such as IMDB or Yahoo Finance. The
social network of actors and their honest signals are computed
differently for each of the different information spheres.
(4) Content: Through automated text and sentiment analysis
coolhunting analyzes if people talk positively or negatively about a
certain trend, and what its key attributes are.
(5) Longitudinal trend observation: By continuously monitoring
changes and applying customized longitudinal social network analysis
algorithms, coolhunting generates trend curves and predictions for
tomorrow’s trends in key blogs, Facebook and Twitter influencers,
Wikipedia, and relevant forums.
|
Crowd (Twitter & Web) |
Swarm (Wikipedia) |
Expert (News, Blogs) |
General
overview |
Megatrends,
key people |
Megatrends,
key people |
Megatrends,
key people |
Compare
key markets/countries |
What
the crowd says in the US, Europe, Asia, etc |
What
the swarm says in the US, Europe, Asia, etc |
What
the experts says in the US, Europe, Asia, etc |
Compare
key competitors |
Comparing
related topics on twitter |
Comparing
related topics on Wikipedia |
Comparing
related topics on blogs |
Coolhunting identifies:
1. What are information producers saying online in forums, Twitter, blogs, and Wikipedia
2. In which different geographic regions
3. Who are the opinion leaders
4. What is the demographics of influencers
5. What are the key Web sites