The predictive power of online chatter

ABSTRACT

An increasing fraction of the global discourse is migrating online in the form of blogs, bulletin boards, web pages, wikis, editorials, and a dizzying array of new collaborative technologies. The migration has now proceeded to the point that topics reflecting certain individual products are sufficiently popular to allow targeted online tracking of the ebb and flow of chatter around these topics. Based on an analysis of around half a million sales rank values for 2,340 books over a period of four months, and correlating postings in blogs, media, and web pages, we are able to draw several interesting conclusions.First, carefully hand-crafted queries produce matching postings whose volume predicts sales ranks. Second, these queries can be automatically generated in many cases. And third, even though sales rank motion might be difficult to predict in general, algorithmic predictors can use online postings to successfully predict spikes in sales rank.

REFERENCES

• 1: E. Adar, L. Zhang, L. A. Adamic, and R. M. Lukose. Implicit structure and the dynamics of blogspace. Workshop on the Weblogging Ecosystem, 13th International World Wide Web Conference, 2004.
• 2: A. Admati and P eiderer. Disclosing information on the internet: Is it noise or is it news? Technical report, Graduate School of Business, Stanford University, 2001.
• 3: J. Allan, J. Carbonell, G. Doddington, J. Yamron, and Y. Yang. Topic detection and tracking pilot study: Final report. Proc. of the DARPA Broadcast News Transcription and Understanding Workshop, 1998.
• 4: W. Antweiler and M. Z. Frank. Is all that talk just noise? The information content of Internet stock message boards. Journal of Finance, 59(3):1259--1295, 2004.
• 5: S. Arbesman. The memespread project: An initial analysis of the contagious nature of information in online networks. http://www.arbesman.net/memespread.pdf, 2004.
• 6: Biz360. Market360 product datasheet. Technical report, Biz360, 2004.
• 7: P. Blackshaw and M. Nazzaro. Consumer-generated media (cgm) 101. Technical report, Intelliseek, 2004.
• 8: George Edward Pelham Box , Gwilym M. Jenkins, Time Series Analysis: Forecasting and Control, Prentice Hall PTR, Upper Saddle River, NJ, 1994
• 9: Carma. How doe we gain an understanding of the media environment on our company as our industry comes under scrutiny? Technical report, Carma, 2004.
• 10: C. Chatfield. The Analysis of Time Series. Chapman and Hall, 1984.
• 11: Stephen Dill , Nadav Eiron , David Gibson , Daniel Gruhl , R. Guha , Anant Jhingran , Tapas Kanungo , Sridhar Rajagopalan , Andrew Tomkins , John A. Tomlin , Jason Y. Zien, SemTag and seeker: bootstrapping the semantic web via automated semantic annotation, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775178]
• 12: D. Sornette, F. Deschâtres, T. Gilbert, and Y. Ageon. Endogenous versus exogenous shocks in complex networks: An empirical test using book sale rankings. Physical Review Letters, 93(228701), 2004.
• 13: D. Gruhl , L. Chavet , D. Gibson , J. Meyer , P. Pattanayak , A. Tomkins , J. Zien, How to build a WebFountain: An architecture for very large-scale text analytics, IBM Systems Journal, v.43 n.1, p.64-77, January 2004
• 14: Daniel Gruhl , R. Guha , David Liben-Nowell , Andrew Tomkins, Information diffusion through blogspace, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988739]
• 15: Jon Kleinberg, Bursty and hierarchical structure in streams, Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, July 23-26, 2002, Edmonton, Alberta, Canada [doi>10.1145/775047.775061]
• 16: Ravi Kumar , Jasmine Novak , Prabhakar Raghavan , Andrew Tomkins, On the bursty evolution of blogspace, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775233]
• 17: Ravi Kumar , Jasmine Novak , Prabhakar Raghavan , Andrew Tomkins, Structure and evolution of blogspace, Communications of the ACM, v.47 n.12, December 2004 [doi>10.1145/1035134.1035162]
• 18: J. Lin and A. Halavais. Mapping the blogosphere in america. Workshop on the Weblogging Ecosystem, 13th International World Wide Web Conference, 2004.
• 19: R. Papka, On-line New Event Detection, Clustering, and Tracking TITLE2:, University of Massachusetts, Amherst, MA, 1999
• 20: David A. Smith, Detecting and Browsing Events in Unstructured text, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564391]
• 21: R. Tong. Detecting and tracking opinions in on-line discussions. UCB/SIMS Web Mining Workshop, 2001.
• 22: R. Tumarkin and R. F. Whitelaw. News or noise? internet postings and stock prices. Financial Analysts Journal, pages 41--51, 2001.
• 23: B. Whitman and S. Lawrence. Inferring descriptions and similarity for music from community metadata. In Proc. of the 2002 International Computer Music Conference, pages 591--598, 2002.
• 24: Yiming Yang , Tom Pierce , Jaime Carbonell, A study of retrospective and on-line event detection, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.28-36, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290953]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Experimentation, Measurement

Keywords:
blogs, prediction, sales rank, time-series analysis