Extracting collective probabilistic forecasts from web games

ABSTRACT

Game sites on the World Wide Web draw people from around the world with specialized interests, skills, and knowledge. Data from the games often reflects the players' expertise and will to win. We extract probabilistic forecasts from data obtained from three online games: the Hollywood Stock Exchange (HSX), the Foresight Exchange (FX), and the Formula One Pick Six (F1P6) competition. We find that all three yield accurate forecasts of uncertain future events. In particular, prices of so-called "movie stocks" on HSX are good indicators of actual box office returns. Prices of HSX securities in Oscar, Emmy, and Grammy awards correlate well with observed frequencies of winning. FX prices are reliable indicators of future developments in science and technology. Collective predictions from players in the F1 competition serve as good forecasts of true race outcomes. In some cases, forecasts induced from game data are more reliable than expert opinions. We argue that web games naturally attract well-informed and well-motivated players, and thus offer a valuable and oft-overlooked source of high-quality data with significant predictive value.

REFERENCES

• 1: M. M. All. Probability and utility estimates for racetrack bettors. Journal of Political Economy, 85(4):803-816, 1977.
• 2: Doug Beeferman , Adam Berger, Agglomerative clustering of a search engine query log, Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, p.407-416, August 20-23, 2000, Boston, Massachusetts, United States [doi>10.1145/347090.347176]
• 3: R. T. Clemen. Combining forecasts: A review and annotated bibliography. International Journal of Forecasting, 5:559-583, 1989.
• 4: William W. Cohen , Henry Kautz , David McAllester, Hardening soft information sources, Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, p.255-259, August 20-23, 2000, Boston, Massachusetts, United States [doi>10.1145/347090.347141]
• 5: Weighted least sum of squares regression. In C. Croarkin, P. Tobias, and W. Guthrie, editors, NIST/SEMATECH Engineering Statistics Internet Handbook. http ://www. itl. nist. gov/div898/handbook/pmd/ section1/pmd143, him.
• 6: B. de Finetti. Theory of Probability: A Critical Introductory Treatment, volume 1. Wiley, New York, 1974.
• 7: T. G. Dietterich. Machine learning research: Four current directions. Artificial Intelligence Magazine, 18(4):97-136, 1997.
• 8: U. Fayyad, G. Piatetsky-Shapiro, and P. Smyth. Knowledge discovery and data mining: Towards a unifying framework. In Second International Conference on Knowledge Discovery and Data Mining, 1996.
• 9: Gary William Flake , Steve Lawrence , C. Lee Giles, Efficient identification of Web communities, Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, p.150-160, August 20-23, 2000, Boston, Massachusetts, United States [doi>10.1145/347090.347121]
• 10: R. Forsythe and R. Lundholm. Information aggregation in an experimental market. Econometrica, 58(2):309-347, 1990.
• 11: R. Forsythe, F. Nelson, G. R. Neumann, and J. Wright. Anatomy of an experimental political stock market. American Economic Review, 82(5):1142-1161, 1992.
• 12: R. Forsythe, T. A. Rietz, and T. W. Ross. Wishes, expectations, and actions: A survey on price formation in election stock markets. Journal of Economic Behavior and Organization, 39:83-110, 1999.
• 13: S. French. Group consensus probability distributions: A critical survey. Bayesian Statistics, 2:183-202, 1985.
• 14: J. M. Gandar, W. H. Dare, C. R. Brown, and R. A. Zuber. Informed traders and price variations in the betting market for professional basketball games. Journal of Finance, LIII(1):385-401, 1998.
• 15: C. Genest and J. V. Zidek. Combining probability distributions: A critique and an annotated bibliography. Statistical Science, 1(1):114-148, 1986.
• 16: Richard D. Hackathorn, Web farming for the data warehouse, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1999
• 17: R. Hanson. Decision markets. IEEE Intelligent Systems, 14(3):16-19, 1999.
• 18: R. D. Hanson. Could gambling save science? Encouraging an honest consensus. Social Epistemology, 9(1):3-33, 1995.
• 19: Edward H. Kaplan , Stanley J. Garstka, March Madness and the Office Pool, Management Science, v.47 n.3, p.369-382, March 2001 [doi>10.1287/mnsc.47.3.369.9769]
• 20: Mong Li Lee , Tok Wang Ling , Wai Lup Low, IntelliClean: a knowledge-based intelligent data cleaner, Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, p.290-294, August 20-23, 2000, Boston, Massachusetts, United States [doi>10.1145/347090.347154]
• 21: D. S. McKay, E. K. G. Jr., K. L. Thomas-Keprta, H. Vali, C. S. Romanek, S. J. Clemett, X. D. F. Chillier, C. R. Maechling, and R. N. Zare. Search for past life on Mars: Possible relic biogenic activity in Martian meteorite alh84001. Science, 273:924-930, Aug. 1996.
• 22: D. M. Pennock, S. Lawrence, C. L. Giles, and F. /,. Nielsen. The power of play: Efficiency and forecast accuracy in web market games. Technical Report 2000-168, NEC Research Institute, 2000.
• 23: D. M. Pennock, S. Lawrence, C. L. Giles, and F. A. Nielsen. The real power of artificial markets. Science, 291:987-988, February 9 2001.
• 24: C. R. Plott. Markets as information gathering tools. Southern Economic Journal, 67(1):1-15, 2000.
• 25: C. R. Plott and S. Sunder. Efficiency of experimental security markets with insider information: An application of rational-expectations models. Journal of Political Economy, 90(4):663-98, 1982.
• 26: C. R. Plott and S. Sunder. Rational expectations and the aggregation of diverse information in laboratory security markets. Econometrica, 56(5):1085-1118, 1988.
• 27: C. R. Plott, J. Wit, and W. C. Yang. Parimutuel betting markets as information aggregation devices: experimental results. Technical Report Social Science Working Paper 986, California Institute of Technology, Apr. 1997.
• 28: Foster Provost , Venkateswarlu Kolluri, A Survey of Methods for Scaling Up Inductive Algorithms, Data Mining and Knowledge Discovery, v.3 n.2, p.131-169, June 1999 [doi>10.1023/A:1009876119989]
• 29: Paul Resnick , Hal R. Varian, Recommender systems, Communications of the ACM, v.40 n.3, p.56-58, March 1997 [doi>10.1145/245108.245121]
• 30: R. N. Rosett. Gambling and rationality. Journal of Political Economy, 73(6):595-607, 1965.
• 31: B. J. Sherrick, P. Garcia, and V. Tirupattur. Recovering probabilistic information from options markets: Tests of distributional assumptions. Journal of Futures Markets, 16(5):545-560, 1996.
• 32: W. W. Snyder. Horse racing: testing the efficient markets model. Journal of Finance, 33(4):1109-1118, 1978.
• 33: R. H. Thaler and W. T. Ziemba. Anomalies: Parimutuel betting markets: Racetracks and lotteries. Journal of Economic Perspectives, 2(2):161-174, 1988.
• 34: D. A. Wasserman. More than enough answers to questions nobody asked about F1Pick6, 1996. http ://www. freenet, edmonton, ab. ca/-davidwss/ flp6anal, pdf.
• 35: M. Weitzman. Utility analysis and group behavior: An empirical study. Journal of Political Economy, 73(1):18-26, 1965.
• 36: R. L. Winkler and A. H. Murphy. Good probability assessors. J. Applied Meteorology, 7:751-758, 1968.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.5 On-line Information Services
          Subjects: Web-based services

  J. Computer Applications
  J.4 SOCIAL AND BEHAVIORAL SCIENCES
      Subjects: Economics

  K. Computing Milieux
  K.8 PERSONAL COMPUTING
      K.8.0 General
          Subjects: Games

Keywords:
Collective probabilistic forecasts, Foresight Exchange, Formula One Pick Six Competition, Hollywood Stock Exchange, World Wide Web games, artificial markets, data mining, knowledge discovery