Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

We investigate the problem of incentivizing an expert to truthfully reveal probabilistic information about a random event. Probabilistic information consists of one or more properties, which are any real-valued functions of the distribution, such as the mean and variance. Not all properties can be elicited truthfully. We provide a simple characterization of elicitable properties, and describe the general form of the associated payment functions that induce truthful revelation. We then consider sets of properties, and observe that all properties can be inferred from sets of elicitable properties. This suggests the concept of elicitation complexity for a property, the size of the smallest set implying the property.

REFERENCES

• 1: BRIER, G. 1950. Verification of Forecasts Expressed In Terms of Probability. Monthly Weather Review 78, 1, 1-3.
• 2: CERVERA, J. AND MUNOZ, J. 1996. Proper scoring rules for fractiles. Bayesian Statistics 5, 513-519.
• 3: GNEITING, T. AND RAFTERY, A. 2007. Strictly proper scoring rules, prediction and estimation. Journal of the American Statistical Association 102, 477, 359-378.
• 4: GOOD, I. 1997. Rational Decisions. Springer-Verlag.
• 5: HENDRICKSON, A. AND BUEHLER, R. 1971. Proper Scores for Probability Forecasters. The Annals of Mathematical Statistics 42, 6, 1916-1921.
• 6: Nicolas S. Lambert , David M. Pennock , Yoav Shoham, Eliciting properties of probability distributions, Proceedings of the 9th ACM conference on Electronic commerce, July 08-12, 2008, Chicago, Il, USA [doi>10.1145/1386790.1386813]
• 7: NELSON, R. AND BESSLER, D. 1989. Subjective Probabilities and Scoring Rules: Experimental Evidence. American Journal of Agricultural Economics 71, 2, 363-369.
• 8: O'CARROLL, F. 1977. Subjective Probabilities and Short-Term Economic Forecasts: An Empirical Investigation. Applied Statistics 26, 3, 269-278.
• 9: SAVAGE, L. 1971. Elicitation of Personal Probabilities and Expectations. Journal of the American Statistical Association 66, 336, 783-801.
• 10: SCHERVISH, M. 1989. A General Method for Comparing Probability Assessors. The Annals of Statistics 17, 4, 1856-1879.
• 11: SPIEGELHALTER, D. 1986. Probabilistic prediction in patient management and clinical trials. Stat Med 5, 5, 421-33.
• 12: WINKLER, R. 1996. Scoring rules and the evaluation of probabilities. TEST 5, 1, 1-60.

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Distribution functions

Additional Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Time series analysis; Probabilistic algorithms (including Monte Carlo)

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.3 Deduction and Theorem Proving
          Subjects: Uncertainty, "fuzzy," and probabilistic reasoning

  J. Computer Applications
  J.1 ADMINISTRATIVE DATA PROCESSING

General Terms:
Algorithms, Design, Theory

Keywords:
elicitation, forecasting, prediction market, scoring rule