Financial incentives and the "performance of crowds"

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback
Take a look at the new version of this page: [ beta version ]. Tell us what you think.

Financial incentives and the "performance of crowds"

Full text

Pdf (618 KB)

Source

International Conference on Knowledge Discovery and Data Mining archive
Proceedings of the ACM SIGKDD Workshop on Human Computation table of contents

Paris, France

SESSION: Labeling cost and efficiency table of contents

Pages: 77-85

Year of Publication: 2009

ISBN:978-1-60558-672-4

Authors

Winter Mason	Yahoo! Research, New York, NY
Duncan J. Watts	Yahoo! Research, New York, NY

Sponsors

Microsoft Research : Microsoft Research
: Carnegie Mellon

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 32, Downloads (12 Months): 109, Citation Count: 0

Additional Information:

abstract references index terms collaborative colleagues

Tools and Actions:	Request Permissions Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/1600150.1600175 What is a DOI?

ABSTRACT

The relationship between financial incentives and performance, long of interest to social scientists, has gained new relevance with the advent of web-based "crowd-sourcing" models of production. Here we investigate the effect of compensation on performance in the context of two experiments, conducted on Amazon's Mechanical Turk (AMT). We find that increased financial incentives increase the quantity, but not the quality, of work performed by participants, where the difference appears to be due to an "anchoring" effect: workers who were paid more also perceived the value of their work to be greater, and thus were no more motivated than workers paid less. In contrast with compensation levels, we find the details of the compensation scheme do matter---specifically, a "quota" system results in better work for less pay than an equivalent "piece rate" system. Although counterintuitive, these findings are consistent with previous laboratory studies, and may have real-world analogs as well.

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	Yochai Benkler, The Wealth of Networks: How Social Production Transforms Markets and Freedom, Yale University Press, New Haven, CT, 2006
	2	Malone, T. W., Laubacher, R. and Dellarocas, C. Harnessing Crowds: Mapping the Genome of Collective Intelligence. MIT, City, 2009.
	3	Luis von Ahn, Games with a Purpose, Computer, v.39 n.6, p.92-94, June 2006 [doi>10.1109/MC.2006.196]
	4	Jeff Howe, Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business, Crown Publishing Group, New York, NY, 2008
	5	Kleeman, F., Voss, G. G. and Rieder, K. Un(der)paid Innovators: The Commercial Utilization of Consumer Work through Crowdsourcing. Science, Technology&Innovation Studies, 4, 1 2008), 5--26.
	6	Oded Nov , Mor Naaman , Chen Ye, What drives content tagging: the case of photos on Flickr, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357225]
	7	Prendergast, C. The provision of incentives in firms. J. Econ. Lit., 37, 1 1999), 7--63.
	8	Lazear, E. P. Performance pay and productivity. American Economic Review, 90, 5 (Dec 2000), 1346--1361.
	9	Murphy, K. J. Executive Compensation. SSRN, 1998.
	10	Hall, B. and Liebman, J. B. Are CEOs Really Paid Like Bureaucrats? SSRN, 1997.
	11	Gneezy, U. and Rustichini, A. Pay enough or don't pay at all. Q. J. Econ., 115, 3 2000), 791--810.
	12	Heyman, J. and Ariely, D. Effort for Payment: A Tale of Two Markets. Psychological Science, 15, 11 2004), 787--793.
	13	Camerer, C., Babcock, L., Loewenstein, G. and Thaler, R. Labor supply of New York City cabdrivers: One day at a time. Q. J. Econ., 112, 2 1997), 407--441.
	14	Ariely, D., Gneezy, U., Loewenstein, G. and Mazar, N. Large Stakes and Big Mistakes. Review of Economic StudiesForthcoming, 2008).
	15	Holmstrom, B. and Milgrom, P. Multitask Principal-Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design. Journal of Law, Economics,&Organization, 71991), 24--52.
	16	Bandiera, O., Barankay, I. and Rasul, I. Social Preferences and the Response to Incentives: Evidence from Personnel Data. Q. J. Econ.*, 120, 3 2005), 917--962.
	17	Kohn, A. Why Incentive Plans Cannot Work. Harvard Business ReviewSeptember-October, 1993), 54--63.
	18	Herzberg, F. One More Time: How do You Motivate Employees? Harvard Business ReviewSeptember-October, 1987), 5--16.
	19	Tversky, A. and Kahneman, D. Judgment under Uncertainty: Heuristics and Biases. Science, 185, 4157 1974), 1124--1131.
	20	Chapman, G. B. and Johnson, E. J. The limits of anchoring. Journal of Behavioral Decision Making, 7, 4 1994), 223--242.
	21	Ariely, D., Loewenstein, G. and Prelec, D. Coherent Arbitrariness: Stable Demand Curves Without Stable Preferences. Q. J. Econ., 118, 1 2003), 73--105.
	22	Aniket Kittur , Ed H. Chi , Bongwon Suh, Crowdsourcing user studies with Mechanical Turk, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357127]
	23	Gelman, A. and Hill, J. Data Analysis Using Regression and Multilevel/hierarchical Models. Cambridge University Press, Cambridge, UK, 2006.
	24	Jenkins, G. D., Mitra, A., Gupta, N. and Shaw, J. D. Are financial incentives related to performance? A meta-analytic review of empirical research. Journal of Applied Psychology, 83, 5 1998), 777--787.
	25	Bonner, S. E., Hastie, R., Sprinkle, G. B. and Young, S. M. A review of the effects of financial incentives on performance in laboratory tasks: Implications for management accounting. Journal of Management Accounting Research, 122000), 19--64.
	26	Conyon, M. J. and Murphy, K. J. The Prince and the Pauper? CEO Pay in the United States and United Kingdom. The Economic Journal, 110, 467 2000), F640--F671.
	27	Blau, F. D. and Kahn, L. M. Gender Differences in Pay. The Journal of Economic Perspectives, 14, 4 2000), 75--99.
	28	Choi, S. J., Gulati, G. M. and Posner, E. A. Are Judges Overpaid? A Skeptical Response to the Judicial Salary Debate. The Journal of Legal Analysis, 1, 1 2009).
	29	Schwarz, N. and Strack, F. Context Effects in Attitude Surveys: Applying Cognitive Theory to Social Research. European Review of Social Psychology(Jan 1 1991).
	30	Herr, P. M., Sherman, S. J. and Fazio, R. H. On the consequences of priming: assimilation and contrast effects. Journal of experimental social psychology(Print)(Jan 1 1983).
	31	Bavelas, A. Communication Patterns in Task-Oriented Groups. The Journal of the Acoustical Society of America(Jan 1 1950).
	32	Nosofsky, R. M. Attention, similarity, and the identificationcategorization relationship. Journal of Experimental Psychology: General(Jan 1 1986).
	33	Kraut, R., Olson, J., Banaji, M., Bruckman, A., Cohen, J. and Couper, M. Psychological Research Online: Report of Board of Scientific Affairs' Advisory Group on the Conduct of Research on the Internet. Am. Psychol., 59, 2 2004), 105--117.