2014 FromLabortoTraderOpinionElicita

• (Cao et al., 2014) ⇒ Caleb Chen Cao, Lei Chen, and Hosagrahar Visvesvaraya Jagadish. (2014). “From Labor to Trader: Opinion Elicitation via Online Crowds As a Market.” In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2014) Journal. ISBN:978-1-4503-2956-9 doi:10.1145/2623330.2623717

Subject Headings:

Cited By

• http://scholar.google.com/scholar?q=%222014%22+From+Labor+to+Trader%3A+Opinion+Elicitation+via+Online+Crowds+As+a+Market
• http://dl.acm.org/citation.cfm?id=2623330.2623717&preflayout=flat#citedby

Quotes

Author Keywords

• Crowdsourcing; data mining; human computation; human information processing; market; social media

Abstract

We often care about people's degrees of belief about certain events: e.g. causality between an action and the outcomes, odds distribution among the outcome of a horse race and so on. It is well recognized that the best form to elicit opinion from human is probability distribution instead of simple voting, because the form of distribution retains the delicate information that an opinion expresses. In the past, opinion elicitation has relied on experts, who are expensive and not always available. More recently, crowdsourcing has gained prominence as an inexpensive way to get a great deal of human input. However, traditional crowdsourcing has primarily focused on issuing very simple (e.g. binary decision) tasks to the crowd. In this paper, we study how to use crowds for Opinion Elicitation. There are three major challenges to eliciting opinion information in the form of probability distributions: how to measure the quality of distribution; how to aggregate the distributions; and, how to strategically implement such a system.

To address these challenges, we design and implement COPE Crowd-powered OPinion Elicitation market. COPE models crowdsourced work as a trading market, where the “workers " behave like “traders " to maximize their profit by presenting their opinion. Among the innovative features in this system, we design COPE updating to combine the multiple elicited distributions following a Bayesian scheme. Also to provide more flexibility while running COPE, we propose a series of efficient algorithms and a slope based strategy to manage the ending condition of COPE. We then demonstrate the implementation of COPE and report experimental results running on real commercial platform to demonstrate the practical value of this system.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2014 FromLabortoTraderOpinionElicita	Lei Chen Caleb Chen Cao Hosagrahar Visvesvaraya Jagadish			From Labor to Trader: Opinion Elicitation via Online Crowds As a Market				10.1145/2623330.2623717		2014