The labor economics of paid crowdsourcing

@article{Horton2010TheLE,
  title={The labor economics of paid crowdsourcing},
  author={John Joseph Horton and Lydia B. Chilton},
  journal={Labor: Human Capital eJournal},
  year={2010}
}
We present a model of workers supplying labor to paid crowdsourcing projects. We also introduce a novel method for estimating a worker's reservation wage - the key parameter in our labor supply model. We tested our model by presenting experimental subjects with real-effort work scenarios that varied in the offered payment and difficulty. As predicted, subjects worked less when the pay was lower. However, they did not work less when the task was more time-consuming. Interestingly, at least some… 

Labor Allocation in Paid Crowdsourcing: Experimental Evidence on Positioning, Nudges and Prices

TLDR
The evidence suggests that user interface and cognitive biases play an important role in online labor markets and that salience can be used by employers as a kind of "incentive multiplier".

Efficient Crowdsourcing of Unknown Experts using Multi-Armed Bandits

TLDR
A novel multi-armed bandit (MAB) model, the bounded MAB, is introduced that naturally captures the problem of expert crowdsourcing and an algorithm to solve it efficiently, called bounded e-first, which uses the first eB of its total budget B to derive estimates of the workers' quality characteristics (exploration), while the remaining eB is used to maximise the total utility based on those estimates (exploitation).

Quality-Based Pricing for Crowdsourced Workers

TLDR
This work presents a comprehensive scheme for managing quality of crowdsourcing processes, and describes a pricing scheme that pays workers based on their expected quality, reservation wage, and expected lifetime, and accommodates measurement uncertainties and allows the workers to receive a fair wage, even in the presence of temporary incorrect estimations of quality.

Truthful incentives in crowdsourcing tasks using regret minimization mechanisms

TLDR
This paper designs a novel, no-regret posted price mechanism, BP-UCB, for budgeted procurement in stochastic online settings and proves strong theoretical guarantees about the mechanism, and extensively evaluate it in simulations as well as on real data from the Mechanical Turk platform.

Workers' Task Choice in Crowdsourcing and Human Computation Markets

TLDR
This process is outlined and a structural model showing the criteria that workers use to choose tasks is proposed, based on the person-job fit theory, which includes the measures demands-abilities fit and needs-supplies fit, in order to explain the work intention.

Pay It Backward: Per-Task Payments on Crowdsourcing Platforms Reduce Productivity

Paid crowdsourcing marketplaces have gained popularity by using piecework, or payment for each microtask, to incentivize workers. This norm has remained relatively unchallenged. In this paper, we

Algorithmic Wage Negotiations: Applications to Paid Crowdsourcing

This paper highlights the problem that buyers and sellers encounter in arriving at prices in a distributed labor market. We argue that automated negotiation might partly remedy this problem and

E ffi cient Crowdsourcing of Unknown Experts using Multi-Armed Bandits Long

TLDR
A novel multi–armed bandit (MAB) model, the bounded MAB, that naturally captures the problem of expert crowdsourcing and an algorithm to solve it efficiently, called bounded ε–first, which uses the first εB of its total budget B to derive estimates of the workers’ quality characteristics (exploration), while the remaining (1 − ε) B is used to maximise the total utility based on those estimates (exploitation).

Matching Workers Expertise with Tasks : Incentives in Heterogeneous Crowdsourcing Markets ∗

TLDR
This paper designs a mechanism TM-UNIFORM which does the allocation of tasks to workers, while ensuring budget feasibility, incentive-compatibility and achieves near-optimal utility, and carries out extensive experimentation using simulations on a realistic case study of Wikipedia translation project using Mechanical Turk.

Adaptive Contract Design for Crowdsourcing

TLDR
A multi-round version of the well-known principal-agent model, whereby in each round a worker makes a strategic choice of the effort level which is not directly observable by the requester, is considered, which significantly generalizes the budget-free online task pricing problems studied in prior work.
...

References

SHOWING 1-10 OF 23 REFERENCES

Financial incentives and the "performance of crowds"

TLDR
It is found that increased financial incentives increase the quantity, but not the quality, of work performed by participants, where the difference appears to be due to an "anchoring" effect.

Do Workers Work More If Wages are High? Evidence from a Randomized Field Experiment

Most previous studies on intertemporal labor supply found very small or insignificant substitution effects. It is not clear, however, whether these results are due to institutional constraints on

Labor Supply of New York City Cabdrivers: One Day at a Time

Life-cycle models of labor supply predict a positive relationship between hours supplied and transitory changes in wages. We tested this prediction using three samples of wages and hours of New York

The Economist as Engineer: Game Theory, Experimentation, and Computation as Tools for Design Economics

Economists have lately been called upon not only to analyze markets, but to design them. Market design involves a responsibility for detail, a need to deal with all of a market's complications, not

Designing incentives for online question and answer forums

TLDR
It is demonstrated that an approval-voting scoring rule and a proportional-share scoring rule can enable the most efficient equilibrium with complements information, under certain conditions, by providing incentives for early responders as well as the user who submits the final answer.

Reference-Dependent Preferences and Labor Supply: The Case of New York City Taxi Drivers

I develop a model of daily labor supply where preferences are dependent on a reference daily income level, and I apply this model to data on the labor supply of New York City taxi drivers. I find

Crowdsourcing: Why the Power of the Crowd is Driving the Future of Business

TLDR
A new concept has emerged that is changing the way the business world operates and many research and development (R&D) problems in a particular area are being solved.

Get another label? improving data quality and data mining using multiple, noisy labelers

TLDR
The results show clearly that when labeling is not perfect, selective acquisition of multiple labels is a strategy that data miners should have in their repertoire; for certain label-quality/cost regimes, the benefit is substantial.

An Essay on the Nature and Significance of Economic Science.

Lionel Robbins’s An Essay on the Nature and Significance of Economic Science (1932) made at least three important contributions: (1) constructing a more modern, focused, and general definition of

Crowdsourcing, attention and productivity

We show through an analysis of a massive data set from YouTube that the productivity exhibited in crowdsourcing exhibits a strong positive dependence on attention, measured by the number of