Utility data annotation with Amazon Mechanical Turk

@article{Sorokin2008UtilityDA,
  title={Utility data annotation with Amazon Mechanical Turk},
  author={Alexander Sorokin and David A. Forsyth},
  journal={2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops},
  year={2008},
  pages={1-8}
}
We show how to outsource data annotation to Amazon Mechanical Turk. Doing so has produced annotations in quite large numbers relatively cheaply. The quality is good, and can be checked and controlled. Annotations are produced quickly. We describe results for several different annotation problems. We describe some strategies for determining when the task is well specified and properly priced. 
Highly Influential
This paper has highly influenced a number of papers. REVIEW HIGHLY INFLUENTIAL CITATIONS
Highly Cited
This paper has 635 citations. REVIEW CITATIONS

Topics

Statistics

05010020082009201020112012201320142015201620172018
Citations per Year

636 Citations

Semantic Scholar estimates that this publication has 636 citations based on the available data.

See our FAQ for additional information.