Dynamic estimation of worker reliability in crowdsourcing for regression tasks: Making it work

@article{Tarasov2014DynamicEO,
  title={Dynamic estimation of worker reliability in crowdsourcing for regression tasks: Making it work},
  author={Alexey Tarasov and Sarah Jane Delany and Brian Mac Namee},
  journal={Expert Syst. Appl.},
  year={2014},
  volume={41},
  pages={6190-6210}
}
Abstract One of the biggest challenges in crowdsourcing is detecting noisy and incompetent workers. A possible way of handling this problem is to dynamically estimate the reliability of workers as they do work and accept only those workers who are deemed to be reliable to date. Although many approaches to dynamic estimation of rater reliability exist, they are often only appropriate for very specific categories of tasks, for example, only for binary classification. They also can make… CONTINUE READING

Citations

Publications citing this paper.
SHOWING 1-10 OF 15 CITATIONS

Adaptive task assignment in spatial crowdsourcing

VIEW 10 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

A Crowdsourcing-based Medical Image Classification Method

Three recent trends in Paralinguistics on the way to omniscient machine intelligence

VIEW 2 EXCERPTS
CITES BACKGROUND

Big Data, Deep Learning - At the Edge of X-Ray Speaker Analysis

VIEW 2 EXCERPTS
CITES BACKGROUND

Bee Colony Based Worker Reliability Estimation Algorithm in Microtask Crowdsourcing

VIEW 2 EXCERPTS
CITES BACKGROUND