Modeling Temporal Crowd Work Quality with Limited Supervision

  title={Modeling Temporal Crowd Work Quality with Limited Supervision},
  author={Hyun Joon Jung and Matthew Lease},
While recent work has shown that a worker’s performance can be more accurately modeled by temporal correlation in task performance, a fundamental challenge remains in the need for expert gold labels to evaluate a worker’s performance. To solve this problem, we explore two methods of utilizing limited gold labels, initial training and periodic updating. Furthermore, we present a novel way of learning a prediction model in the absence of gold labels with uncertaintyaware learning and soft-label… CONTINUE READING