Human-Powered Top-k Lists

@inproceedings{Polychronopoulos2013HumanPoweredTL,
  title={Human-Powered Top-k Lists},
  author={Vassilis Polychronopoulos and Luca de Alfaro and James Davis and Hector Garcia-Molina and Neoklis Polyzotis},
  booktitle={WebDB},
  year={2013}
}
We propose an algorithm that obtains the top-k list of items out of a larger itemset, using human workers (e.g., through crowdsourcing) to perform comparisons among items. An example application is finding the best photographs in a large collection by asking humans to evaluate different photos. Our algorithm has to address several challenges: obtaining worker input has high latency; workers may disagree on their judgments for the same items; some workers may provide wrong input on purpose; and… CONTINUE READING