Evaluation of Algorithms Using Games: The Case of Music Tagging

@inproceedings{Law2009EvaluationOA,
  title={Evaluation of Algorithms Using Games: The Case of Music Tagging},
  author={Edith Law and Kris West and Michael I. Mandel and Mert Bay and J. Stephen Downie},
  booktitle={ISMIR},
  year={2009}
}
Search by keyword is an extremely popular method for retrieving music. To support this, novel algorithms that automatically tag music are being developed. The conventional way to evaluate audio tagging algorithms is to compute measures of agreement between the output and the ground truth set. In this work, we introduce a new method for evaluating audio tagging algorithms on a large scale by collecting set-level judgments from players of a human computation game called TagATune. We present the… CONTINUE READING
Highly Cited
This paper has 88 citations. REVIEW CITATIONS

Citations

Publications citing this paper.
Showing 1-10 of 64 extracted citations

Event Localization in Music Auto-tagging

ACM Multimedia • 2016
View 17 Excerpts
Highly Influenced

88 Citations

0102030'10'12'14'16'18
Citations per Year
Semantic Scholar estimates that this publication has 88 citations based on the available data.

See our FAQ for additional information.

References

Publications referenced by this paper.
Showing 1-8 of 8 references

Information Retrieval

Encyclopedia of GIS • 1979
View 6 Excerpts
Highly Influenced

The problem of accuracy as an evaluation criterion

Edith Law
ICML Workshop on Evaluation Methods in Machine Learning, • 2008
View 1 Excerpt

Ten experiments on the modelling of polyphonic timbre

Jean-Julien Aucouturier
PhD thesis, University of Paris 6, • 2006
View 2 Excerpts

Similar Papers

Loading similar papers…