Crowdsourcing Music Similarity Judgments using Mechanical Turk

@inproceedings{Lee2010CrowdsourcingMS,
  title={Crowdsourcing Music Similarity Judgments using Mechanical Turk},
  author={Jin Ha Lee},
  booktitle={ISMIR},
  year={2010}
}
Collecting human judgments for music similarity evaluation has always been a difficult and time consuming task. This paper explores the viability of Amazon Mechanical Turk (MTurk) for collecting human judgments for audio music similarity evaluation tasks. We compared the similarity judgments collected from Evalutron6000 (E6K) and MTurk using the Music Information Retrieval Evaluation eXchange 2009 Audio Music Similarity and Retrieval task dataset. Our data show that the results are highly… CONTINUE READING
Highly Cited
This paper has 53 citations. REVIEW CITATIONS

Citations

Publications citing this paper.

54 Citations

051015'12'14'16'18
Citations per Year
Semantic Scholar estimates that this publication has 54 citations based on the available data.

See our FAQ for additional information.

References

Publications referenced by this paper.
Showing 1-10 of 16 references

Demographics of Mechanical Turk

  • P. G. Ipeirotis
  • CeDER Working Papers, Retrieved from http://hdl…
  • 2010
1 Excerpt

Ipeirotis : “ Demographics of Mechanical Turk

  • M. C. Jones, J. S. Downie, A. F. Ehmann
  • 2010

Geeks Spot Fossett?

  • D. Axe
  • Wired, Retrieved from http://www.wired.com…
  • 2007
1 Excerpt

McKinney : “ Assessment of Perceptual Music Similarity

  • C. Stevens
  • Proceedings of the 8 th ISMIR
  • 2007

The Effect of Implied Harmony, Contour and Musical Expertise on Judgments of Similarity of Familiar Melodies

  • E. Schubert, C. Stevens
  • Journal of New Music Research, Vol. 35, No. 2, pp…
  • 2006
3 Excerpts

Similar Papers

Loading similar papers…