Tolerance of Effectiveness Measures to Relevance Judging Errors

Abstract

Crowdsourcing relevance judgments for test collection construction is attractive because the practice has the possibility of being more affordable than hiring high quality assessors. A problem faced by all crowdsourced judgments – even judgments formed from the consensus of multiple workers – is that there will be differences in the judgments compared to… (More)
DOI: 10.1007/978-3-319-06028-6_13

Topics

8 Figures and Tables

Slides referencing similar topics