Tolerance of Effectiveness Measures to Relevance Judging Errors


Crowdsourcing relevance judgments for test collection construction is attractive because the practice has the possibility of being more affordable than hiring high quality assessors. A problem faced by all crowdsourced judgments – even judgments formed from the consensus of multiple workers – is that there will be differences in the judgments compared to… (More)
DOI: 10.1007/978-3-319-06028-6_13


8 Figures and Tables

Slides referencing similar topics