Evaluation of retrieval effectiveness with incomplete relevance data: Theoretical and experimental comparison of three measures

@article{Ahlgren2008EvaluationOR,
  title={Evaluation of retrieval effectiveness with incomplete relevance data: Theoretical and experimental comparison of three measures},
  author={Per Ahlgren and Leif Gr{\"o}nqvist},
  journal={Inf. Process. Manage.},
  year={2008},
  volume={44},
  pages={212-225}
}
This paper investigates two relatively new measures of retrieval effectiveness in relation to the problem of incomplete relevance data. The measures, Bpref and RankEff, which do not take into account documents that have not been relevance judged, are compared theoretically and experimentally. The experimental comparisons involve a third measure, the wellknown mean uninterpolated average precision. The results indicate that RankEff is the most stable of the three measures when the amount of… CONTINUE READING

From This Paper

Figures, tables, and topics from this paper.

References

Publications referenced by this paper.
Showing 1-10 of 15 references

The Philosophy of Information Retrieval Evaluation

CLEF • 2001
View 4 Excerpts
Highly Influenced

Evaluating evaluation measure stability

View 5 Excerpts
Highly Influenced

Measuring retrieval effectiveness with incomplete relevance data. In InSCit2006, Current research in information sciences and technologies: Multidisciplinary approaches to global information

P. Ahlgren, L. Grönqvist
2006
View 1 Excerpt

Evaluating latent semantic vector models with synonym tests and document retrieval

D. Hull
2005

The effect of topic set size on retrieval experiment error

J. Zobel
Proceedings of the 24 th annual international ACM SIGIR conference on research and development in information retrieval • 2002

Similar Papers

Loading similar papers…