Head First: Living Labs for Ad-hoc Search Evaluation

  title={Head First: Living Labs for Ad-hoc Search Evaluation},
  author={Krisztian Balog and Liadh Kelly and Anne Schuth},
The information retrieval (IR) community strives to make evaluation more centered on real users and their needs. The living labs evaluation paradigm, i.e., observing users in their natural task environments, offers great promise in this regard. Yet, progress in an academic setting has been limited. This paper presents the first living labs for the IR community benchmarking campaign initiative, taking as test two use-cases: local domain search on a university website and product search on an e… CONTINUE READING
Highly Cited
This paper has 32 citations. REVIEW CITATIONS