• Corpus ID: 14212821

Living Labs for UMAP Evaluation

@inproceedings{Kelly2016LivingLF,
  title={Living Labs for UMAP Evaluation},
  author={Liadh Kelly},
  booktitle={User Modeling, Adaptation, and Personalization},
  year={2016}
}

• L. Kelly
• Published in
  User Modeling, Adaptation…
  2016
• Computer Science

Generating shared task initiatives in the user-modelling, adaptation and personalization (UMAP) space is difficult, especially given individual difference, privacy concerns and the interactive nature of the space. We put forward that the living labs evaluation paradigm, i.e., observing users in their natural task environments, has potential to overcome these difficulties and to allow for comparative evaluation in the UMAP space of research. In particular, the emerging approach to living labs for shared evaluation in other research disciplines, has potential to be adapted to allow for shared evaluation tasks in the UMAP community. Coupled with this, there is the potential to create living labs in different ways for shared UMAP evaluation. In this paper we overview the use to-date of living labs approaches for shared evaluation and set directions for its application in UMAP shared challenges.