Find it if you can: a game for modeling different types of web search success using interaction data

@article{Ageev2011FindII,
  title={Find it if you can: a game for modeling different types of web search success using interaction data},
  author={Mikhail S. Ageev and Qi Guo and Dmitry Lagun and Eugene Agichtein},
  journal={Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval},
  year={2011}
}
  • Mikhail S. Ageev, Qi Guo, Eugene Agichtein
  • Published 24 July 2011
  • Computer Science
  • Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
A better understanding of strategies and behavior of successful searchers is crucial for improving the experience of all searchers. However, research of search behavior has been struggling with the tension between the relatively small-scale, but controlled lab studies, and the large-scale log-based studies where the searcher intent and many other important factors have to be inferred. We present our solution for performing controlled, yet realistic, scalable, and reproducible studies of… 

Figures and Tables from this paper

Struggling and Success in Web Search
TLDR
This mixed methods study analyzes anonymized search logs from the Microsoft Bing Web search engine to characterize aspects of struggling searches and better explain the relationship between struggling and search success, and develops and utilizes a crowd-sourced labeling methodology.
Different Users, Different Opinions: Predicting Search Satisfaction with Mouse Movement Information
TLDR
Inspired by recent studies in predicting result relevance based on mouse movement patterns (namely motifs), this work proposes to estimate the utilities of search results and the efforts in search sessions with motifs extracted from mouse movement data on search result pages (SERPs).
User Satisfaction Prediction with Mouse Movement Information in Heterogeneous Search Environment
TLDR
Experimental results show that the proposed strategies outperform existing methods and have promising generalization capability for unseen users and queries in both a homogeneous and heterogeneous search environment.
Predicting web search success with fine-grained interaction data
TLDR
It is shown that fine-grained interactions, such as mouse cursor movements and scrolling, provide additional clues for better predicting success of a search session as a whole and a new Fine- Grained Session Behavior (FSB) model is designed to capture these patterns.
User Behavior in Asynchronous Slow Search
TLDR
This work conducts a between-subjects user study involving tasks that required multiple queries to complete, providing a Web search system that gave users the option to additionally issue asynchronous queries for which results improve in relevance over time as users continued working.
Understanding how people interact with web search results that change in real-time using implicit feedback
TLDR
A traditional search interface is compared with one that dynamical-ly re-ranks and recommends search results as the user interacts with it in order to build a picture of how and when users should be offered dynamically identified relevant content.
A Crowd-Powered Task Generation Method for Study of Struggling Search
TLDR
A crowd-powered task generation method is proposed and a platform to efficiently generate struggling search tasks on basis of online wikis such as Wikipedia is developed, showing that the generated tasks are qualified to emulate struggling search behaviors consisting of “repeated similar queries” and “quick-back clicks”.
Personalized models of search satisfaction
TLDR
This paper verifies that searcher behavior when satisfied and dissatisfied is indeed different among individual searchers along a number of dimensions, and introduces and evaluates learned models of satisfaction for individual searcher and searcher cohorts.
Exploration or Fact-Finding: Inferring User's Search Activity Just in Time
TLDR
The goal of this research is to build models of users' web information-seeking behavior in order to differentiate between search activities caused by exploratory search tasks and consecutive fact-finding search tasks while users are still searching.
Struggling or exploring?: disambiguating long search sessions
TLDR
This paper analyzes struggling and exploring behavior in Web search using log data from a commercial search engine, and builds classifiers that can accurately distinguish between exploring and struggling sessions using behavioral and topical features.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 38 REFERENCES
How does search behavior change as search becomes more difficult?
TLDR
The behavioral signals that suggest that a user is having trouble in a search task are studied to gain a preliminary understanding on how users' behavior changes when they struggle finding the information they're looking for.
Page hunt: improving search engines using human computation games
TLDR
Page Hunt is described, a single-player game that elicits data from players that can be used to improve search and has several applications including providing metadata for pages, providing query alterations for use in query refinement, and identifying ranking issues.
Beyond DCG: user behavior as a predictor of a successful search
TLDR
This work shows empirically that user behavior alone can give an accurate picture of the success of the user's web search goals, without considering the relevance of the documents displayed.
Investigating the querying and browsing behavior of advanced search engine users
TLDR
The results show that there are marked differences in the queries, result clicks, post-query browsing, and search success of users the authors classify as advanced (based on their use of query operators), relative to those classified as non-advanced.
User adaptation: good results from poor systems
TLDR
It is found that searchers using the authors' degraded systems are as successful as those using the standard system, but that, in achieving this success, they alter their behavior in ways that could be measured, in real time, by a suitably instrumented system.
Predicting short-term interests using activity-based search context
TLDR
This study developed and evaluated user interest models for the current query, its context (from pre-query session activity), and their combination, which is referred to as intent, and investigates optimally combining the query and its context by learning a model that predicts the context weight for each query.
Evaluating web search using task completion time
TLDR
An alternative experimental design is proposed which is proposed to be a substantial improvement over the current design in terms of variance reduction and efficiency and fit a model with the user's total search time as the response to compare two different search algorithms.
Predicting searcher frustration
TLDR
By modeling searcher frustration, search engines can predict the current state of user frustration and decide when to intervene with alternative search strategies to prevent the user from becoming more frustrated, giving up, or switching to another search engine.
Do user preferences and evaluation measures line up?
TLDR
It is established that preferences and evaluation measures correlate: systems measured as better on a test collection are preferred by users and the nDCG measure is found to correlate best with user preferences compared to a selection of other well known measures.
Models of Searching and Browsing: Languages, Studies, and Application
TLDR
The formulation, construction, and evaluation of predictive models of human information seeking from a large dataset of Web search activities are described, including an exploration of the properties of users, queries, and search sessions that are most predictive of future behavior.
...
1
2
3
4
...