Learn More
This paper presents an experimental study of users assessing the quality of Google web search results. In particular we look at how users' satisfaction correlates with the effectiveness of Google as quantified by IR measures such as precision and the suite of Cumulative Gain measures (CG, DCG, NDCG). Results indicate strong correlation between users'(More)
Test collections are extensively used in the evaluation of information retrieval systems. Crucial to their use is the degree to which results from them predict user effectiveness. At first, past studies did not substantiate a relationship between system and user effectiveness; more recently, however, correlations have begun to emerge. The results of this(More)
This paper investigates factors influencing user satisfaction in information retrieval. It is evident from this study that user satisfaction is a subjective variable which can be influenced by several factors such as system effectiveness, user effectiveness, user effort and user characteristics and expectations. Therefore, information retrieval evaluators(More)
This paper investigates the agreement of relevance assessments between official TREC judgments and those generated from an interactive IR experiment. Results show that 63% of documents judged relevant by our users matched official TREC judgments. Several factors contributed to differences in the agreements: the number of retrieved relevant documents; the(More)
This paper investigates the influence of user characteristics (e.g. search experience and cognitive skills) on user effectiveness. A user study was conducted to investigate this effect, 56 participants completed searches for 56 topics using the TREC test collection. Results indicated that participants with search experience and high cognitive skills were(More)
In this paper we describe our submission for iCLEF2006: an interface that allows users to search FLICKR in Arabic for images with captions in a range of languages. We report and discuss the results gained from a user experiment in accordance with directives given by iCLEF, including an analysis of the success of search tasks. To enable the searching of(More)
This paper presents results from an initial user study exploring the relationship between system effectiveness as quantified by traditional measures such as precision and recall, and users' effectiveness and satisfaction of the results. The tasks involve finding images for recall-based tasks. It was concluded that no direct relationship between system(More)
This research explores the relationship between Information Retrieval (IR) systems' effectiveness and users' performance (accuracy and speed) and their satisfaction with the retrieved results (precision of the results, completeness of the results and overall system success). Previous studies have concluded that improvements in IR systems based on increase(More)
This paper reports on experiments that attempt to characterize the relationship between users and their knowledge of the search topic in a Question Answering (QA) system. It also investigates user search behavior with respect to the length of answers presented by a QA system. Two lengths of answers were compared; snippets (one to two sentences of text) and(More)
  • 1