Evaluating expertise recommendations

@article{McDonald2001EvaluatingER,
  title={Evaluating expertise recommendations},
  author={David W. McDonald},
  journal={Proceedings of the 2001 ACM International Conference on Supporting Group Work},
  year={2001}
}
  • David W. McDonald
  • Published 30 September 2001
  • Computer Science
  • Proceedings of the 2001 ACM International Conference on Supporting Group Work
Finding a person who has the expertise to solve a specific problem is an important application of recommender systems to a difficult organizational problem. Prior systems have made attempts to implement solutions to this problem, but few systems have undergone systematic user evaluation. This work describes a systematic evaluation of the Expertise Recommender (ER), a system that recommends people who are likely to have expertise in a specific problem. ER and the organizational context for which… 
View on ACM
cdlib.org
77 Citations
Highly Influential Citations
6
Background Citations
36
Methods Citations
17
Results Citations
4

Figures from this paper

77 Citations

Performance Evaluation of a Bi-directional Feedback Approach for Expertise Recommendation in Academic Research

A bidirectional feedback approach to expertise recommendation in academic research that incorporates user classification is proposed and indicated that the proposed system outperformed the existing system by 2.5%.

Expertise Locating : a Challenge for Recommender Systems

INTRODUCTION People are faced with difficult problems that they cannot solve alone on a day-to-day basis. In these situations the right people are those who have the expertise to answer a specific…

Semantic inference of user's reputation and expertise to improve collaborative recommendations

Expert Finding Systems: A Systematic Review

A Systematic Literature Review on the state-of-the-art expert finding systems and expertise seeking studies published between 2010 and 2019 identifies five gaps in expert finding Systems and identifies the differences between expertise retrieval and seeking.

Collaborative Filtering Recommender Systems

A wide variety of the choices available and their implications are discussed, aiming to provide both practicioners and researchers with an introduction to the important issues underlying recommenders and current best practices for addressing these issues.

Finding and validating expertise

This paper designed a triangulated approach which combines automated expert profile creation and maintenance, validation by the expert, and feedback on the expert and the system by the searching party, and implemented and evaluated a prototype Expert Recommender Systems.

Research Paper Recommender System Evaluation Using Coverage

A range of evaluation metrics and measures as well as some approaches used for evaluating recommendation systems are reviewed, showing that large differences in recommendation accuracy across frameworks and strategies are shown.

An Integrated Approach to Research Paper and Expertise Recommendation in Academic Research

An integrated recommendation system to harmonize research papers and expertise recommendations in the academic domain for enhanced research collaboration was developed and showed that the harmonization of research paper and expertise recommendation approaches provides a holistic and enhanced approach towards research paperand expertise recommendation.

Fast algorithms to evaluate collaborative filtering recommender systems

Evaluating collaborative filtering recommender systems

The key decisions in evaluating collaborative filtering recommender systems are reviewed: the user tasks being evaluated, the types of analysis and datasets being used, the ways in which prediction quality is measured, the evaluation of prediction attributes other than quality, and the user-based evaluation of the system as a whole.
...

References

SHOWING 1-10 OF 40 REFERENCES

Expertise recommender: a flexible recommendation system and architecture

The architecture and implementation of the expertise recommendation system details the work necessary to fit expertise recommendation to a work setting, and begins to tease apart the technical aspects of providing good recommendations from social and collaborative concerns.

Making Recommender Systems Work for Organizations

The incentive issues that have been identified are described and how the research has attempted to alleviate them and results from an internal yearlong trial of the recommender tool, the Knowledge Pump are reported and analyzed.

Beyond Algorithms: An HCI Perspective on Recommender Systems

From a user’s perspective, an effective recommender system inspires trust in the system; has system logic that is at least somewhat transparent; points users towards new, not-yet-experienced items; provides details about recommended items, including pictures and community ratings; and finally, provides ways to refine recommendations by including or excluding particular genres.

Interfaces for Eliciting New User Preferences in Recommender Systems

It is found that the two pure interfaces both produced accurate user models, but that directly asking users for items to rate increases user loyalty in the system.

The role of transparency in recommender systems

Preliminary results indicate that users like and feel more confident about recommendations that they perceive as transparent, and the role of transprency (user understanding of why a particular recommendation was made) in Recommender Systems is examined.

User Choices: A new Yardstick for the Evaluation of Ranking Algorithms for Interactive Query Expansion

Awareness of Organizational Expertise

The architecture and use of collaboration and expert finder systems to enhance organizational awareness, provides some principles of expertise, and concludes with an outline of future research directions.

Just talk to me: a field study of expertise location

A five-month field study of a medium-sized software firm found the participants use complex, iterative behaviors to minimize the number of possible expertise sources, while at the same time, provide a high possibility of garnering the necessary expertise.

A survey on the use of relevance feedback for information access systems

This paper surveys relevance feedback techniques and studies both automatic techniques, in which the system modifies the user's query, and interactive techniques, which the user has control over query modification.

Referral Web: combining social networks and collaborative filtering

Part of the success of social networks can be attributed to the “six degrees of separation’’ phenomena that means the distance between any two individuals in terms of direct personal relationships is…