• Corpus ID: 11409638

Une mesure d'expertise pour le crowdsourcing

@inproceedings{Ouni2017UneMD,
  title={Une mesure d'expertise pour le crowdsourcing},
  author={Hosna Ouni and Arnaud Martin and L{\ae}titia Gros and Mouloud Kharoune and Zolt{\'a}n Mikl{\'o}s},
  booktitle={EGC},
  year={2017}
}
Crowdsourcing, a major economic issue, is the fact that the firm outsources internal task to the crowd. It is a form of digital subcontracting for the general public. The evaluation of the participants work quality is a major issue in crowdsourcing. Indeed, contributions must be controlled to ensure the effectiveness and relevance of the campaign. We are particularly interested in small, fast and not automatable tasks. Several methods have been proposed to solve this problem, but they are… 
3 Citations

Figures from this paper

Contributors profile modelization in crowdsourcing platforms
TLDR
A new method of user expertise modelization in the crowdsourcing platforms based on the theory of belief functions is introduced in order to identify serious and qualificated users.
Modélisation du profil des contributeurs dans les plateformes de crowdsourcing
Le crowdsourcing consiste a l'externalisation de tâches a une foule de contributeurs remuneres pour les effectuer. La foule, generalement tres diversifiee, peut inclure des contributeurs
Explicabilité : vers des dispositifs numériques interagissant en intelligence avec l’utilisateur
Résumé. Cet article de positionnement postule que pour qu’un utilisateur et un dispositif technique numérique soient en intelligence , il est nécessaire de penser l’explicabilité. Nous proposons une

References

SHOWING 1-10 OF 22 REFERENCES
Characterization of Experts in Crowdsourcing Platforms
TLDR
This work addresses the problem of identifying experts among participants, that is, workers, who tend to answer the questions correctly, and derives a measure that characterizes the expertise level of each participant, based on precise and exactitude degrees that represent two parts of the expertiselevel.
Ensuring quality in crowdsourced search relevance evaluation: The effects of training question distribution
TLDR
It is concluded that in a relevance categorization task, a uniform distribution of labels across training data labels produces optimal peaks in 1) individual worker precision and 2) majority voting aggregate result accuracy.
Caractérisation d'experts dans les plate-formes de crowdsourcing Characterization of experts in crowdsourcing platforms
Le " Crowdsourcing " represente aujourd'hui une methode de travail tr es populaire qui a une grande importance. Cette methode consiste a recourir aux ressources de la foule pour accomplir des tâches
Learning From Crowds
TLDR
A probabilistic approach for supervised learning when the authors have multiple annotators providing (possibly noisy) labels but no absolute gold standard, and experimental results indicate that the proposed method is superior to the commonly used majority voting baseline.
Annotation models for crowdsourced ordinal data
TLDR
This work proposes annotator models based on Receiver Operating Characteristic (ROC) curve analysis to consolidate the ordinal annotations from multiple annotat ors and indicates that the proposed algorithm is superior to the commonly used majorit y vo ing rule.
A Distance-Based Decision in the Credal Level
TLDR
This paper aims to demonstrate that the proposed decision rule based on a distance measure is a particular case of the rule proposed in [4], and gives experiments showing that the rule is able to decide on a set of hypotheses.
Machine-Learning for Spammer Detection in Crowd-Sourcing
TLDR
It is shown that a SVM shows itself to have a very high accuracy when the machine-learner is trained and tested on a single task and that the method was portable from more complex tasks to simpler tasks, but not vice versa.
A new distance between two bodies of evidence
Inferring Ground Truth from Subjective Labelling of Venus Images
TLDR
Empirical results suggest that accounting for subjective noise can be quite significant in terms of quantifying both human and algorithm detection performance.
A Mathematical Theory of Evidence
  • G. Shafer
  • Mathematics
    A Mathematical Theory of Evidence
  • 2020
TLDR
This book develops an alternative to the additive set functions and the rule of conditioning of the Bayesian theory: set functions that need only be what Choquet called "monotone of order of infinity." and Dempster's rule for combining such set functions.
...
...