• Publications
  • Influence
Last Words: Amazon Mechanical Turk: Gold Mine or Coal Mine?
To define precisely what MTurk is and what it is not, it is hoped that this will point out opportunities for the community to deliberately value ethics above cost savings.
Influence of Pre-Annotation on POS-Tagged Corpus Development
This article details a series of carefully designed experiments aiming at evaluating the influence of automatic pre-annotation on the manual part-of-speech annotation of a corpus, both from the
Les ressources annotées, un enjeu pour l'analyse de contenu : vers une méthodologie de l'annotation manuelle de corpus. (Annotated resources, a key issue in content analysis : towards a methodology
  • Karën Fort
  • Computer Science, Philosophy
  • 7 December 2012
Nous proposons dans un premier temps une methodologie globale pour la gestion de campagnes d'annotation manuelle de corpus qui repose sur deux piliers majeurs : une organisation des campagne d'Annotation qui met l'evaluation au cœur du processus and une grille d'analyse des dimensions de complexite d'une campagne d' annotation.
Collaborative Annotation for Reliable Natural Language Processing: Technical and Sociological Aspects
There is a need for manual annotation engineering (in the sense of a precisely formalized process), and this book aims to provide a first step towards a holistic methodology, with a global view on annotation.
Using Games to Create Language Resources: Successes and Limitations of the Approach
This chapter discusses some key issues in using a gaming approach to resource creation, including task design, player motivation and data quality, and suggests key considerations for evaluating its utility in different research scenarios.
Deep Syntax Annotation of the Sequoia French Treebank
We define a deep syntactic representation scheme for French, which abstracts away from surface syntactic variation and diathesis alternations, and describe the annotation of deep syntactic
PrepLex: A Lexicon of French Prepositions for Parsing
PrepLex is a lexicon of French prepositions which provides all the syntactic information needed for parsing. It was built by comparing and merging several authoritative lexical sources. This lexicon
Proposal for an Extension of Traditional Named Entities: From Guidelines to Evaluation, an Overview
The definition and novelty of extended named entity annotation guidelines are presented, the human annotation of a global corpus and of a mini reference corpus, and the evaluation of annotations through the computation of inter-annotator agreements are discussed.
Crowdsourcing Complex Language Resources: Playing to Annotate Dependency Syntax
It is shown that with suitable mechanisms (decomposition of the task, training of the players and regular control of the annotation quality during the game), it is possible to obtain annotations whose quality is significantly higher than that obtainable with a parser.
Structured Named Entities in two distinct press corpora: Contemporary Broadcast News and Old Newspapers
This paper compares the reference annotation of structured named entities in two corpora with different origins and properties. It addresses two questions linked to such a comparison. On the one