• Corpus ID: 2778590

Word Sense Disambiguation with LSTM: Do We Really Need 100 Billion Words?

@article{Le2017WordSD,
  title={Word Sense Disambiguation with LSTM: Do We Really Need 100 Billion Words?},
  author={Minh Nguyen Le and Marten Postma and Jacopo Urbani},
  journal={ArXiv},
  year={2017},
  volume={abs/1712.03376}
}
Recently, Yuan et al. (2016) have shown the e ectiveness of using Long Short-Term Memory (LSTM) for performing Word Sense Disambiguation (WSD). Their proposed technique outperformed the previous state-of-the-art with several benchmarks, but neither the training data nor the source code was released. This paper presents the results of a reproduction study of this technique using only openly available datasets (GigaWord, SemCore, OMSTI) and software (TensorFlow). From them, it emerged that state… 
View PDF on arXiv
View With Semantic Reader
3 Citations
Highly Influential Citations
1
Background Citations
2
Methods Citations
3

Figures and Tables from this paper

3 Citations

Fixed-Size Ordinally Forgetting Encoding Based Word Sense Disambiguation
TLDR
This paper presents the method of using fixed-size ordinally forgetting encoding (FOFE) to solve the word sense disambiguation (WSD) problem, and demonstrates that the proposed method can achieve comparable performance to that of the state-of-the-art approach at the expense of much lower computational cost.
KDSL: a Knowledge-Driven Supervised Learning Framework for Word Sense Disambiguation
  • Shi Yin, Yi Zhou, Ruili Wang
  • Computer Science
    2019 International Joint Conference on Neural Networks (IJCNN)
  • 2019
TLDR
KDSL performs relatively well even when manually labeled data is unavailable, thus provides a potential solution for similar tasks in a lack of manual annotations and outperforms several representative state-of-the-art methods on various major benchmarks.
NILC at CWI 2018: Exploring Feature Engineering and Feature Learning
TLDR
The results show that deep neural networks are able to perform as well as traditional machine learning methods using manually engineered features for the task of complex word identification in English.

References

SHOWING 1-10 OF 30 REFERENCES
Embeddings for Word Sense Disambiguation: An Evaluation Study
TLDR
This work proposes different methods through which word embeddings can be leveraged in a state-of-the-art supervised WSD system architecture, and performs a deep analysis of how different parameters affect performance.
One Million Sense-Tagged Instances for Word Sense Disambiguation and Induction
TLDR
It is shown that the open source IMS WSD system trained on the dataset achieves stateof-the-art results in standard disambiguation tasks and a recent word sense induction task, outperforming several task submissions and strong baselines.
More is not always better: balancing sense distributions for all-words Word Sense Disambiguation
TLDR
It is shown that volume and provenance are indeed important, but that approximating the perfect balancing of the selected training data leads to an improvement of 21 points and exceeds state-of-the-art systems by 14 points while using only simple features.
AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes
TLDR
This work presents AutoExtend, a system to learn embeddings for synsets and lexemes that achieves state-of-the-art performance on word similarity and word sense disambiguation tasks.
Deep Semantic Role Labeling: What Works and What's Next
We introduce a new deep learning model for semantic role labeling (SRL) that significantly improves the state of the art, along with detailed analyses to reveal its strengths and limitations. We use…
Word sense disambiguation: A survey
TLDR
This work introduces the reader to the motivations for solving the ambiguity of words and provides a description of the task, and overviews supervised, unsupervised, and knowledge-based approaches.
SemEval-2013 Task 12: Multilingual Word Sense Disambiguation
TLDR
The experience in producing a multilingual sense-annotated corpus for the SemEval-2013 task on multilingual Word Sense Disambiguation is described, and the results of participating systems are presented and analyzed.
Sequence to Sequence Learning with Neural Networks
TLDR
This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.
Random Walks for Knowledge-Based Word Sense Disambiguation
TLDR
This article presents a WSD algorithm based on random walks over large Lexical Knowledge Bases (LKB) that performs better than other graph-based methods when run on a graph built from WordNet and eXtended WordNet.
Addressing the MFS Bias in WSD systems
TLDR
This work addressed the MFS bias in WSD systems by combining the output from a WSD system with a set of mostly static features to create a MFS classifier to decide when to and not to choose the M FS.
...
...