Towards JointUD: Part-of-speech Tagging and Lemmatization using Recurrent Neural Networks

@inproceedings{Arakelyan2018TowardsJP,
  title={Towards JointUD: Part-of-speech Tagging and Lemmatization using Recurrent Neural Networks},
  author={Gor Arakelyan and Karen Hambardzumyan and Hrant Khachatrian},
  booktitle={Conference on Computational Natural Language Learning},
  year={2018}
}
This paper describes our submission to CoNLL UD Shared Task 2018. We have extended an LSTM-based neural network designed for sequence tagging to additionally generate character-level sequences. The network was jointly trained to produce lemmas, part-of-speech tags and morphological features. Sentence segmentation, tokenization and dependency parsing were handled by UDPipe 1.2 baseline. The results demonstrate the viability of the proposed multitask architecture, although its performance still… 

Tables from this paper

CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies

The task and evaluation methodology is defined, how the data sets were prepared, report and analyze the main results, and a brief categorization of the different approaches of the participating systems are provided.

CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies

This overview paper defines the task and the updated evaluation methodology, describes data preparation, report and analyze the main results, and provides a brief categorization of the different approaches of the participating systems.

Deep Learning for Natural Language Parsing

This paper presents a multi-lingual dependency parser architecture that exceeds the accuracy of state-of-the-art parsers on languages with limited training resources by a considerable margin, and implements a parser based on this architecture to utilize transfer learning techniques to address important issues related with limited-resourced language.

Recycling and Comparing Morphological Annotation Models for Armenian Diachronic-Variational Corpus Processing

It is argued that an RNN-based model can be a valid alternative to a rule-based one giving consideration to such factors as time-consumption, reusability for different varieties of a target language and significant qualitative results in morphological annotation.

A survey on syntactic processing techniques

Computational syntactic processing is a fundamental technique in natural language processing. It normally serves as a pre-processing method to transform natural language into structured and

A Free/Open-Source Morphological Transducer for Western Armenian

We present a free/open-source morphological transducer for Western Armenian, an endangered and low-resource Indo-European language. The transducer has virtually complete coverage of the language’s

Reconciling historical data and modern computational models in corpus creation

000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049

Leveraging Pre-Trained Embeddings for Welsh Taggers

The results of the experiments on learning a simple multi-task neural network model for part-of-speech and semantic tagging for Welsh using a pre-trained embedding model from FastText are presented.

References

SHOWING 1-10 OF 29 REFERENCES

Stanford's Graph-based Neural Dependency Parser at the CoNLL 2017 Shared Task

This paper describes the neural dependency parser submitted by Stanford to the CoNLL 2017 Shared Task on parsing Universal Dependencies, which was ranked first according to all five relevant metrics for the system.

A Novel Neural Network Model for Joint POS Tagging and Graph-based Dependency Parsing

A novel neural network model that learns POS tagging and graph-based dependency parsing jointly and outperforms the state-of-the-art neural network-based Stack-propagation model for joint joint tagging and transition- based dependency parsing, resulting in a new state of the art model.

Regularizing and Optimizing LSTM Language Models

This paper proposes the weight-dropped LSTM which uses DropConnect on hidden-to-hidden weights as a form of recurrent regularization and introduces NT-ASGD, a variant of the averaged stochastic gradient method, wherein the averaging trigger is determined using a non-monotonic condition as opposed to being tuned by the user.

End-To-End Memory Networks

A neural network with a recurrent attention model over a possibly large external memory that is trained end-to-end, and hence requires significantly less supervision during training, making it more generally applicable in realistic settings.

UDPipe: Trainable Pipeline for Processing CoNLL-U Files Performing Tokenization, Morphological Analysis, POS Tagging and Parsing

UDPipe, a pipeline processing CoNLL-U-formatted files, performs tokenization, morphological analysis, part-of-speech tagging, lemmatization and dependency parsing for nearly all treebanks of Universal Dependencies 1.2.

Universal Dependencies 2.0 – CoNLL 2017 Shared Task Development and Test Data

This release contains the test data used in the CoNLL 2017 shared task on parsing Universal Dependencies, and complements the UD 2.0 release with 18 new parallel test sets and 4 test sets in surprise languages.

Named Entity Recognition with Bidirectional LSTM-CNNs

A novel neural network architecture is presented that automatically detects word- and character-level features using a hybrid bidirectional LSTM and CNN architecture, eliminating the need for most feature engineering.

CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies

This overview paper defines the task and the updated evaluation methodology, describes data preparation, report and analyze the main results, and provides a brief categorization of the different approaches of the participating systems.

Attention is All you Need

A new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely is proposed, which generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.

Convolutional Sequence to Sequence Learning

This work introduces an architecture based entirely on convolutional neural networks, which outperform the accuracy of the deep LSTM setup of Wu et al. (2016) on both WMT'14 English-German and WMT-French translation at an order of magnitude faster speed, both on GPU and CPU.