A unified architecture for natural language processing: deep neural networks with multitask learning

Abstract

We describe a single convolutional neural network architecture that, given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semantic roles, semantically similar words and the likelihood that the sentence makes sense (grammatically and semantically) using a language model. The entire network is trained <i>jointly</i> on all these tasks using weight-sharing, an instance of <i>multitask learning</i>. All the tasks use labeled data except the language model which is learnt from unlabeled text and represents a novel form of <i>semi-supervised learning</i> for the shared tasks. We show how both <i>multitask learning</i> and <i>semi-supervised learning</i> improve the generalization of the shared tasks, resulting in state-of-the-art-performance.

DOI: 10.1145/1390156.1390177

Extracted Key Phrases

5 Figures and Tables

020040020082009201020112012201320142015201620172018
Citations per Year

1,912 Citations

Semantic Scholar estimates that this publication has 1,912 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@inproceedings{Collobert2008AUA, title={A unified architecture for natural language processing: deep neural networks with multitask learning}, author={Ronan Collobert and Jason Weston}, booktitle={ICML}, year={2008} }