Number Entity Recognition

  title={Number Entity Recognition},
  author={Dhanasekar Sundararaman and Vivek Subramanian and Guoyin Wang and Liyan Xu and Lawrence Carin},
Numbers are essential components of text, like any other word tokens, from which natural language processing (NLP) models are built and deployed. Though numbers are typically not accounted for distinctly in most NLP tasks, there is still an underlying amount of numeracy already exhibited by NLP models. In this work, we attempt to tap this potential of state-of-the-art NLP models and transfer their ability to boost performance in related tasks. Our proposed classification of numbers into entities… 

Figures and Tables from this paper


Do NLP Models Know Numbers? Probing Numeracy in Embeddings
This work investigates the numerical reasoning capabilities of a state-of-the-art question answering model on the DROP dataset and finds this model excels on questions that require numerical reasoning, i.e., it already captures numeracy.
Exploring Numeracy in Word Embeddings
Inspired by cognitive studies on how humans perceive numbers, an analysis framework is developed to test how well word embeddings capture two essential properties of numbers: magnitude and numeration.
Joint Embedding of Words and Labels for Text Classification
This work proposes to view text classification as a label-word joint embedding problem: each label is embedded in the same space with the word vectors, and introduces an attention framework that measures the compatibility of embeddings between text sequences and labels.
Syntax-Infused Transformer and BERT models for Machine Translation and Natural Language Understanding
This work shows that the syntax-infused Transformer with multiple features achieves an improvement of 0.7 BLEU when trained on the full WMT 14 English to German translation dataset, and finds that the incorporation of syntax into BERT fine-tuning outperforms baseline on a number of downstream tasks from the GLUE benchmark.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
An Empirical Investigation of Contextualized Number Prediction
A suite of output distribution parameterizations are introduced that incorporate latent variables to add expressivity and better fit the natural distribution of numeric values in running text, and combine them with both recur-rent and transformer-based encoder architectures.
Injecting Numerical Reasoning Skills into Language Models
This work shows that numerical reasoning is amenable to automatic data generation, and thus one can inject this skill into pre-trained LMs, by generating large amounts of data, and training in a multi-task setup.
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
This work proposes a method to pre-train a smaller general-purpose language representation model, called DistilBERT, which can be fine-tuned with good performances on a wide range of tasks like its larger counterparts, and introduces a triple loss combining language modeling, distillation and cosine-distance losses.
How do lexical semantics affect translation? An empirical study
This work investigates relationships on a variety of low-resource language pairs from the OpenSubtitles2016 database and finds that the more similar the target language is to English, the greater the translation performance.
Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition
The CoNLL-2003 shared task: language-independent named entity recognition is described and a general overview of the systems that have taken part in the task and discuss their performance is presented.