Zero-Resource Cross-Domain Named Entity Recognition

@inproceedings{Liu2020ZeroResourceCN,
  title={Zero-Resource Cross-Domain Named Entity Recognition},
  author={Zihan Liu and Genta Indra Winata and Pascale Fung},
  booktitle={REPL4NLP},
  year={2020}
}
Existing models for cross-domain named entity recognition (NER) rely on numerous unlabeled corpus or labeled NER training data in target domains. However, collecting data for low-resource target domains is not only expensive but also time-consuming. Hence, we propose a cross-domain NER model that does not use any external resources. We first introduce a Multi-Task Learning (MTL) by adding a new objective function to detect whether tokens are named entities or not. We then introduce a framework… 

Figures and Tables from this paper

CrossNER: Evaluating Cross-Domain Named Entity Recognition
TLDR
Results show that focusing on the fractional corpus containing domain-specialized entities and utilizing a more challenging pre-training strategy in domain-adaptive pre- training are beneficial for the NER domain adaptation, and the proposed method can consistently outperform existing cross-domain NER baselines.
Coach: A Coarse-to-Fine Approach for Cross-domain Slot Filling
TLDR
This paper proposes a Coarse-to-fine approach (Coach) for cross-domain slot filling, and proposes a template regularization approach to improve the adaptation robustness by regularizing the representation of utterances based on utterance templates.
AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization
TLDR
A study of domain adaptation for the abstractive summarization task across six diverse target domains in a low-resource setting and finds that continuing pre-training could lead to the pre-trained model's catastrophic forgetting, and a learning method with less forgetting can alleviate this issue.
A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios
TLDR
A structured overview of methods that enable learning when training data is sparse including mechanisms to create additional labeled data like data augmentation and distant supervision as well as transfer learning settings that reduce the need for target supervision are given.
NLP Service APIs and Models for Efficient Registration of New Clients
TLDR
This work initiates a study of practical and lightweight adaptation of centralized NLP services to clients, and demonstrates the proposed architecture using sentiment labeling, NER, and predictive language modeling.
Contrastive Zero-Shot Learning for Cross-Domain Slot Filling with Adversarial Attack
TLDR
A Contrastive Zero-Shot Learning with Adversarial Attack (CZSL-Adv) method for the cross-domain slot filling that aims to map slot value contextual representations to the corresponding slot description representations and introduces an adversarial attack training strategy to improve model robustness.
GEMNET: Effective Gated Gazetteer Representations for Recognizing Complex Entities in Low-context Input
TLDR
GEMNET is proposed, a novel approach for gazetteer knowledge integration, including a flexible Contextual Gazetteer Representation encoder that can be fused with any word-level model; and a Mixture-of- Experts gating network that overcomes the feature overuse issue by learning to conditionally combine the context and gazetteser features, instead of assigning them fixed weights.
Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling
TLDR
A novel approach based on prototypical contrastive learning with a dynamic label confusion strategy for zero-shot slot filling to establish the label dependence between the source domains and the target domain on-the-fly.
Cross-domain Named Entity Recognition via Graph Matching
TLDR
By representing label relationships as graphs, this work formulate cross-domain NER as a graph matching problem and shows that the proposed method outperforms a series of transfer learning, multi-task learning, and few-shot learning methods.
What Can Knowledge Bring to Machine Learning?—A Survey of Low-shot Learning for Structured Data
TLDR
The fundamental factors of low-shot learning technologies are reviewed, with a focus on the operation of structured knowledge under different low- shot conditions, and the prospects and gaps of industrial applications and future research directions are pointed out.
...
...

References

SHOWING 1-10 OF 26 REFERENCES
Cross-Domain NER using Cross-Domain Language Modeling
TLDR
This work considers using cross- domain LM as a bridge cross-domains for NER domain adaptation, performing cross-domain and cross-task knowledge transfer by designing a novel parameter generation network and shows that this method can effectively extract domain differences from cross- domains LM contrast, allowing unsupervised domain adaptation while also giving state-of-the-art results among supervised domain adaptation methods.
Neural Adaptation Layers for Cross-domain Named Entity Recognition
TLDR
This paper proposes lightweight yet effective methods for performing domain adaptation for neural models, which introduce adaptation layers on top of existing neural architectures, where no re-training using the source domain data is required.
Named Entity Recognition with Bidirectional LSTM-CNNs
TLDR
A novel neural network architecture is presented that automatically detects word- and character-level features using a hybrid bidirectional LSTM and CNN architecture, eliminating the need for most feature engineering.
Transfer Learning for Named-Entity Recognition with Neural Networks
TLDR
It is demonstrated that transferring an ANN model trained on a large labeled dataset to another dataset with a limited number of labels improves upon the state-of-the-art results on two different datasets for patient note de-identification.
Label-Aware Double Transfer Learning for Cross-Specialty Medical Named Entity Recognition
TLDR
A label-aware double transfer learning framework (La-DTL) for cross-specialty NER, so that a medical NER system designed for one specialty could be conveniently applied to another one with minimal annotation efforts, is proposed.
Neural Cross-Lingual Named Entity Recognition with Minimal Resources
TLDR
This work proposes a method that finds translations based on bilingual word embeddings and uses self-attention to improve robustness to word order differences, and demonstrates that these methods achieve state-of-the-art or competitive NER performance on commonly tested languages under a cross-lingual setting, with much lower resource requirements than past approaches.
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
TLDR
A novel neutral network architecture is introduced that benefits from both word- and character-level representations automatically, by using combination of bidirectional LSTM, CNN and CRF, thus making it applicable to a wide range of sequence labeling tasks.
Robust Zero-Shot Cross-Domain Slot Filling with Example Values
TLDR
This work proposes utilizing both the slot description and a small number of examples of slot values, which may be easily available, to learn semantic representations of slots which are transferable across domains and robust to misaligned schemas.
Towards Zero-Shot Frame Semantic Parsing for Domain Scaling
TLDR
This paper proposes a deep learning based approach that can utilize only the slot description in context without the need for any labeled or unlabeled in-domain examples, to quickly bootstrap a new domain.
Do We Need Word Order Information for Cross-lingual Sequence Labeling
TLDR
An order-agnostic model is built for cross-lingual sequence labeling tasks that shows that getting rid of word order information is able to achieve better zero-shot crosslingual performance than baseline models.
...
...