Zero-Resource Cross-Domain Named Entity Recognition

  title={Zero-Resource Cross-Domain Named Entity Recognition},
  author={Zihan Liu and Genta Indra Winata and Pascale Fung},
Existing models for cross-domain named entity recognition (NER) rely on numerous unlabeled corpus or labeled NER training data in target domains. However, collecting data for low-resource target domains is not only expensive but also time-consuming. Hence, we propose a cross-domain NER model that does not use any external resources. We first introduce a Multi-Task Learning (MTL) by adding a new objective function to detect whether tokens are named entities or not. We then introduce a framework… 

Figures and Tables from this paper

CrossNER: Evaluating Cross-Domain Named Entity Recognition

Results show that focusing on the fractional corpus containing domain-specialized entities and utilizing a more challenging pre- training strategy in domain-adaptive pre-training are beneficial for the NER domain adaptation, and the proposed method can consistently outperform existing cross-domain NER baselines.

Coach: A Coarse-to-Fine Approach for Cross-domain Slot Filling

This paper proposes a Coarse-to-fine approach (Coach) for cross-domain slot filling, and proposes a template regularization approach to improve the adaptation robustness by regularizing the representation of utterances based on utterance templates.

AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization

A study of domain adaptation for the abstractive summarization task across six diverse target domains in a low-resource setting and finds that continuing pre-training could lead to the pre-trained model's catastrophic forgetting, and a learning method with less forgetting can alleviate this issue.

A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios

A structured overview of methods that enable learning when training data is sparse including mechanisms to create additional labeled data like data augmentation and distant supervision as well as transfer learning settings that reduce the need for target supervision are given.

NLP Service APIs and Models for Efficient Registration of New Clients

This work initiates a study of practical and lightweight adaptation of centralized NLP services to clients, and demonstrates the proposed architecture using sentiment labeling, NER, and predictive language modeling.

Contrastive Zero-Shot Learning for Cross-Domain Slot Filling with Adversarial Attack

A Contrastive Zero-Shot Learning with Adversarial Attack (CZSL-Adv) method for the cross-domain slot filling that aims to map slot value contextual representations to the corresponding slot description representations and introduces an adversarial attack training strategy to improve model robustness.

GEMNET: Effective Gated Gazetteer Representations for Recognizing Complex Entities in Low-context Input

GEMNET is proposed, a novel approach for gazetteer knowledge integration, including a flexible Contextual Gazetteer Representation encoder that can be fused with any word-level model; and a Mixture-of- Experts gating network that overcomes the feature overuse issue by learning to conditionally combine the context and gazetteser features, instead of assigning them fixed weights.

Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling

A novel approach based on prototypical contrastive learning with a dynamic label confusion strategy for zero-shot slot filling to establish the label dependence between the source domains and the target domain on-the-fly.

What Can Knowledge Bring to Machine Learning?—A Survey of Low-shot Learning for Structured Data

The fundamental factors of low-shot learning technologies are reviewed, with a focus on the operation of structured knowledge under different low- shot conditions, and the prospects and gaps of industrial applications and future research directions are pointed out.

PDALN: Progressive Domain Adaptation over a Pre-trained Model for Low-Resource Cross-Domain Named Entity Recognition

PDALN can effectively adapt high-resource domains to low-resource target domains, even if they are diverse in terms and writing styles, and comparison with other baselines indicates the state-of-the-art performance of PDALN.



Cross-Domain NER using Cross-Domain Language Modeling

This work considers using cross- domain LM as a bridge cross-domains for NER domain adaptation, performing cross-domain and cross-task knowledge transfer by designing a novel parameter generation network and shows that this method can effectively extract domain differences from cross- domains LM contrast, allowing unsupervised domain adaptation while also giving state-of-the-art results among supervised domain adaptation methods.

Neural Adaptation Layers for Cross-domain Named Entity Recognition

This paper proposes lightweight yet effective methods for performing domain adaptation for neural models, which introduce adaptation layers on top of existing neural architectures, where no re-training using the source domain data is required.

Named Entity Recognition with Bidirectional LSTM-CNNs

A novel neural network architecture is presented that automatically detects word- and character-level features using a hybrid bidirectional LSTM and CNN architecture, eliminating the need for most feature engineering.

Transfer Learning for Named-Entity Recognition with Neural Networks

It is demonstrated that transferring an ANN model trained on a large labeled dataset to another dataset with a limited number of labels improves upon the state-of-the-art results on two different datasets for patient note de-identification.

Label-Aware Double Transfer Learning for Cross-Specialty Medical Named Entity Recognition

A label-aware double transfer learning framework (La-DTL) for cross-specialty NER, so that a medical NER system designed for one specialty could be conveniently applied to another one with minimal annotation efforts, is proposed.

Neural Cross-Lingual Named Entity Recognition with Minimal Resources

This work proposes a method that finds translations based on bilingual word embeddings and uses self-attention to improve robustness to word order differences, and demonstrates that these methods achieve state-of-the-art or competitive NER performance on commonly tested languages under a cross-lingual setting, with much lower resource requirements than past approaches.

End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF

A novel neutral network architecture is introduced that benefits from both word- and character-level representations automatically, by using combination of bidirectional LSTM, CNN and CRF, thus making it applicable to a wide range of sequence labeling tasks.

Robust Zero-Shot Cross-Domain Slot Filling with Example Values

This work proposes utilizing both the slot description and a small number of examples of slot values, which may be easily available, to learn semantic representations of slots which are transferable across domains and robust to misaligned schemas.

Towards Zero-Shot Frame Semantic Parsing for Domain Scaling

This paper proposes a deep learning based approach that can utilize only the slot description in context without the need for any labeled or unlabeled in-domain examples, to quickly bootstrap a new domain.

Do We Need Word Order Information for Cross-lingual Sequence Labeling

An order-agnostic model is built for cross-lingual sequence labeling tasks that shows that getting rid of word order information is able to achieve better zero-shot crosslingual performance than baseline models.