Adapting Event Extractors to Medical Data: Bridging the Covariate Shift

@article{Naik2020AdaptingEE,
  title={Adapting Event Extractors to Medical Data: Bridging the Covariate Shift},
  author={Aakanksha Naik and Jill Fain Lehman and Carolyn Penstein Ros{\'e}},
  journal={ArXiv},
  year={2020},
  volume={abs/2008.09266}
}
We tackle the task of adapting event extractors to new domains without labeled data, by aligning the marginal distributions of source and target domains. As a testbed, we create two new event extraction datasets using English texts from two medical domains: (i) clinical notes, and (ii) doctor-patient conversations. We test the efficacy of three marginal alignment techniques: (i) adversarial domain adaptation (ADA), (ii) domain adaptive fine-tuning (DAFT), and (iii) a new instance weighting… 

Figures and Tables from this paper

Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks

This work conceptualizes the long tail using macro-level dimensions, performs a qualitative meta-analysis of 100 representative papers on transfer learning research for NLU, and performs a case study comparing the performance of various adaptation methods on clinical narratives, which provides interesting insights.

Year 2021: COVID-19, Information Extraction and BERTization among the Hottest Topics in Medical Natural Language Processing

Summary Objectives : Analyze the content of publications within the medical natural language processing (NLP) domain in 2021. Methods : Automatic and manual preselection of publications to be

References

SHOWING 1-10 OF 57 REFERENCES

Towards Open Domain Event Trigger Identification using Adversarial Domain Adaptation

This work leverages the adversarial domain adaptation (ADA) framework to introduce domain-invariance and preliminary experiments reveal that finetuning on 1% labeled data, followed by self-training leads to substantial improvement, reaching 51.5 and 67.2 F1 on literature and news respectively.

Publicly Available Clinical BERT Embeddings

This work explores and releases two BERT models for clinical text: one for generic clinical text and another for discharge summaries specifically, and demonstrates that using a domain-specific model yields performance improvements on 3/5 clinical NLP tasks, establishing a new state-of-the-art on the MedNLI dataset.

Unsupervised Domain Adaptation of Contextualized Embeddings for Sequence Labeling

Domain-adaptive fine-tuning offers a simple and effective approach for the unsupervised adaptation of sequence labeling to difficult new domains and is tested on sequence labeling in two challenging domains: Early Modern English and Twitter.

Part-of-Speech Tagging for Twitter with Adversarial Neural Networks

A novel neural network to make use of out-of-domain labeled data, unlabeled in- domain data, and labeled in-domain data is proposed to learn common features through adversarial discriminator for Tweets tagging.

Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks

It is consistently found that multi-phase adaptive pretraining offers large gains in task performance, and it is shown that adapting to a task corpus augmented using simple data selection strategies is an effective alternative, especially when resources for domain-adaptive pretraining might be unavailable.

Domain-Adversarial Training of Neural Networks

A new representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions, which can be achieved in almost any feed-forward model by augmenting it with few standard layers and a new gradient reversal layer.

Neural Structural Correspondence Learning for Domain Adaptation

This model is a three-layer neural network that learns to encode the nonpivot features of an input example into a low-dimensional representation, so that the existence of pivot features in the example can be decoded from that representation.

Instance Weighting for Domain Adaptation in NLP

This paper formally analyze and characterize the domain adaptation problem from a distributional view, and shows that there are two distinct needs for adaptation, corresponding to the different distributions of instances and classification functions in the source and the target domains.

SemEval-2017 Task 12: Clinical TempEval

Nine sub-tasks were included, covering problems in time expression identification, event expression identification and temporal relation identification, and most tasks observed about a 20 point drop over Clinical TempEval 2016.

Open domain event extraction from twitter

TwiCal is described-- the first open-domain event-extraction and categorization system for Twitter, and a novel approach for discovering important event categories and classifying extracted events based on latent variable models is presented.
...