Generalizing through Forgetting - Domain Generalization for Symptom Event Extraction in Clinical Notes

@article{Zhou2022GeneralizingTF,
  title={Generalizing through Forgetting - Domain Generalization for Symptom Event Extraction in Clinical Notes},
  author={Sitong Zhou and Kevin Lybarger and Meliha Yetisgen-Yildiz and Mari Ostendorf},
  journal={ArXiv},
  year={2022},
  volume={abs/2209.09485}
}
Symptom information is primarily documented in free-text clinical notes and is not directly accessible for downstream applications. To address this challenge, information extraction approaches that can handle clinical language variation across different institutions and specialties are needed. In this paper, we present domain generalization for symptom extraction using pretraining and fine-tuning data that differs from the target domain in terms of institution and/or specialty and patient… 
[PDF] Semantic Reader

Figures and Tables from this paper

References

SHOWING 1-10 OF 43 REFERENCES

Publicly Available Clinical BERT Embeddings

This work explores and releases two BERT models for clinical text: one for generic clinical text and another for discharge summaries specifically, and demonstrates that using a domain-specific model yields performance improvements on 3/5 clinical NLP tasks, establishing a new state-of-the-art on the MedNLI dataset.

Chinese clinical named entity recognition with variant neural structures based on BERT methods

UDALM: Unsupervised Domain Adaptation through Language Modeling

UDALM is introduced, a fine-tuning procedure, using a mixed classification and Masked Language Model loss, that can adapt to the target domain distribution in a robust and sample efficient manner and be effectively used as a stopping criterion during UDA training.

Unsupervised Domain Clusters in Pretrained Language Models

It is shown that massive pre-trained language models implicitly learn sentence representations that cluster by domains without supervision – suggesting a simple data-driven definition of domains in textual data and proposing domain data selection methods based on such models, which require only a small set of in-domain monolingual data.

DoCoGen: Domain Counterfactual Generation for Low Resource Domain Adaptation

The proposed DoCoGen model outperforms strong baselines and improves the accuracy of a state-of-the-art unsupervised DA algorithm and can generate coherent counterfactuals consisting of multiple sentences.

Neural Unsupervised Domain Adaptation in NLP—A Survey

This survey reviews neural unsupervised domain adaptation techniques which do not require labeled target domain data, and revisits the notion of domain, and uncovers a bias in the type of Natural Language Processing tasks which received most attention.

Towards Open Domain Event Trigger Identification using Adversarial Domain Adaptation

This work leverages the adversarial domain adaptation (ADA) framework to introduce domain-invariance and preliminary experiments reveal that finetuning on 1% labeled data, followed by self-training leads to substantial improvement, reaching 51.5 and 67.2 F1 on literature and news respectively.

BioBERT: a pre-trained biomedical language representation model for biomedical text mining

This article introduces BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining), which is a domain-specific language representation model pre-trained on large-scale biomedical corpora that largely outperforms BERT and previous state-of-the-art models in a variety of biomedical text mining tasks when pre- trained on biomedical Corpora.

Unsupervised Domain Adaptation of Contextualized Embeddings: A Case Study in Early Modern English

Contextualized word embeddings such as ELMo and BERT provide a foundation for strong performance across a range of natural language processing tasks, in part by pretraining on a large and…

Adversarial and Domain-Aware BERT for Cross-Domain Sentiment Analysis

A post-training procedure is designed, which contains the target domain masked language model task and a novel domain-distinguish pre-training task that will encourage BERT to be domain-aware and distill the domain-specific features in a self-supervised way.