Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction

@inproceedings{Huang2022MultilingualGL,
  title={Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction},
  author={Kuan-Hao Huang and I-Hung Hsu and P. Natarajan and Kai-Wei Chang and Nanyun Peng},
  booktitle={ACL},
  year={2022}
}
We present a study on leveraging multilingual pre-trained generative language models for zero-shot cross-lingual event argument extraction (EAE). By formulating EAE as a language generation task, our method effectively encodes event structures and captures the dependencies between arguments. We design language-agnostic templates to represent the event argument structures, which are compatible with any language, hence facilitating the cross-lingual transfer. Our proposed model finetunes… 

Figures and Tables from this paper

A Simple and Unified Tagging Model with Priming for Relational Structure Predictions
TLDR
This work demonstrates that simple tagging models can surprisingly achieve competitive performances with a small trick — priming, and proposes an efficient approximation to the method, which can perform almost the same performance but with faster inference speed.
Summarization as Indirect Supervision for Relation Extraction
TLDR
S U RE is presented, which converts RE into a summarization formulation and develops sentence and relation conversion techniques that essentially bridge the formulation of summarization and RE tasks, showing that summarization is a promising source of indirect supervision to improve RE models.

References

SHOWING 1-10 OF 61 REFERENCES
Improving Cross-Lingual Transfer for Event Argument Extraction with Language-Universal Sentence Structures
TLDR
This paper introduces two novel sources of language-independent information for CEAE models based on the semantic similarity and the universal dependency relations of the word pairs in different languages and proposes to use the two sources of information to produce shared sentence structures to bridge the gap between languages and improve the cross-lingual performance of the CEae models.
Language Model Priming for Cross-Lingual Event Extraction
We present a novel, language-agnostic approach to “priming” language models for the task of event extraction, providing particularly effective performance in low-resource and zeroshot cross-lingual
Cross-lingual Structure Transfer for Relation and Event Extraction
TLDR
It is found that language-universal symbolic and distributional representations are complementary for cross-lingual structure transfer, and the event argument role labeling model transferred from English to Chinese achieves similar performance as the model trained from Chinese.
Cross-Lingual Dependency Parsing with Unlabeled Auxiliary Languages
TLDR
This work explores adversarial training for learning contextual encoders that produce invariant representations across languages to facilitate cross-lingual transfer and proposes to leverage unannotated sentences from auxiliary languages to help learning language-agnostic representations.
Target Language-Aware Constrained Inference for Cross-lingual Dependency Parsing
TLDR
It is shown that weak supervisions of linguistic knowledge for the target languages can improve a cross-lingual graph-based dependency parser substantially and propose new algorithms that adapt two techniques, Lagrangian relaxation and posterior regularization, to conduct inference with corpus-statistics constraints.
GATE: Graph Attention Transformer Encoder for Cross-lingual Relation and Event Extraction
TLDR
This work introduces GATE, a self-attention mechanism where it explicitly fuse structural information to learn the dependencies between words at different syntactic distances, and test its cross-lingual transferability on relation and event extraction tasks.
Neural Cross-Lingual Relation Extraction Based on Bilingual Word Embedding Mapping
TLDR
A new approach for cross-lingual RE model transfer based on bilingual word embedding mapping is proposed, which projects word embeddings from a target language to a source language, so that a well-trained source-language neural network RE model can be directly applied to the target language.
Neural Cross-Lingual Event Detection with Minimal Parallel Resources
TLDR
To construct a lexical mapping between different languages, a context-dependent translation method is devised; to treat the word order difference problem, a shared syntactic order event detector for multilingual co-training is proposed.
Language Models are Few-shot Multilingual Learners
TLDR
It is shown that, given a few English examples as context, pre-trained language models can predict not only English test samples but also non-English ones, and they are competitive compared to the existing state-of-the-art cross-lingual models and translation models.
On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing
TLDR
Investigating crosslingual transfer and posit that an orderagnostic model will perform better when transferring to distant foreign languages shows that RNN-based architectures transfer well to languages that are close to English, while self-attentive models have better overall cross-lingualtransferability and perform especially well on distant languages.
...
...