• Corpus ID: 210966428

Do We Need Word Order Information for Cross-lingual Sequence Labeling

  title={Do We Need Word Order Information for Cross-lingual Sequence Labeling},
  author={Zihan Liu and Pascale Fung},
Most of the recent work in cross-lingual adaptation does not consider the word order variances in different languages. We hypothesize that cross-lingual models that fit into the source language word order might fail to handle target languages whose word orders are different. To test our conjecture, we build an order-agnostic model for cross-lingual sequence labeling tasks. Our model does not encode the word order information of the input sequences, and the predictions for each token are based… 

Tables from this paper

Reinforced Iterative Knowledge Distillation for Cross-Lingual Named Entity Recognition

The NER techniques reported in this paper are on their way to become a fundamental component for Web ranking, Entity Pane, Answers Triggering, and Question Answering in the Microsoft Bing search engine.

Zero-Resource Cross-Domain Named Entity Recognition

This work introduces a Multi-Task Learning (MTL) by adding a new objective function to detect whether tokens are named entities or not, and introduces a framework called Mixture of Entity Experts (MoEE) to improve the robustness for zero-resource domain adaptation.

Word Reordering for Zero-shot Cross-lingual Structured Prediction

This paper builds structured prediction models with bag-of-words inputs, and introduces a new reordering module to organizing words following the source language order, which learns task-specific reordering strategies from a general-purpose order predictor model.



On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing

Investigating crosslingual transfer and posit that an orderagnostic model will perform better when transferring to distant foreign languages shows that RNN-based architectures transfer well to languages that are close to English, while self-attentive models have better overall cross-lingualtransferability and perform especially well on distant languages.

Zero-shot Cross-lingual Dialogue Systems with Transferable Latent Variables

A zero-shot adaptation of task-oriented dialogue system to low-resource languages to cope with the variance of similar sentences across different languages, which is induced by imperfect cross-lingual alignments and inherent differences in languages is proposed.

Attention-Informed Mixed-Language Training for Zero-shot Cross-lingual Task-oriented Dialogue Systems

Attention-Informed Mixed-Language Training (MLT) is introduced, a novel zero-shot adaptation method for cross-lingual task-oriented dialogue systems that leverages very few task-related parallel word pairs to generate code-switching sentences for learning the inter-lingUAL semantics across languages.

Cross-lingual Transfer Learning for Multilingual Task Oriented Dialog

This paper presents a new data set of 57k annotated utterances in English, Spanish, Spanish and Thai and uses this data set to evaluate three different cross-lingual transfer methods, finding that given several hundred training examples in the the target language, the latter two methods outperform translating the training data.

Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond

An architecture to learn joint multilingual sentence representations for 93 languages, belonging to more than 30 different families and written in 28 different scripts using a single BiLSTM encoder with a shared byte-pair encoding vocabulary for all languages, coupled with an auxiliary decoder and trained on publicly available parallel corpora.

Synthetic Treebanking for Cross-Lingual Dependency Parsing

This contribution addresses the cross-lingual viewpoint on statistical dependency parsing, in which it attempts to make use of resource-rich source language treebanks to build and adapt models for the under-resourced target languages.

Incorporating Word and Subword Units in Unsupervised Machine Translation Using Language Model Rescoring

CAiRE’s submission to the unsupervised machine translation track of the WMT’19 news shared task from German to Czech is described, and a rescoring mechanism is proposed that reuses the pre-trained language model to select the translation candidates generated through beam search.

Cross-lingual Language Model Pretraining

This work proposes two methods to learn cross-lingual language models (XLMs): one unsupervised that only relies on monolingual data, and one supervised that leverages parallel data with a new cross-lingsual language model objective.

Cross-Lingual Dependency Parsing Using Code-Mixed TreeBank

  • Meishan ZhangZhang MeishanZhang YueFu Guo-hong
  • Computer Science
    Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
  • 2019
Experiments on University Dependency Treebanks show that code-mixed treebanks are more effective than translated treebanks, giving highly competitive performances among cross-lingual parsing methods.

Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing

A novel method for multilingual transfer that utilizes deep contextual embeddings, pretrained in an unsupervised fashion, that consistently outperforms the previous state-of-the-art on 6 tested languages, yielding an improvement of 6.8 LAS points on average.