Cross-lingual transfer of abstractive summarizer to less-resource language

  title={Cross-lingual transfer of abstractive summarizer to less-resource language},
  author={Alevs vZagar and Marko Robnik-vSikonja},
  journal={Journal of Intelligent Information Systems},
Automatic text summarization extracts important information from texts and presents the information in the form of a summary. Abstractive summarization approaches progressed significantly by switching to deep neural networks, but results are not yet satisfactory, especially for languages where large training sets do not exist. In several natural language processing tasks, a cross-lingual model transfer is successfully applied in less-resource languages. For summarization, the cross-lingual… 



NCLS: Neural Cross-Lingual Summarization

This work presents an end-to-end CLS framework, which it refers to as Neural Cross-Lingual Summarization (NCLS), and proposes to further improve NCLS by incorporating two related tasks, monolingual summarization and machine translation, into the training process of CLS under multi-task learning.

Cross-Lingual Natural Language Generation via Pre-Training

Experimental results on question generation and abstractive summarization show that the model outperforms the machine-translation-based pipeline methods for zero-shot cross-lingual generation and improves NLG performance of low-resource languages by leveraging rich-resource language data.

Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond

An architecture to learn joint multilingual sentence representations for 93 languages, belonging to more than 30 different families and written in 28 different scripts using a single BiLSTM encoder with a shared byte-pair encoding vocabulary for all languages, coupled with an auxiliary decoder and trained on publicly available parallel corpora.

Sequential Transfer Learning in NLP for German Text Summarization

The experiments suggest that pre-trained language models can improve summarizing texts and it is found that using multilingual BERT as contextual embeddings lifts the model by about 9 points of ROUGE-1 and RouGE-2 on a German summarization task.

PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization

This work proposes pre-training large Transformer-based encoder-decoder models on massive text corpora with a new self-supervised objective, PEGASUS, and demonstrates it achieves state-of-the-art performance on all 12 downstream datasets measured by ROUGE scores.

A Robust Abstractive System for Cross-Lingual Summarization

A robust neural abstractive summarization system for cross-lingual summarization that achieves significantly higher fluency than a standard copy-attention summarizer on automatically translated input documents, as well as comparable content selection.

Cross-Lingual Word Embeddings for Low-Resource Language Modeling

This work investigates the use of bilingual lexicons to improve language models when textual training data is limited to as few as a thousand sentences, and involves learning cross-lingual word embeddings as a preliminary step in training monolingual language models.

Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting

An accurate and fast summarization model that first selects salient sentences and then rewrites them abstractively to generate a concise overall summary is proposed, which achieves the new state-of-the-art on all metrics on the CNN/Daily Mail dataset, as well as significantly higher abstractiveness scores.

Abstractive Text Summarization based on Language Model Conditioning and Locality Modeling

A new method of BERT-windowing, which allows chunk-wise processing of texts longer than the BERT window size and how locality modeling, i.e., the explicit restriction of calculations to the local context, can affect the summarization ability of the Transformer.

Word Translation Without Parallel Data

It is shown that a bilingual dictionary can be built between two languages without using any parallel corpora, by aligning monolingual word embedding spaces in an unsupervised way.