Summaformers @ LaySumm 20, LongSumm 20

  title={Summaformers @ LaySumm 20, LongSumm 20},
  author={Sayar Ghosh Roy and Nikhil Pinnaparaju and Risubh Jain and Manish Gupta and Vasudeva Varma},
Automatic text summarization has been widely studied as an important task in natural language processing. Traditionally, various feature engineering and machine learning based systems have been proposed for extractive as well as abstractive text summarization. Recently, deep learning based, specifically Transformer-based systems have been immensely popular. Summarization is a cognitively challenging task – extracting summary worthy sentences is laborious, and expressing semantics in brief when… 

Figures and Tables from this paper

TSTR: Too Short to Represent, Summarize with Details! Intro-Guided Extended Summary Generation
This paper proposes T STR, an extractive summarizer that utilizes the introductory information of documents as pointers to their salient information, and generates extended summaries that facilitate a faster read while providing details beyond coarse information.
On Generating Extended Summaries of Long Documents
This paper exploits hierarchical structure of the documents and incorporates it into an extractive summarization model through a multi-task learning approach and shows that the multi-tasking approach can adjust extraction probability distribution to the favor of summary-worthy sentences across diverse sections.
CNLP-NITS @ LongSumm 2021: TextRank Variant for Generating Long Summaries
This paper discusses the extractive summarization approach to solve the task of summarization of scholarly documents using TextRank algorithm with the BM25 score as a similarity function and produced pretty decent results with minimal compute power and time.
Cross-domain multi-task learning for sequential sentence classification in research papers
A novel uniform deep learning architecture and multi-task learning for cross-domain sequential sentence classification in scientific texts is suggested and comprehensive experimental results indicate that models, which are trained on datasets from different scientific domains, benefit from one another when using the proposed multi- task learning architecture.
Sequential Sentence Classification in Research Papers using Cross-Domain Multi-Task Learning
It is demonstrated that models, which are trained on datasets from different scientific domains, benefit from one another when using the proposed multi-task learning architecture, and the approach outperforms the state of the art on three benchmark datasets.


On Extractive and Abstractive Neural Document Summarization with Transformer Language Models
A simple extractive step is performed before generating a summary, which is then used to condition the transformer language model on relevant information before being tasked with Generating a summary.
Hybrid MemNet for Extractive Summarization
A fully data-driven end-to-end deep network which is called as Hybrid MemNet for single document summarization task, which learns the continuous unified representation of a document before generating its summary.
Text Summarization with Pretrained Encoders
This paper introduces a novel document-level encoder based on BERT which is able to express the semantics of a document and obtain representations for its sentences and proposes a new fine-tuning schedule which adopts different optimizers for the encoder and the decoder as a means of alleviating the mismatch between the two.
Unity in Diversity: Learning Distributed Heterogeneous Sentence Representation for Extractive Summarization
This work develops a data-driven novel summary system called HNet, which exploits the various semantic and compositional aspects latent in a sentence to capture document independent features.
A Supervised Approach to Extractive Summarisation of Scientific Papers
This paper introduces a new dataset for summarisation of computer science publications by exploiting a large resource of author provided summaries and develops models on the dataset making use of both neural sentence encoding and traditionally used summarisation features.
Extractive Summarization of Long Documents by Combining Global and Local Context
A novel neural single-document extractive summarization model for long documents, incorporating both the global context of the whole document and the local context within the current topic, where it outperforms previous work, both extractive and abstractive models.
HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization
This work proposes Hibert (as shorthand for HIerachical Bidirectional Encoder Representations from Transformers) for document encoding and a method to pre-train it using unlabeled data and achieves the state-of-the-art performance on these two datasets.
SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents
We present SummaRuNNer, a Recurrent Neural Network (RNN) based sequence model for extractive summarization of documents and show that it achieves performance better than or comparable to
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
This systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks and achieves state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more.
A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents
This work proposes the first model for abstractive summarization of single, longer-form documents (e.g., research papers), consisting of a new hierarchical encoder that models the discourse structure of a document, and an attentive discourse-aware decoder to generate the summary.