The SENSEI Annotated Corpus: Human Summaries of Reader Comment Conversations in On-line News

@inproceedings{Barker2016TheSA,
  title={The SENSEI Annotated Corpus: Human Summaries of Reader Comment Conversations in On-line News},
  author={E. Barker and M. Paramita and Ahmet Aker and Emina Kurtic and M. Hepple and R. Gaizauskas},
  booktitle={SIGDIAL Conference},
  year={2016}
}
Researchers are beginning to explore how to generate summaries of extended argumentative conversations in social media, such as those found in reader comments in on-line news. To date, however, there has been little discussion of what these summaries should be like and a lack of humanauthored exemplars, quite likely because writing summaries of this kind of interchange is so difficult. In this paper we propose one type of reader comment summary – the conversation overview summary… Expand

Figures, Tables, and Topics from this paper

The SENSEI Overview of Newspaper Readers' Comments
TLDR
An evaluation to assess how well summarization systems support users in time-limited tasks (identifying issues and characterizing opinions) gives good results for this prototype. Expand
Finding Good Conversations Online: The Yahoo News Annotated Comments Corpus
TLDR
This work presents a dataset and annotation scheme for the new task of identifying “good” conversations that occur online, which it is called ERICs: Engaging, Respectful, and/or Informative Conversations, which is one of the largest annotated corpora of online human dialogues, with the most detailed set of annotations. Expand
ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining
TLDR
Annotation protocols motivated by an issues–viewpoints–assertions framework are designed to crowdsource four new datasets on diverse online conversation forms of news comments, discussion forums, community question answering forums, and email threads, and benchmark state-of-the-art models on these datasets and analyze characteristics associated with the data. Expand
Creating a reference data set for the summarization of discussion forum threads
TLDR
An elaborate user evaluation study to determine human preferences in forum summarization and to create a reference data set is presented and shows that even for a summarization task with low inter-rater agreement, a model can be trained that generates sensible summaries. Expand
Query-based summarization of discussion threads
TLDR
It is concluded that query-based summarization with web queries is challenging because the queries are short, and a click on a result is not a direct indicator for the relevance of the result. Expand
Automatic label generation for news comment clusters
TLDR
It is illustrated how cluster labels can be used to generate cluster summaries and two alternative sum- mary formats are presented: a pie chart summary and an ab- stractive summary. Expand
Generating Multi-Sentence Abstractive Summaries of Interleaved Texts
TLDR
An end-to-end trainable trainable encoder-decoder network for summarizing interleaved posts enhances performance over a sequence to sequence framework by 8% on a synthetic interleaves texts dataset. Expand
A Survey on Dialogue Summarization: Recent Advances and New Frontiers
  • Xiachong Feng, Xiaocheng Feng, Bing Qin
  • Computer Science
  • ArXiv
  • 2021
TLDR
This first survey of dialogue summarization can provide the community with a quick access and a general picture to this task and motivate future researches. Expand
Automatic Article Commenting: the Task and Dataset
TLDR
A large-scale Chinese dataset with millions of real comments and a human-annotated subset characterizing the comments’ varying quality is introduced, and automatic metrics that generalize a broad set of popular reference-based metrics and exhibit greatly improved correlations with human evaluations are developed. Expand
Commenting : the Task and Dataset
Comments of online articles provide extended views and improve user engagement. Automatically making comments thus become a valuable functionality for online forums, intelligent chatbots, etc. ThisExpand
...
1
2
...

References

SHOWING 1-10 OF 19 REFERENCES
What's the Issue Here?: Task-based Evaluation of Reader Comment Summarization Systems
TLDR
A task-based evaluation framework for reader comment summarization is defined that allows summarization systems to be assessed in terms of how well they support users in a time-limited task of identifying issues and characterising opinion on issues in comments. Expand
Using Summarization to Discover Argument Facets in Online Idealogical Dialog
TLDR
It is hypothesize that the CENTRAL PROPOSITIONS are exactly those arguments that people find most salient, and use human summarization as a probe for discovering them. Expand
Summarizing Multi-Party Argumentative Conversations in Reader Comment on News
TLDR
An issue-centred scheme for analysing and graphically representing argument in reader comment discussion in on-line news is proposed and summaries capturing the argumentative nature of reader comment can be generated from this graphical representation. Expand
Topic-driven reader comments summarization
TLDR
The experimental results show that EXTM significantly outperforms MSTM by perplexity and through a user study, it is confirmed that the comment summary generated by EXTM achieves better intra-clusters topic cohesion and inter-cluster topic diversity. Expand
Summarizing Newspaper Comments
TLDR
PageRank was found to be the prefered ranking system when compared with TF-IDF, Mutual Information gain and Maximal Marginal Relevance and evaluated against sets of comments summarized by a journalist for the Guardian letters page. Expand
Manual and automatic evaluation of summaries
TLDR
This paper shows the instability of the manual evaluation of summaries, and investigates the feasibility of automated summary evaluation based on the recent BLEU method from machine translation using accumulative n-gram overlap scores between system and human summaries. Expand
Argument Mining: Extracting Arguments from Online Dialogue
TLDR
This work can train regressors to predict the quality of extracted arguments with RRSE values as low as .73 for some topics, and develops regressors that are topic independent, when trained on topic independent features. Expand
Automatic label generation for news comment clusters
TLDR
It is illustrated how cluster labels can be used to generate cluster summaries and two alternative sum- mary formats are presented: a pie chart summary and an ab- stractive summary. Expand
Summarizing Spoken and Written Conversations
TLDR
By treating meetings and emails as conversations with general conversational features in common, the system can achieve competitive results with state-of-the-art systems that rely on more domain-specific features. Expand
Summarizing User-Contributed Comments
TLDR
This paper proposes a clustering-based approach for identifying correlated groups of comments; and a precedence-based ranking framework for automatically selecting informative user-contributed comments, and finds that in combination, these two salient features yield promising results. Expand
...
1
2
...