• Corpus ID: 7377523

Using the Omega Index for Evaluating Abstractive Community Detection

  title={Using the Omega Index for Evaluating Abstractive Community Detection},
  author={Gabriel Murray and Giuseppe Carenini and Raymond T. Ng},
Numerous NLP tasks rely on clustering or community detection algorithms. For many of these tasks, the solutions are disjoint, and the relevant evaluation metrics assume nonoverlapping clusters. In contrast, the relatively recent task of abstractive community detection (ACD) results in overlapping clusters of sentences. ACD is a sub-task of an abstractive summarization system and represents a twostep process. In the first step, we classify sentence pairs according to whether the sentences should… 

Figures and Tables from this paper

Metrics for Community Analysis
A survey of the start-of-the-art metrics used for the detection and the evaluation of community structure and a comparative analysis of these metrics in measuring the goodness of the underlying community structure is presented.
Metrics for Community Analysis: A Survey
A comprehensive and structured overview of the start-of-the-art metrics used for the detection and the evaluation of community structure and conducts experiments on synthetic and real-world networks to present a comparative analysis of these metrics in measuring the goodness of the underlying community structure.
Abstractive Meeting Summarization with Entailment and Fusion
We propose a novel end-to-end framework for abstractive meeting summarization. We cluster sentences in the input into communities and build an entailment graph over the sentence communities to
Overlapping Community Detection by Node-Weighting
The main idea of the algorithm is to find a good seed and then greedily expand it based on an improved community quality metric and optimizes the community structure to ensure the quality of community partitioning.
Efficient detection of communities with significant overlaps in networks: Partial community merger algorithm
A Partial Community Merger Algorithm (PCMA) is developed for detecting communities with significant overlaps as well as slightly overlapping and disjoint ones and it is found that the significant overlapping pattern found in the detected communities confirms the need of new algorithms to handle multiple memberships of vertices in social networks.
Ensemble-Based Discovery of Disjoint, Overlapping and Fuzzy Community Structures in Networks
This paper proposes two ensemble methods: ENDISCO and MEDOC, which perform disjoint, overlapping, and fuzzy community detection and represents the first ever ensemble method for fuzzy and overlapping community detection.
Ensemble Detection and Analysis of Communities in Complex Networks
MeDOF, an ensemble method which performs disjoint, overlapping, and fuzzy community detection and represents one of the first ever ensemble methods for fuzzy and overlapping community detection, is proposed and beats the best-known existing stand-alone community detection algorithms.
Community Detection in Multiplex Networks
A taxonomy of community detection algorithms in multiplex networks is provided to help scholars and practitioners to choose the right methods for the data and the task at hand, while also emphasizing when such choice is problematic.
Unsupervised Abstractive Meeting Summarization with Multi-Sentence Compression and Budgeted Submodular Maximization
A novel graph-based framework for abstractive meeting speech summarization that is fully unsupervised and does not rely on any annotations to take exterior semantic knowledge into account, and to design custom diversity and informativeness measures.
A Template-based Abstractive Meeting Summarization: Leveraging Summary and Source Text Relationships
This system extends a novel multi-sentence fusion algorithm in order to generate abstract templates and leverages the relationship between summaries and their source meeting transcripts to select the best templates for generating abstractive summaries of meetings.


Experiments in Single and Multi-Document Summarization Using MEAD
This paper describes four experiments in text summarization about the identification by human subjects of cross-document structural relationships such as identity, paraphrase, elaboration, and fulfillment and presents numerical evaluations of all four experiments.
The use of MMR, diversity-based reranking for reordering documents and producing summaries
This paper presents a method for combining query-relevance with information-novelty in the context of text retrieval and summarization, and preliminary results indicate some benefits for MMR diversity ranking in document retrieval and in single document summarization.
Exploiting Conversation Structure in Unsupervised Topic Segmentation for Emails
Empirical evaluation shows that LCSeg is a better model than LDA for segmenting an email thread into topical clusters and incorporating conversation structure into these models improves the performance significantly.
Community structure in social and biological networks
  • M. Girvan, M. Newman
  • Computer Science
    Proceedings of the National Academy of Sciences of the United States of America
  • 2002
This article proposes a method for detecting communities, built around the idea of using centrality indices to find community boundaries, and tests it on computer-generated and real-world graphs whose community structure is already known and finds that the method detects this known structure with high sensitivity and reliability.
A Fast Algorithm to Find Overlapping Communities in Networks
This work presents an improved algorithm, CONGA, that could detect overlapping communities using the new concept of split betweenness, which is especially effective in discovering small-diameter communities in large networks, and has a time complexity of only O(nlog n) for sparse networks.
An Algorithm to Find Overlapping Community Structure in Networks
A new algorithm for discovering overlapping communities in networks is presented, by extending Girvan and Newman's well-known algorithm based on the betweennesscentrality measure, which performs hierarchical clustering -- partitioning a network into any desired number of clusters -- but allows them to overlap.
Generating and Validating Abstracts of Meeting Conversations: a User Study
It is found that the automatically generated summaries are ranked significantly higher than human-selected extracts on coherence and usability criteria, and more generally, users demonstrate a strong preference for abstract-style summaries over extracts.
Sentence Fusion for Multidocument News Summarization
This article introduces sentence fusion, a novel text-to-text generation technique for synthesizing common information across documents that moves the summarization field from the use of purely extractive methods to the generation of abstracts that contain sentences not found in any of the input documents and can synthesize information across sources.
Objective Criteria for the Evaluation of Clustering Methods
This article proposes several criteria which isolate specific aspects of the performance of a method, such as its retrieval of inherent structure, its sensitivity to resampling and the stability of its results in the light of new data.
Automatic Summarization
Experimental results show that the proposed automatic speech summarization technique for English effectively extracts relatively important information and remove redundant and irrelevant information from English news speech.