Rethinking Summarization and Storytelling for Modern Social Multimedia

@inproceedings{Rudinac2018RethinkingSA,
  title={Rethinking Summarization and Storytelling for Modern Social Multimedia},
  author={Stevan Rudinac and Tat-Seng Chua and Nicol{\'a}s Emilio D{\'i}az Ferreyra and Gerald Friedland and Tatjana Gornostaja and Benoit Huet and Rianne Kaptein and Krister Lind{\'e}n and Marie-Francine Moens and Jaakko Peltonen and Miriam Redi and Markus Schedl and David A. Shamma and Alan F. Smeaton and Lexing Xie},
  booktitle={MMM},
  year={2018}
}
Traditional summarization initiatives have been focused on specific types of documents such as articles, reviews, videos, image feeds, or tweets, a practice which may result in pigeonholing the summarization task in the context of modern, content-rich multimedia collections. Consequently, much of the research to date has revolved around mostly toy problems in narrow domains and working on single-source media types. We argue that summarization and story generation systems need to refocus the… 

Deep Multimodal Features for Movie Genre and Interestingness Prediction

  • O. B. AhmedB. Huet
  • Computer Science
    2018 International Conference on Content-Based Multimedia Indexing (CBMI)
  • 2018
TLDR
It is demonstrated that the proposed multimodal framework for video segment interestingness prediction based on the genre and affective impact of movie content outperforms the existing video interestingness approaches on the PMIT dataset in terms of Mean Average Precision.

Interactive Search and Exploration in Social Multimedia Networks

TLDR
This thesis presents a novel interactive multimodal learning system which facilitates search and exploration in large networks of social multimedia users and demonstrates the capabilities of the proposed system on two different multimedia collections originating from the violent online extremism forum Stormfront and the microblogging platform Twitter.

Interactive Search and Exploration in Online Discussion Forums Using Multimodal Embeddings

TLDR
A novel interactive multimodal learning system, which facilitates search and exploration in large networks of social multimedia users and allows the analyst to identify and select users of interest, and to find similar users in an interactive learning setting, is presented.

TIPS: A Framework for Text Summarising with Illustrative Pictures

TLDR
The use of TIPS resulted in an increase in matching highly relevant images to text, ranging from 5% to 8% and images relevant to text ranging from 3% to 7% compared to the approach based on single-embedding schema.

Exquisitor: Interactive Learning at Large

TLDR
Exquisitor is a highly scalable interactive learning approach, capable of intelligent exploration of the large-scale YFCC100M image collection with extremely efficient responses from the interactive classifier.

Challenges Associated with Generative Forms of Multimedia Content (Keynote Talk)

TLDR
The paper introduces GANs, outlines how they are used to generate visual media and summarises this major challenge, which is around automatically determining the authenticity of an image/video.

Multimodal Classification of Urban Micro-Events

TLDR
This paper explores several methods of creating a classifier for urban micro-events, including early, late and hybrid fusion as well as representation learning using multimodal graphs and shows that a multi-modal approach yields higher performance than unimodal alternatives.

UvA-DARE (Digital Academic Repository) Multimodal Classification of Urban Micro-Events

TLDR
This paper explores several methods of creating a classifier capable of combining early, late and hybrid fusion as well as representation learning using multimodal graphs, and shows that a multi-modal approach yields higher performance than unimodal alternatives.

References

SHOWING 1-10 OF 56 REFERENCES

Multimedia Summarization for Social Events in Microblog Stream

TLDR
This paper proposes a multimedia social event summarization framework to automatically generate visualized summaries from the microblog stream of multiple media types and conducts extensive experiments on two real-world microblog datasets to demonstrate the superiority of the proposed framework as compared to the state-of-the-art approaches.

Story-Driven Summarization for Egocentric Video

  • Zheng LuK. Grauman
  • Computer Science
    2013 IEEE Conference on Computer Vision and Pattern Recognition
  • 2013
TLDR
A video summarization approach that discovers the story of an egocentric video, and defines a random-walk based metric of influence between sub shots that reflects how visual objects contribute to the progression of events.

Tweet the debates: understanding community annotation of uncollected sources

TLDR
It is found that the level of Twitter activity serves as a predictor of changes in topics in the media event and conversational cues can identify the key players in theMedia object and that the content of the Twitter posts can somewhat reflect the topics of discussion in the Media object, but are mostly evaluative, in that they express the poster's reaction to the media.

Less talk, more rock: automated organization of community-contributed collections of concert videos

TLDR
The timing and link structure generated by the synchronization algorithm is used to improve the findability and representation of the event content, including identifying key moments of interest and descriptive text for important captured segments of the show.

A Survey of Text Summarization Techniques

TLDR
This chapter gives a broad overview of existing approaches based on how representation, sentence scoring or summary selection strategies alter the overall performance of the summarizer, and points out some of the peculiarities of the task of summarization.

Learning Crowdsourced User Preferences for Visual Summarization of Image Collections

TLDR
The phenomenon of a low inter-user agreement, which makes an automatic evaluation of visual summaries a challenging task and a solution inspired by the text summarization and machine translation communities are discussed.

Video Manga: generating semantically meaningful video summaries

TLDR
Methods for automatically creating pictorial video summaries that resemble comic books are presented and how the automatically generated summaries are used to simplify access to a large collection of videos is described.

Personalized time-aware tweets summarization

TLDR
A time-aware user behavior model is proposed, the Tweet Propagation Model (TPM), in which dynamic probabilistic distributions over interests and topics are inferred and an iterative optimization algorithm for selecting tweets is proposed.

Intelligent Access to Digital Video: Informedia Project

TLDR
Carnegie Mellon's Informedia Digital Video Library project will establish a large, on-line digital video library featuring full-content and knowledge-based search and retrieval, and focused the work on two corpuses.

VERT: automatic evaluation of video summaries

TLDR
The VERT (Video Evaluation by Relevant Threshold) algorithm is proposed to automatically evaluate the quality of video summaries, and mimics the theories of BLEU and ROUGE, and counts the weighted number of overlapping selected units between the computer-generated video summary and several human-made references.
...