Multimodal Video-to-Video Linking: Turning to the Crowd for Insight and Evaluation

@inproceedings{Eskevich2017MultimodalVL,
  title={Multimodal Video-to-Video Linking: Turning to the Crowd for Insight and Evaluation},
  author={Maria Eskevich and Martha Larson and Robin Aly and Serwah Sabetghadam and G. Jones and Roeland Ordelman and Benoit Huet},
  booktitle={MMM},
  year={2017}
}
Video-to-video linking systems allow users to explore and exploit the content of a large-scale multimedia collection interactively and without the need to formulate specific queries. We present a short introduction to video-to-video linking (also called ‘video hyperlinking’), and describe the latest edition of the Video Hyperlinking (LNK) task at TRECVid 2016. The emphasis of the LNK task in 2016 is on multimodality as used by videomakers to communicate their intended message. Crowdsourcing… 
Linking segments of video using text-based methods and a flexible form of segmentation : How to index, query and re-rank data from the TRECVid (Blip.tv) dataset?
TLDR
The payload of terms in the form of position and offset in Elastic Search are used to obtain time-based information along the speech transcripts to link users directly to spoken text and show that TF-IDF and the cosine similarity work the best for the proposed system.
IRISA at TrecVid 2017: Beyond Crossmodal and Multimodal Models for Video Hyperlinking
TLDR
The runs that were submitted to the TRECVid Challenge 2017 for the Video Hyperlinking task show a gain in performance over the baseline BiDNN model both when the metadata filter was used and when the keyframe fusion was done with a pseudo-inverse.
On the Selection of Anchors and Targets for Video Hyperlinking
TLDR
Insight is provided from the perspective of hubness and local intrinsic dimensionality, which are two statistical properties in assessing the popularity and complexity of data space and two novel algorithms are proposed for low-risk automatic selection of anchors and targets.
OffVid: A System for Linking Off-Topic Concepts to Topically Relevant Video Lecture Segments
TLDR
A system for automatically connecting off-topic concepts from a video lecture to appropriate and topically relevant video lecture segments and user study on the quality of recommendation has been found to be promising.
TRECVID 2016: Evaluating Video Search, Video Event Detection, Localization, and Hyperlinking
TLDR
TRECVID 2016: Evaluating Video Search, Video Event Detection, Localization, and Hyperlinking George Awad, Jonathan Fiscus, David Joy, Martial Michel, Alan Smeaton, Wessel Kraaij, Maria Eskevich, Robin Aly, Roeland Ordelman, Marc Ritter, et al.
Deep neural architectures for automatic representation learning from multimedia multimodal data. (Architectures neuronales profondes pour l'apprentissage de représentations multimodales de données multimédias)
TLDR
The thesis that deep neural networks are suited for analysis of visual, textual and fused visual and textual content is discussed and an architecture that allow us to predict human actions from a single image is proposed.
EURECOM at TRECVID 2016: The Adhoc Video Search and Video Hyperlinking Tasks
This paper describes the submissions of the EURECO M team to the TRECVID 2016 AVS and LNK tasks.
Hypergraphes multimédias dirigés navigables, construction et exploitation
Cette these en informatique s’interesse a la structuration et a l’exploration de collections journalistiques. Elle fait appel a plusieurs domaines de recherches : sciences sociales, a travers l’etude
Exploiting Multimodality in Video Hyperlinking to Improve Target Diversity
TLDR
This paper compares two recently introduced cross-modal approaches, namely, deep auto-encoders and bimodal LDA, and experimentally shows that both provide significantly more diverse targets than a state-of-the-art baseline.

References

SHOWING 1-10 OF 13 REFERENCES
Overview of VideoCLEF 2009: New Perspectives on Speech-based Multimedia Content Enrichment
TLDR
VideoCLEF 2009 offered three tasks related to enriching video content for improved multimedia access in a multilingual environment, involving automatic tagging of videos with subject theme labels and linking video to material on the same subject in a different language.
TRECVID 2016: Evaluating Video Search, Video Event Detection, Localization, and Hyperlinking
TLDR
TRECVID 2016: Evaluating Video Search, Video Event Detection, Localization, and Hyperlinking George Awad, Jonathan Fiscus, David Joy, Martial Michel, Alan Smeaton, Wessel Kraaij, Maria Eskevich, Robin Aly, Roeland Ordelman, Marc Ritter, et al.
Creating a Data Collection for Evaluating Rich Speech Retrieval
TLDR
This collection focuses on satisfying user information needs for queries associated with specific types of speech acts, based on an archive of the Internet video from Internet video sharing platform (blip.tv), provided by the MediaEval benchmarking initiative.
Blip10000: a social video dataset containing SPUG content for tagging and retrieval
TLDR
This work presents a dataset that contains comprehensive semi-professional user-generated (SPUG) content, including audiovisual content, user-contributed metadata, automatic speech recognition transcripts, automatic shot boundary files, and social information for multiple 'social levels'.
The Search and Hyperlinking Task at MediaEval 2013
TLDR
The method for adjustment of the jump-in points achieves higher scores for all LIMSI/Vocapia, LIUM, and subtitles based runs.
Feature-based video key frame extraction for low quality video sequences
We present an approach to key frame extraction for structuring user generated videos on video sharing websites (e. g. YouTube). Our approach is intended to link existing image search engines to video
Search and Hyperlinking Task at MediaEval 2012
The Search and Hyperlinking Task was one of the Brave New Tasks at MediaEval 2012. The Task consisted of two subtasks which focused on search and linking in retrieval from a collection of
Learning to link with wikipedia
TLDR
This paper explains how machine learning can be used to identify significant terms within unstructured text, and enrich it with links to the appropriate Wikipedia articles, and performs very well, with recall and precision of almost 75%.
Wikify!: linking documents to encyclopedic knowledge
This paper introduces the use of Wikipedia as a resource for automatic keyword extraction and word sense disambiguation, and shows how this online encyclopedia can be used to achieve state-of-the-art
Multilingual Speech Processing Activities in Quaero: Application to Multimedia Search in Unstructured Data
Spoken language processing technologies are principle components in most of the applications being developed as part of the Quaero program. Quaero is a large research and industrial innovation
...
...