Multimodal Video-to-Video Linking: Turning to the Crowd for Insight and Evaluation

@inproceedings{Eskevich2017MultimodalVL,
  title={Multimodal Video-to-Video Linking: Turning to the Crowd for Insight and Evaluation},
  author={Maria Eskevich and Martha Larson and Robin Aly and Serwah Sabetghadam and G. Jones and Roeland Ordelman and Benoit Huet},
  booktitle={Conference on Multimedia Modeling},
  year={2017}
}
Video-to-video linking systems allow users to explore and exploit the content of a large-scale multimedia collection interactively and without the need to formulate specific queries. We present a short introduction to video-to-video linking (also called ‘video hyperlinking’), and describe the latest edition of the Video Hyperlinking (LNK) task at TRECVid 2016. The emphasis of the LNK task in 2016 is on multimodality as used by videomakers to communicate their intended message. Crowdsourcing… 
View via Publisher
eprints.eemcs.utwente.nl
9 Citations
Highly Influential Citations
1
Background Citations
3
Methods Citations
1

9 Citations

Exploiting Multimodality in Video Hyperlinking to Improve Target Diversity

This paper compares two recently introduced cross-modal approaches, namely, deep auto-encoders and bimodal LDA, and experimentally shows that both provide significantly more diverse targets than a state-of-the-art baseline.

Linking segments of video using text-based methods and a flexible form of segmentation : How to index, query and re-rank data from the TRECVid (Blip.tv) dataset?

The payload of terms in the form of position and offset in Elastic Search are used to obtain time-based information along the speech transcripts to link users directly to spoken text and show that TF-IDF and the cosine similarity work the best for the proposed system.

IRISA at TrecVid 2017: Beyond Crossmodal and Multimodal Models for Video Hyperlinking

The runs that were submitted to the TRECVid Challenge 2017 for the Video Hyperlinking task show a gain in performance over the baseline BiDNN model both when the metadata filter was used and when the keyframe fusion was done with a pseudo-inverse.

On the Selection of Anchors and Targets for Video Hyperlinking

Insight is provided from the perspective of hubness and local intrinsic dimensionality, which are two statistical properties in assessing the popularity and complexity of data space and two novel algorithms are proposed for low-risk automatic selection of anchors and targets.

OffVid: A System for Linking Off-Topic Concepts to Topically Relevant Video Lecture Segments

A system for automatically connecting off-topic concepts from a video lecture to appropriate and topically relevant video lecture segments and user study on the quality of recommendation has been found to be promising.

TRECVID 2016: Evaluating Video Search, Video Event Detection, Localization, and Hyperlinking

TRECVID 2016: Evaluating Video Search, Video Event Detection, Localization, and Hyperlinking George Awad, Jonathan Fiscus, David Joy, Martial Michel, Alan Smeaton, Wessel Kraaij, Maria Eskevich, Robin Aly, Roeland Ordelman, Marc Ritter, et al.

Deep neural architectures for automatic representation learning from multimedia multimodal data. (Architectures neuronales profondes pour l'apprentissage de représentations multimodales de données multimédias)

The thesis that deep neural networks are suited for analysis of visual, textual and fused visual and textual content is discussed and an architecture that allow us to predict human actions from a single image is proposed.

EURECOM at TRECVID 2016: The Adhoc Video Search and Video Hyperlinking Tasks

This paper describes the submissions of the EURECO M team to the TRECVID 2016 AVS and LNK tasks.

Hypergraphes multimédias dirigés navigables, construction et exploitation

Cette these en informatique s’interesse a la structuration et a l’exploration de collections journalistiques. Elle fait appel a plusieurs domaines de recherches : sciences sociales, a travers l’etude…

References

SHOWING 1-10 OF 13 REFERENCES

Overview of VideoCLEF 2009: New Perspectives on Speech-based Multimedia Content Enrichment

VideoCLEF 2009 offered three tasks related to enriching video content for improved multimedia access in a multilingual environment, involving automatic tagging of videos with subject theme labels and linking video to material on the same subject in a different language.

TRECVID 2016: Evaluating Video Search, Video Event Detection, Localization, and Hyperlinking

TRECVID 2016: Evaluating Video Search, Video Event Detection, Localization, and Hyperlinking George Awad, Jonathan Fiscus, David Joy, Martial Michel, Alan Smeaton, Wessel Kraaij, Maria Eskevich, Robin Aly, Roeland Ordelman, Marc Ritter, et al.

Creating a Data Collection for Evaluating Rich Speech Retrieval

This collection focuses on satisfying user information needs for queries associated with specific types of speech acts, based on an archive of the Internet video from Internet video sharing platform (blip.tv), provided by the MediaEval benchmarking initiative.

Blip10000: a social video dataset containing SPUG content for tagging and retrieval

This work presents a dataset that contains comprehensive semi-professional user-generated (SPUG) content, including audiovisual content, user-contributed metadata, automatic speech recognition transcripts, automatic shot boundary files, and social information for multiple 'social levels'.

The Search and Hyperlinking Task at MediaEval 2013

The method for adjustment of the jump-in points achieves higher scores for all LIMSI/Vocapia, LIUM, and subtitles based runs.

Linking Archives Using Document Enrichment and Term Selection

It is found that the difference in textual richness of annotations presents a challenge and two approaches are investigated: to enrich sparsely annotated items with textually rich content; and to reduce rich news archive items using term selection.

Feature-based video key frame extraction for low quality video sequences

We present an approach to key frame extraction for structuring user generated videos on video sharing websites (e. g. YouTube). Our approach is intended to link existing image search engines to video…

Search and Hyperlinking Task at MediaEval 2012

The Search and Hyperlinking Task was one of the Brave New Tasks at MediaEval 2012. The Task consisted of two subtasks which focused on search and linking in retrieval from a collection of…

Learning to link with wikipedia

This paper explains how machine learning can be used to identify significant terms within unstructured text, and enrich it with links to the appropriate Wikipedia articles, and performs very well, with recall and precision of almost 75%.

Wikify!: linking documents to encyclopedic knowledge

This paper introduces the use of Wikipedia as a resource for automatic keyword extraction and word sense disambiguation, and shows how this online encyclopedia can be used to achieve state-of-the-art…