Semantic Model Vectors for Complex Video Event Recognition


We propose Semantic Model Vectors, an intermediate level semantic representation, as a basis for modeling and detecting complex events in unconstrained real-world videos, such as those from YouTube. The Semantic Model Vectors are extracted using a set of discriminative semantic classifiers, each being an ensemble of SVM models trained from thousands of labeled web images, for a total of 280 generic concepts. Our study reveals that the proposed Semantic Model Vectors representation outperforms—and is complementary to—other low-level visual descriptors for video event modeling. We hence present an endto-end video event detection system, which combines Semantic Model Vectors with other static or dynamic visual descriptors, extracted at the frame, segment or full clip level. We perform a comprehensive empirical study on the 2010 TRECVID Multimedia Event Detection task, which validates the Semantic Model Vectors representation not only as the best individual descriptor, outperforming state-of-the-art global and local static features as well as spatio-temporal HOG and HOF descriptors, but also as the most compact. We also study early and late feature fusion across the various approaches, leading to a 15% performance boost and an overall system performance of 0.46 Mean Average Precision. In order to promote further research in this direction, we made our Semantic Model Vectors for the TRECVID MED 2010 set publicly available for the community to use.

DOI: 10.1109/TMM.2011.2168948

Extracted Key Phrases

12 Figures and Tables

Citations per Year

135 Citations

Semantic Scholar estimates that this publication has 135 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@article{Merler2012SemanticMV, title={Semantic Model Vectors for Complex Video Event Recognition}, author={Michele Merler and Bert Huang and Lexing Xie and Gang Hua and Apostol Natsev}, journal={IEEE Trans. Multimedia}, year={2012}, volume={14}, pages={88-101} }