Annotation Reranking, this combines multimodal features in the manner of cross reference. The fundamental idea of Annotation Reranking lies in the fact that the semantic understanding of video content from different modalities can reach an agreement. Actually, this idea is derived from the multi-view learning strategy. Multiview strategy has been successfully applied to various research fields, such as concept detection. However, this strategy, here, is utilized for inferring the most relevant shots in the initial search results, which is different from its original role. Annotation Reranking method contains three main stages: clustering the initial search results separately in diverse feature spaces, ranking the clusters by their relevance to the query, and hierarchically fusing all the ranked clusters using a cross-reference strategy. In our scheme, NCuts clustering algorithm is employed for clustering.