The Violent Scenes Detection task continues to pose challenge in detecting violent scenes in Hollywood movies. In this working notes paper, we present the framework of our system and briefly discuss the performance results obtained in both objective and subjective subtasks. Besides using the low-level features for training the SVM classifiers for violent scenes detection, we show the feasibility in using the concept detectors to infer the occurrence of violent scenes. External Youtube data is exploited in our implementation to provide more diverse definition to violent scene concepts. Furthermore, we explore the feasibility of using Conditional Random Fields (CRF) to refine the concept detection of movie shots holistically, given the relationships extracted from ConceptNet and the co-occurrence information defined by normalized Google distance (NGD). We demonstrate solid improvements in performance by using mid-level concept based detectors and CRF refinement in both objective and subjective subtasks.