Corpus ID: 4435972

Who's Better, Who's Best: Skill Determination in Video using Deep Ranking

@article{Doughty2017WhosBW,
  title={Who's Better, Who's Best: Skill Determination in Video using Deep Ranking},
  author={Hazel Doughty and Dima Damen and W. Mayol-Cuevas},
  journal={ArXiv},
  year={2017},
  volume={abs/1703.09913}
}
This paper presents a method for assessing skill of performance from video, for a variety of tasks, ranging from drawing to surgery and rolling dough. We formulate the problem as pairwise and overall ranking of video collections, and propose a supervised deep ranking model to learn discriminative features between pairs of videos exhibiting different amounts of skill. We utilise a two-stream Temporal Segment Network to capture both the type and quality of motions and the evolving task state… Expand
Skill Determination from Egocentric Video
In this extended abstract we describe our work on determining relative skill from video [3]. We formulate the problem as pairwise and overall ranking of video collections, and propose a supervisedExpand
Group-aware Contrastive Regression for Action Quality Assessment
TLDR
It is shown that the relations among videos can provide important clues for more accurate action quality assessment during both training and inference, and a new Contrastive Regression framework is proposed to learn the relative scores by pair-wise comparison, which highlights the differences between videos and guides the models to learning the key hints for assessment. Expand
Temporal Distance Matrices for Squat Classification
TLDR
An action dataset of squats where different types of poor form have been annotated with a diversity of users and backgrounds is presented, and a model, based on temporal distance matrices, is proposed, which significantly outperforms existing approaches for the classification task. Expand
Action Assessment by Joint Relation Graphs
TLDR
This work presents a new model to assess the performance of actions from videos, through graph-based joint relation modelling, and proposes two novel modules, the Joint Commonality Module and the Joint Difference Module, for joint motion learning. Expand
Bidirectional long short-term memory for surgical skill classification of temporally segmented tasks
TLDR
Questions of the reliability of crowd sourced labels in videos of surgical tasks are brought about by a bidirectional long short-term memory model used with 8-s windows of multidimensional time-series data from the Basic Laparoscopic Urologic Skills dataset. Expand
Integration of Experts' and Beginners' Machine Operation Experiences to Obtain a Detailed Task Model
TLDR
A novel framework for integrating beginners’ machine operational experiences with those of experts’ to obtain a detailed task model is proposed using two aggregation steps that align and integrate sequences of interactions. Expand
Video-based surgical skill assessment using 3D convolutional neural networks
TLDR
The results demonstrate the feasibility of deep learning-based assessment of technical skill from surgical video, and the 3D ConvNet is able to learn meaningful patterns directly from the data, alleviating the need for manual feature engineering. Expand
An Asymmetric Modeling for Action Assessment
TLDR
This work proposes an asymmetric interaction module (AIM), to explicitly model asymmetric interactions between intelligent agents within an action, where they group these agents into a primary one and secondary ones (e.g., human and objects). Expand
Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX
TLDR
A representative-discriminative open-set recognition (RDOSR) framework, which projects data from the raw image space to the embedding feature space that facilitates differentiating similar classes, and further enhances both the representative and discriminative capacity through transformation to a so-called abundance space. Expand
Action Quality Assessment Across Multiple Actions
TLDR
The experimental results show that there is utility in learning a single model across multiple actions in the action quality assessment (AQA) setting. Expand
...
1
2
...

References

SHOWING 1-10 OF 30 REFERENCES
Assessing the Quality of Actions
TLDR
A learning-based framework that takes steps towards assessing how well people perform actions in videos by training a regression model from spatiotemporal pose features to scores obtained from expert judges and can provide interpretable feedback on how people can improve their action. Expand
Relative Hidden Markov Models for Video-Based Evaluation of Motion Skills in Surgical Training
  • Qiang Zhang, Baoxin Li
  • Computer Science, Medicine
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • 2015
TLDR
Experimental results suggest that the proposed approach provides a promising solution to video-based motion skill evaluation and the potential of generalizing the method to other applications of temporal analysis is illustrated. Expand
Pairwise Comparison-Based Objective Score for Automated Skill Assessment of Segments in a Surgical Task
TLDR
A framework to automatically generate an objective score for assessing skill in maneuvers within a surgical task was developed and evaluated and reliability analyses indicate that automated preference annotations by the classifier are consistent with those by the seven individuals. Expand
A study of crowdsourced segment-level surgical skill assessment using pairwise rankings
TLDR
The reliability and validity of a framework to generate objective skill assessments for segments within a task, and compared assessments from this framework using crowdsourced segment ratings from surgically untrained individuals and expert surgeons against manually assigned global rating scores, are investigated. Expand
Two-Stream Convolutional Networks for Action Recognition in Videos
TLDR
This work proposes a two-stream ConvNet architecture which incorporates spatial and temporal networks and demonstrates that a ConvNet trained on multi-frame dense optical flow is able to achieve very good performance in spite of limited training data. Expand
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Deep convolutional networks have achieved great success for visual recognition in still images. However, for action recognition in videos, the advantage over traditional methods is not so evident.Expand
Highlight Detection with Pairwise Deep Ranking for First-Person Video Summarization
  • Ting Yao, Tao Mei, Y. Rui
  • Computer Science
  • 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2016
TLDR
A novel pairwise deep ranking model that employs deep learning techniques to learn the relationship between high-light and non-highlight video segments is proposed and achieves the improvement over the state-of-the-art RankSVM method by 10.5% in terms of accuracy. Expand
HMDB: A large video database for human motion recognition
TLDR
This paper uses the largest action video database to-date with 51 action categories, which in total contain around 7,000 manually annotated clips extracted from a variety of sources ranging from digitized movies to YouTube, to evaluate the performance of two representative computer vision systems for action recognition and explore the robustness of these methods under various conditions. Expand
Human motion assessment in real time using recurrent self-organization
  • G. I. Parisi, S. Magg, S. Wermter
  • Computer Science
  • 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)
  • 2016
TLDR
A novel recursive neural network is proposed that uses growing self-organization for the efficient learning of body motion sequences and provides visual assistance to the person performing an exercise by displaying real-time feedback, thus enabling the user to correct inaccurate postures and motion intensity. Expand
Video-based motion expertise analysis in simulation-based surgical training using hierarchical dirichlet process hidden markov model
  • Qiang Zhang, Baoxin Li
  • Computer Science
  • MMAR '11
  • 2011
TLDR
The Hierarchical Dirichlet Process Hidden Markov Model is extended for simulation-based surgical training and provides some important cues on the patterns of motion for each expertise level. Expand
...
1
2
3
...