Multimodal feature fusion for CNN-based gait recognition: an empirical comparison

@article{Castro2020MultimodalFF,
  title={Multimodal feature fusion for CNN-based gait recognition: an empirical comparison},
  author={Francisco Manuel Castro and Manuel J. Mar{\'i}n-Jim{\'e}nez and Nicol{\'a}s Guil Mata and Nicol{\'a}s P{\'e}rez de la Blanca},
  journal={Neural Computing and Applications},
  year={2020},
  pages={1-21}
}
People identification in video based on the way they walk (i.e. gait) is a relevant task in computer vision using a non-invasive approach. [...] Key Method Therefore, we present a comparative study of different Convolutional Neural Network (CNN) architectures on three low-level features (i.e. gray pixels, optical flow channels and depth maps) on two widely-adopted and challenging datasets: TUM-GAID and CASIA-B. In addition, we perform a comparative study between different early and late fusion methods used to…Expand
Review of gait recognition approaches and their challenges on view changes
TLDR
The existing methods addressing one of the most important real-world challenges of gait recognition, i.e. view changes, are emphasised and summarised in this study. Expand
Gait Lateral Network: Learning Discriminative and Compact Representations for Gait Recognition
TLDR
This work proposes a novel network named Gait Lateral Network (GLN) which can learn both discriminative and compact representations from the silhouettes for gait recognition and leverages the inherent feature pyramid in deep convolutional neural networks to enhance the gait representations. Expand
Multimodal Gait Recognition for Neurodegenerative Diseases
TLDR
A novel hybrid model is proposed to learn the gait differences between three neurodegenerative diseases, between patients with different severity levels of Parkinson's disease, and between healthy individuals and patients, by fusing and aggregating data from multiple sensors. Expand
MuPeG—The Multiple Person Gait Framework
TLDR
A new framework (MuPeG) is released that generates augmented datasets with multiple subjects using existing datasets as input and is able to generate useful datasets withmultiple subjects which are more similar to real life situations. Expand
iLGaCo: Incremental Learning of Gait Covariate Factors
TLDR
iLGaCo is proposed, the first incremental learning approach of covariate factors for gait recognition, where the deep model can be updated with new information without re-training it from scratch by using the whole dataset. Expand
Deep Residual Matrix Factorization for Gait Recognition
TLDR
A deep dimension reduction framework named Deep Residual 2D PCA (DR-2DPCA) for gait recognition is proposed and experiments on the CASIA-B gait dataset illustrates that the proposed algorithm outperforms the existing ones using traditional matrix factorization techniques. Expand
A Survey of Behavioral Biometric Gait Recognition: Current Success and Future Perspectives
TLDR
This article extensively investigates feature representation techniques, classified into model-based and model-free, and proposes future perspectives after investigating state-of-art literature that can be more helpful to experts and new comers in gait recognition. Expand
RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild
TLDR
A trained ConvNet model, coined RealHePoNet, is obtained that given a low-resolution grayscale input image, and without the need of using facial landmarks, is able to estimate with low error both tilt and pan angles. Expand
Improved Deep Learning Technique to Detect Freezing of Gait in Parkinson’s Disease Based on Wearable Sensors
TLDR
A novel FOG detection system that takes multi-channel acceleration signals as input, uses one-dimensional deep convolutional neural network to automatically learn feature representations, and uses recurrent neuralnetwork to model the temporal dependencies between feature activations to indicate that it has high operating efficiency and excellent detection performance. Expand
Reduction of covariate factors from Silhouette image for robust gait recognition
  • Sanjay Kumar Gupta
  • Computer Science
  • Multimedia Tools and Applications
  • 2021
TLDR
The proposed approach to gait recognition is more accurate after the omission of covariate objects from silhouette images, and a new gait invariant feature is named as Covariate Factors Omitted Silhouette Image (CFOSI). Expand
...
1
2
...

References

SHOWING 1-10 OF 92 REFERENCES
Evaluation of Cnn Architectures for Gait Recognition Based on Optical Flow Maps
TLDR
This work targets people identification in video based on the way they walk by using deep learning architectures and shows that even the simplest CNN models greatly improve the results using shallow classifiers. Expand
Multimodal features fusion for gait, gender and shoes recognition
TLDR
The experimental results show that using either early or late fusion techniques to combine feature descriptors from three kinds of modalities improves the state-of-the-art results on the standard experiments defined on the dataset for the tasks of gait, gender and shoes recognition. Expand
Empirical Study of Audio-Visual Features Fusion for Gait Recognition
TLDR
The experimental results show that using late fusion to combine two kinds of tracklet-based visual features with audio features improves the state-of-the-art results on the standard experiments defined on the dataset. Expand
A Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs
TLDR
Experimental results show that this first work based on deep CNNs for gait recognition in the literature outperforms the previous state-of-the-art methods by a significant margin, and shows great potential for practical applications. Expand
A Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs.
TLDR
Experimental results show that this first work based on deep CNNs for gait recognition in the literature outperforms the previous state-of-the-art methods by a significant margin, and shows great potential for practical applications. Expand
Automatic Learning of Gait Signatures for People Identification
TLDR
The experimental results indicate that using spatio-temporal cuboids of optical flow as input data for CNN allows to obtain state-of-the-art results on the gait task with an image resolution eight times lower than the previously reported results. Expand
Improved Gait recognition based on specialized deep convolutional neural networks
TLDR
A specialized deep CNN architecture, which consists of multilayers of convolutional and subsampling layers is developed, which outperforms the other state of art gait recognition techniques in several cases. Expand
Improved gait recognition based on specialized deep convolutional neural network
TLDR
A specialized deep convolutional neural network architecture for gait recognition that is less sensitive to several cases of the common variations and occlusions that affect and degrade gait Recognition performance. Expand
An End-to-End Multi-Task and Fusion CNN for Inertial-Based Gait Recognition
TLDR
This work presents a new deep learning-based end-to-end approach that employs raw inertial data as input and suggests that the use of hand-crafted features is not necessary for this task as deep learning approaches using raw data achieve better results. Expand
Fisher Motion Descriptor for Multiview Gait Recognition
TLDR
The goal of this paper is to identify individuals by analyzing their gait by proposing and evaluating the use of motion descriptors based on densely sampled short-term trajectories, and taking advantage of state-of-the-art people detectors to define custom spatial configurations of the descriptors around the target person. Expand
...
1
2
3
4
5
...