• Corpus ID: 244117742

Metric-based multimodal meta-learning for human movement identification via footstep recognition

  title={Metric-based multimodal meta-learning for human movement identification via footstep recognition},
  author={Muhammad Shakeel and Katsutoshi Itoyama and Kenji Nishida and Kazuhiro Nakadai},
We describe a novel metric-based learning approach that introduces a multimodal framework and uses deep audio and geophone encoders in siamese configuration to design an adaptable and lightweight supervised model. This framework eliminates the need for expensive data labeling procedures and learns general-purpose representations from low multisensory data obtained from omnipresent sensing systems. These sensing systems provide numerous applications and various use cases in activity recognition… 

Figures and Tables from this paper


Multimodal Deep Learning for Activity and Context Recognition
This paper studies the benefits of adopting deep learning algorithms for interpreting user activity and context as captured by multi-sensor systems under wearable data by evaluating four variations of deep neural networks based either on fully-connected Deep Neural Networks (DNNs) or Convolutional Neural networks (CNNs).
Person Identification by Footstep Sound Using Convolutional Neural Networks
A method for person identification based on footstep sounds is proposed, and the Grad-CAM method is applied to visualize the background of its decision in order to verify the functionality of the proposed CNN.
Acoustic Gait-based Person Identification using Hidden Markov Models
A cyclic model topology is employed to represent individual gait cycles and this topology allows to model and detect individual steps, leading to very promising identification rates.
Siamese Style Convolutional Neural Networks for Sound Search by Vocal Imitation
Experimental results show that both versions of the proposed Siamese style convolutional neural networks outperform a state-of-the-art system for sound search by vocal imitation, and the performance can be further improved when they are fused with the state of the art system.
Few-Shot Siamese Neural Networks Employing Audio Features for Human-Fall Detection
A first study on few-shot learning Siamese Neural Network applied to human falls detection by using audio signals is shown and is able to learn the differences between signals belonging to different classes of events.
Decoding Brain Representations by Multimodal Learning of Neural Activity and Visual Features
This work proposes a model, EEG-ChannelNet, to learn a brain manifold for EEG classification and introduces a multimodal approach that uses deep image and EEG encoders, trained in a siamese configuration, for learning a joint manifold that maximizes a compatibility measure between visual features and brain representations.
Gait-based person identification by spectral, cepstral and energy-related audio features
This work addresses the problem of acoustic gait-based person identification by using a wrapper-based feature selection technique, which reduces the feature set while at the same time increasing the identification accuracy by 10% (relative).
Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification
It is shown that the improved performance stems from the combination of a deep, high-capacity model and an augmented training set: this combination outperforms both the proposed CNN without augmentation and a “shallow” dictionary learning model with augmentation.
Siamese Neural Networks for One-Shot Image Recognition
A method for learning siamese neural networks which employ a unique structure to naturally rank similarity between inputs and is able to achieve strong results which exceed those of other deep learning models with near state-of-the-art performance on one-shot classification tasks.
Silhouette Analysis-Based Gait Recognition for Human Identification
A simple but efficient gait recognition algorithm using spatial-temporal silhouette analysis is proposed that implicitly captures the structural and transitional characteristics of gait.