In-Place Gestures Classification via Long-term Memory Augmented Network

  title={In-Place Gestures Classification via Long-term Memory Augmented Network},
  author={Lizhi Zhao and Xuequan Lu and Qianyue Bao and Meili Wang},
In-place gesture-based virtual locomotion techniques enable users to control their viewpoint and intuitively move in the 3D virtual environment. A key research problem is to accurately and quickly recognize in-place gestures, since they can trigger specific movements of virtual viewpoints and enhance user experience. However, to achieve real-time experience, only short-term sensor sequence data (up to about 300ms, 6 to 10 frames) can be taken as input, which actually affects the classification… 

Figures and Tables from this paper



Classifying In-Place Gestures with End-to-End Point Cloud Learning

An end-to-end joint framework involving both a supervised loss for supervised point learning and an unsupervised loss for un supervised domain adaptation for accurate and efficient classification of in-place gestures is proposed.

Video Prediction Recalling Long-term Motion Context via Memory Alignment Learning

Experimental results show that the proposed method outperforms other sophisticated RNN-based methods, especially in long-term condition, and the effectiveness of the proposed network designs is validated by conducting ablation studies and memory feature analysis.

Accurate and Fast Classification of Foot Gestures for Virtual Locomotion

A novel method, Dual-Check Till Consensus, is proposed to reduce the latency of gesture recognition from 2 seconds to 0.5 seconds and increase the accuracy to over 97%.

Video-based Person Re-identification with Spatial and Temporal Memory Networks

A novel Spatial and Temporal Memory Networks (STMN) is introduced, where the spatial memory stores features for spatial distractors that frequently emerge across video frames, while the temporal memory saves attentions which are optimized for typical temporal patterns in person videos.

I Feel More Engaged When I Move!: Deep Learning-based Backward Movement Detection and its Application

The development of a prediction model for forward/backward movement while considering a user's orientation and the verification of the model's effectiveness is presented, suggesting its potential for use in many VR applications.

MANTRA: Memory Augmented Networks for Multiple Trajectory Prediction

The method is able to natively perform multi-modal trajectory prediction obtaining state-of-the art results on three datasets and can continuously improve by ingesting novel patterns thanks to the non-parametric nature of the memory module.

Constructing Stronger and Faster Baselines for Skeleton-based Action Recognition

This work designs an efficient Graph Convolutional Network baseline for skeleton-based action recognition, and designs a compound scaling strategy to expand the model's width and depth synchronously, and obtains a family of efficient GCN baselines with high accuracies and small amounts of trainable parameters, termed EfficientGCN-Bx.

NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis

A large-scale dataset for RGB+D human action recognition with more than 56 thousand video samples and 4 million frames, collected from 40 distinct subjects is introduced and a new recurrent neural network structure is proposed to model the long-term temporal correlation of the features for each body part, and utilize them for better action classification.

History Repeats Itself: Human Motion Prediction via Motion Attention

An attention-based feed-forward network is introduced that explicitly leverages the observation that human motion tends to repeat itself to capture motion attention to capture the similarity between the current motion context and the historical motion sub-sequences.

Memory Enhanced Global-Local Aggregation for Video Object Detection

This paper introduces memory enhanced global-local aggregation (MEGA) network, which is among the first trials that takes full consideration of both global and local information.