• Corpus ID: 209324730

# Developing the Path Signature Methodology and its Application to Landmark-based Human Action Recognition

@article{Yang2017DevelopingTP,
title={Developing the Path Signature Methodology and its Application to Landmark-based Human Action Recognition},
author={Weixin Yang and Terry Lyons and Hao Ni and Cordelia Schmid and Lianwen Jin},
journal={arXiv: Computer Vision and Pattern Recognition},
year={2017}
}
• Published 13 July 2017
• Computer Science
• arXiv: Computer Vision and Pattern Recognition
Landmark-based human action recognition in videos is a challenging task in computer vision. One key step is to design a generic approach that generates discriminative features for the spatial structure and temporal dynamics. To this end, we regard the evolving landmark data as a high-dimensional path and apply non-linear path signature techniques to provide an expressive, robust, non-linear, and interpretable representation for the sequential events. We do not extract signature features from…

## Figures and Tables from this paper

### Skeleton-based Gesture Recognition Using Several Fully Connected Layers with Path Signature Features and Temporal Transformer Module

• Computer Science
AAAI
• 2019
A robust feature descriptor, path signature (PS), is leveraged, and three PS features are proposed to explicitly represent the spatial and temporal motion characteristics, i.e., spatial PS (S_PS), temporal PS (T_PS) and temporal spatial PS(T_S-PS), which achieves the state-of-the-art performance on skeleton-based gesture recognition with high computational efficiency.

### ImageSig: A signature transform for ultra-lightweight image recognition

• Computer Science
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
• 2022
A new lightweight method for image recognition based on computing signatures and does not require a convolutional structure or an attention-based encoder that achieves an accuracy for 64 X 64 RGB images that exceeds many of the state-of-the-art methods and simultaneously requires orders of magnitude less FLOPS, power and memory footprint.

### Neural-Signature Methods for Structured EHR Prediction

• Computer Science
• 2022
This first application of neural-signature methods in real-world healthcare data shows a competitive performance when compared to strong baselines and thus warrants further investigation within the health domain.

### Path Signature Neural Network of Cortical Features for Prediction of Infant Cognitive Scores

• Computer Science
IEEE Transactions on Medical Imaging
• 2022
A novel cortical feature based path signature neural network (CF-PSNet) is proposed with stacked differentiable temporal path signature layers for prediction of individual cognitive scores, and the existence embedding in path generation can improve the robustness against the missing data.

### The Signature Kernel Is the Solution of a Goursat PDE

• Computer Science
SIAM J. Math. Data Sci.
• 2021
It is shown that for continuously differentiable paths, the signature kernel solves a hyperbolic PDE and recognize the connection with a well known class of differential equations known in the literature as Goursat problems.

### Deep Learning for Estimating Synaptic Health of Primary Neuronal Cell Culture

• Biology
ArXiv
• 2019
The image recognition model which is based on deep convolutional neural network (CNN) architecture with residual connections achieved accuracy of 99.6% on a binary classification task of distinguishing untreated and treated rodent primary neuronal cells with Amyloid-$\beta_{(25-35)}$.

### Learning stochastic differential equations using RNN with log signature features

• Computer Science
ArXiv
• 2019
A hybrid Logsig-RNN algorithm that learns functionals on streamed data with outstanding accuracy with superior efficiency and robustness is proposed.

### Offline Writer Identification Based on the Path Signature Feature

• Computer Science
2019 International Conference on Document Analysis and Recognition (ICDAR)
• 2019
A codebook method based on the log path signature—a more compact way to express the path signature)—is used in this work and shows competitive results on several benchmark offline writer identification datasets, namely the IAM, Firemaker, CVL and ICDAR2013 writer identification contest dataset.

## References

SHOWING 1-10 OF 98 REFERENCES

### Rotation-free online handwritten character recognition using dyadic path signature features, hanging normalization, and deep neural network

• Computer Science
2016 23rd International Conference on Pattern Recognition (ICPR)
• 2016
This paper proposes a variation of path signature representation, namely the dyadic path signature feature (D-PSF), to fully characterize the trajectory using a hierarchical structure to solve the rotation-free online handwritten character recognition (OLHCR) problem.

### P-CNN: Pose-Based CNN Features for Action Recognition

• Computer Science
2015 IEEE International Conference on Computer Vision (ICCV)
• 2015
A new Pose-based Convolutional Neural Network descriptor (P-CNN) for action recognition that aggregates motion and appearance information along tracks of human body parts is proposed.

### INTEGRATION OF PATHS—A FAITHFUL REPRE- SENTATION OF PATHS BY NONCOMMUTATIVE FORMAL POWER SERIES

in the noncommutative indeterminates X1, , Xm. The main result of this paper is briefly as follows: If 0(oa) =0() for two irreducible piecewise C' and continuous curves oa and f, then f can be

### Action recognition with novel high-level pose features

• Computer Science
2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)
• 2016
A set of novel high-level pose features (NHLPF) that combine the spatial and temporal information, and a joint energy change feature, which is designed using observations of the magnitude and direction of the force between joints, are presented.

### NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis

• Computer Science
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
• 2016
A large-scale dataset for RGB+D human action recognition with more than 56 thousand video samples and 4 million frames, collected from 40 distinct subjects is introduced and a new recurrent neural network structure is proposed to model the long-term temporal correlation of the features for each body part, and utilize them for better action classification.

### Berkeley MHAD: A comprehensive Multimodal Human Action Database

• Computer Science
2013 IEEE Workshop on Applications of Computer Vision (WACV)
• 2013
A controlled multimodal dataset consisting of temporally synchronized and geometrically calibrated data from an optical motion capture system, multi-baseline stereo cameras from multiple views, depth sensors, accelerometers and microphones, provides researchers an inclusive testbed to develop and benchmark new algorithms across multiple modalities under known capture conditions in various research domains.

### Action bank: A high-level representation of activity in video

• Computer Science
2012 IEEE Conference on Computer Vision and Pattern Recognition
• 2012
Inspired by the recent object bank approach to image representation, Action Bank is presented, a new high-level representation of video comprised of many individual action detectors sampled broadly in semantic space as well as viewpoint space that is capable of highly discriminative performance.

### Two-person interaction detection using body-pose features and multiple instance learning

• Computer Science
2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
• 2012
A complex human activity dataset depicting two person interactions, including synchronized video, depth and motion capture data is created, and techniques related to Multiple Instance Learning (MIL) are explored, finding that the MIL based classifier outperforms SVMs when the sequences extend temporally around the interaction of interest.

### Di erential equations driven by rough signals

This paper aims to provide a systematic approach to the treatment of differential equations of the type dyt = Si fi(yt) dxti where the driving signal xt is a rough path. Such equations are very

### Towards Understanding Action Recognition

• Computer Science
2013 IEEE International Conference on Computer Vision
• 2013
It is found that high-level pose features greatly outperform low/mid level features, in particular, pose over time is critical, but current pose estimation algorithms are not yet reliable enough to provide this information.