• Corpus ID: 209324730

Developing the Path Signature Methodology and its Application to Landmark-based Human Action Recognition

  title={Developing the Path Signature Methodology and its Application to Landmark-based Human Action Recognition},
  author={Weixin Yang and Terry Lyons and Hao Ni and Cordelia Schmid and Lianwen Jin},
  journal={arXiv: Computer Vision and Pattern Recognition},
Landmark-based human action recognition in videos is a challenging task in computer vision. One key step is to design a generic approach that generates discriminative features for the spatial structure and temporal dynamics. To this end, we regard the evolving landmark data as a high-dimensional path and apply non-linear path signature techniques to provide an expressive, robust, non-linear, and interpretable representation for the sequential events. We do not extract signature features from… 

Skeleton-based Gesture Recognition Using Several Fully Connected Layers with Path Signature Features and Temporal Transformer Module

A robust feature descriptor, path signature (PS), is leveraged, and three PS features are proposed to explicitly represent the spatial and temporal motion characteristics, i.e., spatial PS (S_PS), temporal PS (T_PS) and temporal spatial PS(T_S-PS), which achieves the state-of-the-art performance on skeleton-based gesture recognition with high computational efficiency.

Embedding and learning with signatures

ImageSig: A signature transform for ultra-lightweight image recognition

  • Mohamed IbrahimTerry Lyons
  • Computer Science
    2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
  • 2022
A new lightweight method for image recognition based on computing signatures and does not require a convolutional structure or an attention-based encoder that achieves an accuracy for 64 X 64 RGB images that exceeds many of the state-of-the-art methods and simultaneously requires orders of magnitude less FLOPS, power and memory footprint.

Neural-Signature Methods for Structured EHR Prediction

This first application of neural-signature methods in real-world healthcare data shows a competitive performance when compared to strong baselines and thus warrants further investigation within the health domain.

Path Signature Neural Network of Cortical Features for Prediction of Infant Cognitive Scores

A novel cortical feature based path signature neural network (CF-PSNet) is proposed with stacked differentiable temporal path signature layers for prediction of individual cognitive scores, and the existence embedding in path generation can improve the robustness against the missing data.

Functional linear regression with truncated signatures

The Signature Kernel Is the Solution of a Goursat PDE

It is shown that for continuously differentiable paths, the signature kernel solves a hyperbolic PDE and recognize the connection with a well known class of differential equations known in the literature as Goursat problems.

Deep Learning for Estimating Synaptic Health of Primary Neuronal Cell Culture

The image recognition model which is based on deep convolutional neural network (CNN) architecture with residual connections achieved accuracy of 99.6% on a binary classification task of distinguishing untreated and treated rodent primary neuronal cells with Amyloid-$\beta_{(25-35)}$.

Learning stochastic differential equations using RNN with log signature features

A hybrid Logsig-RNN algorithm that learns functionals on streamed data with outstanding accuracy with superior efficiency and robustness is proposed.

Offline Writer Identification Based on the Path Signature Feature

  • Songxuan LaiLianwen Jin
  • Computer Science
    2019 International Conference on Document Analysis and Recognition (ICDAR)
  • 2019
A codebook method based on the log path signature—a more compact way to express the path signature)—is used in this work and shows competitive results on several benchmark offline writer identification datasets, namely the IAM, Firemaker, CVL and ICDAR2013 writer identification contest dataset.



Rotation-free online handwritten character recognition using dyadic path signature features, hanging normalization, and deep neural network

This paper proposes a variation of path signature representation, namely the dyadic path signature feature (D-PSF), to fully characterize the trajectory using a hierarchical structure to solve the rotation-free online handwritten character recognition (OLHCR) problem.

P-CNN: Pose-Based CNN Features for Action Recognition

A new Pose-based Convolutional Neural Network descriptor (P-CNN) for action recognition that aggregates motion and appearance information along tracks of human body parts is proposed.


in the noncommutative indeterminates X1, , Xm. The main result of this paper is briefly as follows: If 0(oa) =0() for two irreducible piecewise C' and continuous curves oa and f, then f can be

Action recognition with novel high-level pose features

A set of novel high-level pose features (NHLPF) that combine the spatial and temporal information, and a joint energy change feature, which is designed using observations of the magnitude and direction of the force between joints, are presented.

NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis

A large-scale dataset for RGB+D human action recognition with more than 56 thousand video samples and 4 million frames, collected from 40 distinct subjects is introduced and a new recurrent neural network structure is proposed to model the long-term temporal correlation of the features for each body part, and utilize them for better action classification.

Berkeley MHAD: A comprehensive Multimodal Human Action Database

A controlled multimodal dataset consisting of temporally synchronized and geometrically calibrated data from an optical motion capture system, multi-baseline stereo cameras from multiple views, depth sensors, accelerometers and microphones, provides researchers an inclusive testbed to develop and benchmark new algorithms across multiple modalities under known capture conditions in various research domains.

Action bank: A high-level representation of activity in video

Inspired by the recent object bank approach to image representation, Action Bank is presented, a new high-level representation of video comprised of many individual action detectors sampled broadly in semantic space as well as viewpoint space that is capable of highly discriminative performance.

Two-person interaction detection using body-pose features and multiple instance learning

A complex human activity dataset depicting two person interactions, including synchronized video, depth and motion capture data is created, and techniques related to Multiple Instance Learning (MIL) are explored, finding that the MIL based classifier outperforms SVMs when the sequences extend temporally around the interaction of interest.

Di erential equations driven by rough signals

This paper aims to provide a systematic approach to the treatment of differential equations of the type dyt = Si fi(yt) dxti where the driving signal xt is a rough path. Such equations are very

Towards Understanding Action Recognition

It is found that high-level pose features greatly outperform low/mid level features, in particular, pose over time is critical, but current pose estimation algorithms are not yet reliable enough to provide this information.