• Corpus ID: 116607689

On-line Human Gait Stability Prediction using LSTMs for the fusion of Deep-based Pose Estimation and LRF-based Augmented Gait State Estimation in an Intelligent Robotic Rollator

  title={On-line Human Gait Stability Prediction using LSTMs for the fusion of Deep-based Pose Estimation and LRF-based Augmented Gait State Estimation in an Intelligent Robotic Rollator},
  author={Georgia Chalvatzaki and Petros Koutras and Jack Hadfield and Xanthi S. Papageorgiou and Costas S. Tzafestas and Petros Maragos},
In this work we present a novel Long Short Term Memory (LSTM) based on-line human gait stability prediction framework for the elderly users of an intelligent robotic rollator, using only non-wearable sensors, fusing multimodal RGB-D and Laser Range Finder (LRF) data. A deep learning (DL) based approach is used for the upper body pose estimation. The detected pose is used for estimating the Center of Mass (CoM) of the body using Unscented Kalman Filter (UKF). An Augmented Gait State Estimation… 

Figures and Tables from this paper

Comparing the Impact of Robotic Rollator Control Schemes on Elderly Gait using on-line LRF-based Gait Analysis
A thorough experimental analysis is presented, using an on-line gait tracking and analysis system, to examine the impact of different control designs on the gait performance of elderly subjects who use an intelligent robotic rollator.
Learn to Adapt to Human Walking: A Model-Based Reinforcement Learning Approach for a Robotic Assistant Rollator
A novel approach using model-based reinforcement learning (MBRL) for adapting the control policy of the robotic assistant to patients with different mobility status is proposed, which encapsulates previous work on human tracking and gait analysis from RGB-D and laser streams into a human-in-the-loop decision making strategy.
Feature-Based Gait Pattern Classification for a Robotic Walking Frame
A system for fast detection of gait patterns of walking frame users, where the challenge is to recognize a change in activity before the signal behaves stationary, to realize a fast real-time online gait classification.
User Centered HRI Design for an Intelligent Robotic Rollator
Emphasis is placed on adopting multimodal communication patterns from actual human-human interaction in the context of mobility rehabilitation, which may enrich human-robot communication by increasing the naturalness in interaction from the side of the robotic device by adding more “human” characteristics both in respect to understanding and reaction capabilities of the robot.
Air Quality Prediction based on LSTM-Kalman Model
This work compares the RMSE (Root Mean Square Error) and R-Squared (determination coefficient) of the LSTM model with the L STM-Kalman model predictions, and shows that the LstM- Kalman model has a better fit to the predicted values of the model data.
Analysis, Visualization and Forecasting of COVID-19 Outbreak Using LSTM Model
The LSTM model is used to predict the outbreak of COVID-19 all over the world with limited epidemiological data and shows that the growth rate of infected cases of CO VID-19 increased exponentially every week.


Augmented Human State Estimation Using Interacting Multiple Model Particle Filters With Probabilistic Data Association
A novel human gait tracking approach that uses two particle filters (PFs) and probabilistic data association (PDA) with an interacting multiple model (IMM) scheme for a real-time selection of the appropriate motion model according to thehuman gait analysis and the use of the Viterbi algorithm for an augmentedHuman gait state estimation.
User-Adaptive Human-Robot Formation Control for an Intelligent Robotic Walker Using Augmented Human State Estimation and Pathological Gait Characterization
The experimental evaluation comprises gait data from real patients, and demonstrates the stability of the human-robot formation control, indicating the importance of incorporating an on-line gait characterization of the user, using non-wearable and non-invasive methods, in the context of a robotic MAD.
Hidden markov modeling of human pathological gait using laser range finder for an assisted living intelligent robotic walker
This paper presents a completely non-invasive framework for analyzing and recognizing a pathological human walking gait pattern, using a laser range finder sensor to detect and track the human legs, and an appropriately synthesized Hidden Markov Model for state estimation, and recognition of the gait patterns.
Hidden Markov modeling of human normal gait using laser range finder for a mobility assistance robot
A completely non-invasive framework for analyzing a normal human walking gait pattern and has the potential to be used for the recognition of abnormal gait patterns and the subsequent classification of specific walking pathologies, needed for the development of a context-aware robot mobility assistant.
Estimating double support in pathological gaits using an HMM-based analyzer for an intelligent robotic walker
The results obtained and presented demonstrate that the proposed human data analysis (modeling, learning and inference) framework has the potential to support efficient detection and classification of specific walking pathologies, as needed to empower a cognitive robotic mobility-assistance device with user-adaptive and context-aware functionalities.
Towards a user-adaptive context-aware robotic walker with a pathological gait assessment system: First experimental study
The results demonstrate that a generic control scheme does not meet every patient's needs, and therefore, an Adaptive Context-Aware MAD (ACA MAD), that can understand the specific needs of the user, is important for enhancing the human-robot physical interaction.
Experimental validation of human pathological gait analysis for an assisted living intelligent robotic walker
Experimental validation of an in house developed gait analysis system using a Hidden Markov Model for gait cycle estimation, and extraction of the gait parameters demonstrates that the added complexity of the HMM system is necessary for improving the accuracy and efficacy of the system.
Visual human action classification for control of a passive walker
Human action/behavior classification plays an important role for controlling systems having interaction with human users. Safety and dependability of such systems are crucial especially for walking
Particle filter based lower limb prediction and motion control for JAIST Active Robotic Walker
A particle-filter-based prediction technique and a motion controller are developed to help JARoW smoothly generate the direction and velocity of its movements in a way that reflects the prediction.
Gait phase analysis based on a Hidden Markov Model