Toward Expedited Impedance Tuning of a Robotic Prosthesis for Personalized Gait Assistance by Reinforcement Learning Control

  title={Toward Expedited Impedance Tuning of a Robotic Prosthesis for Personalized Gait Assistance by Reinforcement Learning Control},
  author={Minhan Li and Yue Wen and Xiang Gao and Jennie Si and H. Huang},
  journal={IEEE Transactions on Robotics},
Personalizing medical devices such as lower limb wearable robots is challenging. While the initial feasibility of automating the process of knee prosthesis control parameter tuning has been demonstrated in a principled way, the next critical issue is to improve tuning efficiency and speed it up for the human user, in clinic settings, while maintaining human safety. We, therefore, propose a policy iteration with constraint embedded (PICE) method as an innovative solution to the problem under the… 

Figures and Tables from this paper

Imposing Healthy Hip Motion Pattern and Range by Exoskeleton Control for Individualized Assistance

A model-free reinforcement learning (RL)-based control framework to achieve a normative range of motion and gait pattern of the hip joint during walking and provides personalized assistance torque profile by heuristically manipulating three control parameters for hip flexion and extension, respectively, during walking.

Understanding the Preferences for Lower-Limb Prosthesis: A Think-Aloud Study during User-Guided Auto-Tuning

Prostheses help amputees to maintain physical health and quality of life. Prosthesis wearers’ satisfaction and adherence to the prosthesis are closely related to the preferences for prosthesis tuning

Toward Safe Wearer-Prosthesis Interaction: Evaluation of Gait Stability and Human Compensation Strategy Under Faults in Robotic Transfemoral Prostheses

Although advanced wearable robots can assist human wearers, their internal faults (i.e., sensors or control errors) also pose a challenge. To ensure safe wearer-robot interactions, how internal

A New Robotic Knee Impedance Control Parameter Optimization Method Facilitated by Inverse Reinforcement Learning

Recent efforts in the design of intelligent controllers for configuring robotic prostheses have demonstrated new possibilities in improving mobility and restoring locomotion for individuals with

Characterizing Prosthesis Control Fault During Human-Prosthesis Interactive Walking Using Intrinsic Sensors

The physical interactions between wearable lower limb robots and humans have been investigated to inform effective robot design for walking augmentation. However, human-robot interactions when

Admittance Control Based Human-in-the-Loop Optimization for Hip Exoskeleton Reduces Human Exertion during Walking

Human-in-the-loop (HIL) optimization usually optimizes assistive torque of exoskeletons to minimize the human's energetic expenditure in walking, quantified by metabolic cost. This formulation can,

Human-in-the-Loop Optimization of Exoskeleton Assistance Via Online Simulation of Metabolic Cost

A HIL methodology, which optimizes the assistive torques provided by a powered hip exoskeleton, is presented, and the time investment required to identify assistance strategies via this protocol is significantly lower when compared to existing protocols relying on calorimetry.

Shaping Individualized Impedance Landscapes for Gait Training via Reinforcement Learning

The proposed adaptive AAN controller is capable of autonomously reshaping the force field in a phase-dependent manner according to each individual's motor abilities and task requirements and its ability to promote shortterm motor adaptations is experimentally validated.

Robotic Knee Tracking Control to Mimic the Intact Human Knee Profile Based on Actor-Critic Reinforcement Learning

This work addresses a state-of-the-art reinforcement learning (RL) control approach to automatically configure robotic prosthesis impedance parameters to enable end-to-end, continuous locomotion intended for transfemoral amputee subjects and provides tracking control of a robotic knee prosthesis to mimic the intact knee profile.



Least-Squares Policy Iteration

The new algorithm, least-squares policy iteration (LSPI), learns the state-action value function which allows for action selection without a model and for incremental policy improvement within a policy-iteration framework.

Improving Finite State Impedance Control of Active-Transfemoral Prosthesis Using Dempster-Shafer Based State Transition Rules

This study aimed to generate more robust FSI control of ATP against these variations by using Dempster-Shafer theory (DST)-based transition rules, which yielded enhanced accuracy in state transition timing and reduced control errors when intra- and inter-subject variations were presented.

Knowledge-Guided Reinforcement Learning Control for Robotic Lower Limb Prosthesis

The knowledge-guided Q-learning (KG-QL) control method is proposed as a principled way for Reinforcement learning (RL) control and its feasibility is demonstrated by employing OpenSim, a well-established human locomotion simulator.

Offline Policy Iteration Based Reinforcement Learning Controller for Online Robotic Knee Prosthesis Parameter Tuning

The results showed that the RL control resulted in good convergent behavior in kinematic states, and the offline learning control policy successfully adjusted the prosthesis control parameters to produce near-normal knee kinematics in 10 updates of the impedance control parameters.

Model-Based Adaptive Control of Transfemoral Prostheses: Theory, Simulation, and Experiments

This paper presents and experimentally implements three different adaptive and robust adaptive controllers as the first steps toward using model-based controllers for transfemoral prostheses, from bipedal robotic walking to prosthesis walking, including a rigorous stability analysis.

Robust Ground Reaction Force Estimation and Control of Lower-Limb Prostheses: Theory and Simulation

Four robust MRAI observer/controller combinations for GRF estimation-based control of a prosthesis and a legged robot model in the presence of parametric uncertainties and unmodeled dynamics, in which the robot model is employed to mimic able-bodied walking.

Alternating Projection Methods

This book describes and analyzes all available alternating projection methods for solving the general problem of finding a point in the intersection of several given sets belonging to a Hilbert

The biomechanics and motor control of human gait

In a sodium-sulphur battery cell of tubular form, the sulphur compartment is provided inside a tubular solid electrolyte and a carbon current collector extends into the sodium compartment.

Online Reinforcement Learning Control for the Personalization of a Robotic Knee Prosthesis

The feasibility of direct heuristic dynamic programming, an approximate dynamic programming (ADP) approach, to automatically tune the 12 robotic knee prosthesis parameters to meet individual human users’ needs is demonstrated.