ROIAL: Region of Interest Active Learning for Characterizing Exoskeleton Gait Preference Landscapes

  title={ROIAL: Region of Interest Active Learning for Characterizing Exoskeleton Gait Preference Landscapes},
  author={Kejun Li and Maegan Tucker and Erdem Biyik and Ellen R. Novoseller and Joel W. Burdick and Yanan Sui and Dorsa Sadigh and Yisong Yue and A. Ames},
  journal={2021 IEEE International Conference on Robotics and Automation (ICRA)},
  • Kejun Li, Maegan Tucker, A. Ames
  • Published 9 November 2020
  • Computer Science
  • 2021 IEEE International Conference on Robotics and Automation (ICRA)
Characterizing what types of exoskeleton gaits are comfortable for users, and understanding the science of walking more generally, require recovering a user’s utility landscape. Learning these landscapes is challenging, as walking trajectories are defined by numerous gait parameters, data collection from human trials is expensive, and user safety and comfort must be ensured. This work proposes the Region of Interest Active Learning (ROIAL) framework, which actively learns each user’s underlying… 

Figures from this paper

An Approach to Investigate Public Opinion, Views, and Perspectives Towards Exoskeleton Technology
Over the last decade, exoskeletons have had an extensive impact on different disciplines and application domains such as assisted living, military, healthcare, firefighting, and industries, on
Policy-Based Bayesian Experimental Design for Non-Differentiable Implicit Models
Reinforcement Learning for Deep Adaptive Design (RL-DAD), a method for simulation-based optimal experimental design for non-differentiable implicit models, is introduced and it is found that it performs competitively with baselines on three benchmarks.
Safety-Aware Preference-Based Learning for Safety-Critical Control
This work presents a design paradigm for systematically achieving behaviors that balance performance and robust safety by integrating safety-aware Preference-Based Learning (PBL) with Control Barrier Functions (CBFs).
The role of user preference in the customized control of robotic exoskeletons
User preference is a promising objective for the control of robotic exoskeletons because it may capture the multifactorial nature of exoskeleton use. However, to use it, we must first understand its
APReL: A Library for Active Preference-based Reward Learning Algorithms
APReL is presented, a library for active preference-based reward learning algorithms, which enable researchers and practitioners to experiment with the existing techniques and easily develop their own algorithms for various modules of the problem.
Learning Controller Gains on Bipedal Walking Robots via User Preferences
The extent to which preference-based learning can be used to optimize controller gains online by repeatedly querying the user for their preferences is explored and the proposed method is repeatably able to learn gains that yield stable and robust locomotion.
Learning Multimodal Rewards from Rankings
This workforms the multimodal reward learning as a mixture learning problem and develops a novel ranking-based learning approach, where the experts are only required to rank a given set of trajectories, and develops an active querying approach to accelerate the learning process.
Learning Reward Functions from Scale Feedback
A probabilistic model on how users would provide feedback and derive a learning framework for the robot is introduced and demonstrated, and the approach is validated in two user studies suggesting that scale feedback enables more effective learning in practice.
Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and preferences
This work presents an algorithm that first utilizes user demonstrations to initialize a belief about the reward function, and then proactively probes the user with preference queries to zero-in on their true reward, enabling a framework to integrate multiple sources of information, which are either passively or actively collected from human users.


Preference-Based Learning for Exoskeleton Gait Optimization
In the experiments, CoSpar consistently found user-preferred parameters of the exoskeleton’s walking gait, which suggests that it is a promising starting point for adapting and personalizing exoskeletons (or other assistive devices) to individual users.
Preference-based Online Learning with Dueling Bandits: A Survey
The aim of this paper is to provide a survey of the state of the art in this field, referred to as preference-based multi-armed bandits or dueling bandits, and to provide an overview of problems that have been considered in the literature as well as methods for tackling them.
Active Preference Learning using Maximum Regret
This work proposes a query selection that greedily reduces the maximum error ratio over the solution space and demonstrates that the proposed approach outperforms other state of the art techniques in both learning efficiency and ease of queries for the user.
Active Preference-Based Gaussian Process Regression for Reward Learning
This work model the reward function using a Gaussian Process and proposes a mathematical formulation to actively find a GP using only human preferences, enabling it to tackle both inflexibility and data-inefficiency problems within a preference-based learning framework.
Human Preference-Based Learning for High-dimensional Optimization of Exoskeleton Walking Gaits
This work empirically verify that LINECOSPAR is a sample-efficient approach for high-dimensional preference optimization and analysis of the experimental data reveals a correspondence between human preferences and objective measures of dynamicity, while also highlighting differences in the utility functions underlying individual users’ gait preferences.
Asking Easy Questions: A User-Friendly Approach to Active Reward Learning
This paper explores an information gain formulation for optimally selecting questions that naturally account for the human's ability to answer, and determines when these questions become redundant or costly.
Personalized gait trajectory generation based on anthropometric features using Random Forest
A personalized gait generation method based anthropometric features is proposed and the performance of the proposed method is demonstrated by several comparison experiments.
A Method for Online Optimization of Lower Limb Assistive Devices with High Dimensional Parameter Spaces
The offline portion of the optimization method indeed produces control parameters that can adapt to different gaits, and for three out of the four parameter sets the authors tested, the procedure also generates parameters that improve the ability of the prosthesis to adapt to increasing gait speed by increasing ankle net work production.
Dynamic Humanoid Locomotion: A Scalable Formulation for HZD Gait Optimization
A methodology that allows for fast and reliable generation of dynamic robotic walking gaits through the HZD framework, even in the presence of underactuation, and develops a defect-variable substitution formulation to simplify expressions, which ultimately allows for compact analytic Jacobians of the constraints.
Feedback Control of an Exoskeleton for Paraplegics: Toward Robustly Stable, Hands-Free Dynamic Walking
This article describes the hardware that was designed to achieve hands-free dynamic walking, the control laws that were deployed (and those being developed) to provide enhanced mobility and robustness, and preliminary test results.