• Corpus ID: 53223106

Adaptive Stress Testing: Finding Failure Events with Reinforcement Learning

  title={Adaptive Stress Testing: Finding Failure Events with Reinforcement Learning},
  author={Ritchie Lee and Ole Jakob Mengshoel and Anshu Saksena and Ryan Gardner and Daniel Genin and Joshua Silbermann and Michael P. Owen and Mykel J. Kochenderfer},
Finding the most likely path to a set of failure states is important to the analysis of safety-critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due to the complex stochastic environment in which the system operates. As a result, safety validation is not only concerned about whether a failure can occur, but also discovering which… 

Efficient Black-box Assessment of Autonomous Vehicle Safety

A simulation testing framework is implemented that evaluates an entire modern AV system as a black box, estimating the probability of accidents under a base distribution governing standard traffic behavior and efficiently learn to identify and rank failure scenarios via adaptive importance-sampling methods.

Health Monitoring System for Autonomous Vehicles using Dynamic Bayesian Networks for Diagnosis and Prognosis

A Hierarchical Component-based Health Monitoring System with Fault Detection, Diagnosis and Prognosis using Dynamic Bayesian Network (DBN) with residue generation, a combination of knowledge-based and model-based detection, diagnosis and prognosis approaches is presented.

AV-FUZZER: Finding Safety Violations in Autonomous Driving Systems

  • Guanpeng LiYiran Li R. Iyer
  • Computer Science
    2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)
  • 2020
This paper proposes AV-FUZZER, a testing framework, to find the safety violations of an autonomous vehicle (AV) in the presence of an evolving traffic environment and designs a local fuzzer that increases the exploitation of local optima in the areas where highly likely safety-hazardous situations are observed.

Simulation-based Testing for Early Safety-Validation of Robot Systems

This work addresses the problem of safety flaws in industrial human-robot collaborative systems by using a human model and an optimization algorithm to generate high-risk human behavior in simulation, thereby exposing potential hazards.

Virtual Adversarial Humans finding Hazards in Robot Workplaces

Although this approach cannot replace a thorough hazard analysis, it can help uncover hazards that otherwise may have been overlooked, especially in early development stages, and helps to prevent costly re-designs at later development stages.

An Evaluation of Monte-Carlo Tree Search for Property Falsification on Hybrid Flight Control Laws

An evaluation of a simple Monte-Carlo Tree Search property falsification algorithm, applied to select properties of a longitudinal hybrid flight control law: a threshold overshoot property, two frequential properties, and a discrete event-based property.

Adaptive Stress Testing of Trajectory Predictions in Flight Management Systems

This work analyzes a trajectory predictor from a developmental commercial flight management system which takes as input a collection of lateral waypoints and en-route environmental conditions and uses a modified Monte Carlo tree search algorithm with progressive widening as its adversarial reinforcement learner.



Adaptive Stress Testing for Autonomous Vehicles

It is shown that DRL can find more likely failure scenarios than MCTS with fewer calls to the simulator and can be easily applied to other scenarios given the appropriate models of the vehicle and the environment.

Falsification of Cyber-Physical Systems Using Deep Reinforcement Learning

State-of-the-art Deep Reinforcement Learning techniques are explored, i.e., Asynchronous Advantage Actor-Critic and Double Deep Q Network (DDQN), to reduce the number of simulation runs required to find such counterexamples and identify two factors of CPS which make DRL based methods better than existing methods.

Efficient Guiding Strategies for Testing of Temporal Properties of Hybrid Systems

This paper presents an approach that uses the rapidly exploring random trees (RRT) technique to explore the state-space of a CPS, and shows that it scales to industrial-scale CPSs by demonstrating its efficacy on an automotive powertrain control system.

Formally Verified Safe Vertical Maneuvers for Non-deterministic, Accelerating Aircraft Dynamics

We present the formally verified predicate and strategy used to independently evaluate the safety of the final version (Run 15) of the FAAs next-generation air-traffic collision avoidance system,

Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation

A simulation framework that can test an entire modern autonomous driving system, including, in particular, systems that employ deep-learning perception and control algorithms, is implemented.

Decision Making Under Uncertainty: Theory and Application

This book provides an introduction to the challenges of decision making under uncertainty from a computational perspective and presents both the theory behind decision making models and algorithms and a collection of example applications that range from speech recognition to aircraft collision avoidance.

Reinforcement Learning: An Introduction

This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.

A decision-theoretic approach to developing robust collision avoidance logic

An automated approach for optimizing collision avoidance logic based on probabilistic models of aircraft behavior and a performance metric that balances the competing objectives of maximizing safety and minimizing alert rate is presented.

Accelerated Evaluation of Automated Vehicles Safety in Lane-Change Scenarios Based on Importance Sampling Techniques

Frontal collision due to unsafe cut-ins is the target crash type of this paper and the cross-entropy method is used to recursively search for the optimal skewing parameters to accelerate the verification of AVs in simulations and controlled experiments.

A Formally Verified Hybrid System for the Next-Generation Airborne Collision Avoidance System

The geometric configurations under which the advice given by ACAS X is safe under a precise set of assumptions are determined and formally verify these configurations using hybrid systems theorem proving techniques.