• Corpus ID: 53223106

Adaptive Stress Testing: Finding Failure Events with Reinforcement Learning

  title={Adaptive Stress Testing: Finding Failure Events with Reinforcement Learning},
  author={Ritchie Lee and Ole Jakob Mengshoel and Anshu Saksena and Ryan Gardner and Daniel Genin and Joshua Silbermann and Michael P. Owen and Mykel J. Kochenderfer},
Finding the most likely path to a set of failure states is important to the analysis of safety-critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due to the complex stochastic environment in which the system operates. As a result, safety validation is not only concerned about whether a failure can occur, but also discovering which… 
Adaptive Stress Testing of Trajectory Predictions in Flight Management Systems
This work analyzes a trajectory predictor from a developmental commercial flight management system which takes as input a collection of lateral waypoints and en-route environmental conditions and uses a modified Monte Carlo tree search algorithm with progressive widening as its adversarial reinforcement learner.
Efficient Black-box Assessment of Autonomous Vehicle Safety
A simulation testing framework is implemented that evaluates an entire modern AV system as a black box, estimating the probability of accidents under a base distribution governing standard traffic behavior and efficiently learn to identify and rank failure scenarios via adaptive importance-sampling methods.
Health Monitoring System for Autonomous Vehicles using Dynamic Bayesian Networks for Diagnosis and Prognosis
A Hierarchical Component-based Health Monitoring System with Fault Detection, Diagnosis and Prognosis using Dynamic Bayesian Network (DBN) with residue generation, a combination of knowledge-based and model-based detection, diagnosis and prognosis approaches is presented.
AV-FUZZER: Finding Safety Violations in Autonomous Driving Systems
  • Guanpeng Li, Yiran Li, R. Iyer
  • Computer Science
    2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)
  • 2020
This paper proposes AV-FUZZER, a testing framework, to find the safety violations of an autonomous vehicle (AV) in the presence of an evolving traffic environment and designs a local fuzzer that increases the exploitation of local optima in the areas where highly likely safety-hazardous situations are observed.
Simulation-based Testing for Early Safety-Validation of Robot Systems
This work addresses the problem of safety flaws in industrial human-robot collaborative systems by using a human model and an optimization algorithm to generate high-risk human behavior in simulation, thereby exposing potential hazards.
Virtual Adversarial Humans finding Hazards in Robot Workplaces
Although this approach cannot replace a thorough hazard analysis, it can help uncover hazards that otherwise may have been overlooked, especially in early development stages, and helps to prevent costly re-designs at later development stages.
An Evaluation of Monte-Carlo Tree Search for Property Falsification on Hybrid Flight Control Laws
An evaluation of a simple Monte-Carlo Tree Search property falsification algorithm, applied to select properties of a longitudinal hybrid flight control law: a threshold overshoot property, two frequential properties, and a discrete event-based property.


Adaptive stress testing of airborne collision avoidance systems
This paper presents a scalable method to efficiently search for the most likely state trajectory leading to an event given only a simulator of a system using Monte Carlo Tree Search (MCTS), and presents results for both single and multi-threat encounters.
Probabilistic model checking of the next-generation airborne collision avoidance system
A probabilistic model checking approach to efficiently compute the probabilities of generically specified events and the most likely sequences of states leading to those events within a discrete-time Markov chain model of aircraft flight and ACAS X.
Efficient Guiding Strategies for Testing of Temporal Properties of Hybrid Systems
This paper presents an approach that uses the rapidly exploring random trees (RRT) technique to explore the state-space of a CPS, and shows that it scales to industrial-scale CPSs by demonstrating its efficacy on an automotive powertrain control system.
Formally Verified Safe Vertical Maneuvers for Non-deterministic, Accelerating Aircraft Dynamics
We present the formally verified predicate and strategy used to independently evaluate the safety of the final version (Run 15) of the FAAs next-generation air-traffic collision avoidance system,
Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation
A simulation framework that can test an entire modern autonomous driving system, including, in particular, systems that employ deep-learning perception and control algorithms, is implemented.
Decision Making Under Uncertainty: Theory and Application
This book provides an introduction to the challenges of decision making under uncertainty from a computational perspective and presents both the theory behind decision making models and algorithms and a collection of example applications that range from speech recognition to aircraft collision avoidance.
A decision-theoretic approach to developing robust collision avoidance logic
An automated approach for optimizing collision avoidance logic based on probabilistic models of aircraft behavior and a performance metric that balances the competing objectives of maximizing safety and minimizing alert rate is presented.
Accelerated Evaluation of Automated Vehicles Safety in Lane-Change Scenarios Based on Importance Sampling Techniques
Frontal collision due to unsafe cut-ins is the target crash type of this paper and the cross-entropy method is used to recursively search for the optimal skewing parameters to accelerate the verification of AVs in simulations and controlled experiments.
A Formally Verified Hybrid System for the Next-Generation Airborne Collision Avoidance System
The geometric configurations under which the advice given by ACAS X is safe under a precise set of assumptions are determined and formally verify these configurations using hybrid systems theorem proving techniques.
Optimizing the Next Generation Collision Avoidance System for Safe, Suitable, and Acceptable Operational Performance
An iterative tuning process reduced the operational impact on the air traffic system and improved acceptability of alerts, and a 15-month effort that resulted in substantial improvements are summarized.