Classical Risk-Averse Control for a Finite-Horizon Borel Model

  title={Classical Risk-Averse Control for a Finite-Horizon Borel Model},
  author={Margaret P. Chapman and Kevin M. Smith},
  journal={IEEE Control Systems Letters},
We study a risk-averse optimal control problem for a finite-horizon Borel model, where a cumulative cost is assessed via exponential utility. The setting permits non-linear dynamics, non-quadratic costs, and continuous state and control spaces but is less general than the problem of optimizing an expected utility. Our contribution is to show the existence of an optimal risk-averse controller without using state space augmentation and therefore offer a simpler solution method from first… 

Figures from this paper

On Exponential Utility and Conditional Value-at-Risk as Risk-Averse Performance Criteria
This work studies the applications of risk-averse functionals to controller synthesis and safety analysis through the development of numerical examples, with emphasis on EU and CVaR.
On Optimizing the Conditional Value-at-Risk of a Maximum Cost for Risk-Averse Safety Analysis
It is proved that the optimal CVaR of a maximum cost enjoys an equivalent representation in terms of the solutions to this family of dynamic programs under appropriate assumptions, and the existence of an optimal policy that depends on the dynamics of an augmented state under a measurable selection condition is shown.
Risk-averse autonomous systems: A brief history and recent developments from the perspective of optimal control
We offer a historical overview of methodologies for quantifying the notion of risk and optimizing risk-aware autonomous systems, with emphasis on riskaverse settings in which safety may be critical.
iRiSC: Iterative Risk Sensitive Control for Nonlinear Systems with Imperfect Observations
This work derives an algorithm that can compute local solutions with computational complexity similar to the iterative linear quadratic regulator algorithm that is the first algorithm that computes risk aware optimal controls that are a function of both the process noise and measurement uncertainty.


Risk-averse dynamic programming for Markov decision processes
The concept of a Markov risk measure is introduced and it is used to formulate risk-averse control problems for two Markov decision models: a finite horizon model and a discounted infinite horizon model.
Markov decision processes with recursive risk measures
A Bellman equation is derived and it is proved the existence of Markovian optimal policies and a connection to distributionally robust MDPs is established, which provides a global interpretation of the recursively defined objective function.
Average-Cost Optimality Results for Borel-Space Markov Decision Processes with Universally Measurable Policies
We consider discrete-time Markov Decision Processes with Borel state and action spaces and universally measurable policies. For several long-run average cost criteria, we establish the following
Impedance Optimization for Uncertain Contact Interactions Through Risk Sensitive Optimal Control
A stochastic optimal control algorithm known as Risk Sensitive Control is extended to take into account measurement uncertainty and a formal way to include such uncertainty for unknown contact locations is proposed.
Remote State Estimation for Nonlinear Systems via a Fading Channel: A Risk-sensitive Approach.
The risk-sensitive (RS) approach is introduced to formulate the estimation problem with intermittent measurements such that an exponential cost criterion is minimized, and the closed-form expression of the nonlinear RS estimator is derived.
Risk-sensitive safety analysis using Conditional Value-at-Risk
The method provides a novel, theoretically guaranteed, parameter-dependent upper bound to the CVaR of a maximum cost without the need to augment the state space, and proposes a tractable method for risk-sensitive safe sets estimation without using a parameter- dependent upper bound.
Toward a Scalable Upper Bound for a CVaR-LQ Problem
This work takes steps toward deriving a scalable dynamic programming approach to upper-bound the optimal value function for this problem, and yields a novel, tunable risk-averse control policy which is compared to existing state-of-the-art methods.
Discounted approximations in risk-sensitive average Markov cost chains with finite state space
It is proved that, as the discount factor increases to 1, an appropriate normalization of the discounted value functions converges to the average cost, extending recent results derived under the assumption that the state space is communicating.
Risk-Constrained Linear-Quadratic Regulators
A new risk constraint is introduced, which explicitly restricts the total expected predictive variance of the state penalty by a user-prescribed level, and it is shown that, under rather minimal conditions on the process noise, the optimal risk-aware controller can be evaluated explicitly and in closed form.