Explicit solution of relative entropy weighted control

@article{Bierkens2014ExplicitSO,
  title={Explicit solution of relative entropy weighted control},
  author={Joris Bierkens and Hilbert J. Kappen},
  journal={Syst. Control. Lett.},
  year={2014},
  volume={72},
  pages={36-43}
}
Abstract We consider the minimization over probability measures of the expected value of a random variable, regularized by relative entropy with respect to a given probability distribution. In the general setting we provide a complete characterization of the situations in which a finite optimal value exists and the situations in which a minimizing probability distribution exists. Specializing to the case where the underlying probability distribution is Wiener measure, we characterize finite… Expand
Malliavin Calculus Method for Asymptotic Expansion of Dual Control Problems
  • M. Monoyios
  • Mathematics, Computer Science
  • SIAM J. Financial Math.
  • 2013
We develop a technique based on Malliavin-Bismut calculus ideas, for asymptotic expansion of dual control problems arising in connection with exponential indifference valuation of claims, and withExpand
Variational and optimal control representations of conditioned and driven processes
TLDR
These interpretations of the driven process generalize and unify many previous results on maximum entropy approaches to nonequilibrium systems, spectral characterizations of positive operators, and control approaches to large deviation theory and lead to new methods for analytically or numerically approximating large deviation functions. Expand
Linear PDEs and eigenvalue problems corresponding to ergodic stochastic optimization problems on compact manifolds
Long term average or 'ergodic' optimal control problems on a compact manifold are considered. The problems exhibit a special structure which is typical of control problems related to large deviationsExpand
Stochastic Optimal Control as Non-equilibrium Statistical Mechanics: Calculus of Variations over Density and Current
TLDR
This work proposes derivation of the generalized gauge-invariant Hamilton-Jacobi-Bellman equation as a variation over density and current, suggest hydrodynamic interpretation and discusses examples, e.g., ergodic control of a particle-within-a-circle, illustrating non-equilibrium space-time complexity. Expand
Linear PDEs and eigenvalue problems corresponding to ergodic stochastic optimization problems on compact manifolds
We consider long term average or `ergodic' optimal control poblems with a special structure: Control is exerted in all directions and the control costs are proportional to the square of the norm ofExpand
Nonlinear stochastic receding horizon control: stability, robustness and Monte Carlo methods for control approximation
TLDR
An approximation method based on Monte Carlo simulation derived via the Feynman–Kac formula gives a stochastic interpretation for the solution of a Hamilton–Jacobi–Bellman equation associated with the true optimal controller. Expand
Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space
TLDR
A principled framework based on divergences between path measures is developed, inspired by connections to forward-backward SDEs, and proposed and studied the novel log-variance divergence, showing favourable properties of corresponding Monte Carlo estimators. Expand
A Variational Formula for Risk-Sensitive Reward
We derive a variational formula for the optimal growth rate of reward in the infinite horizon risk-sensitive control problem for discrete time Markov decision processes with compact metric state andExpand
Adaptive Smoothing Path Integral Control
TLDR
A model-free algorithm called ASPIC (Adaptive Smoothing of Path Integral Control) is proposed that applies an inf-convolution to the cost function to speedup convergence of policy optimization and shows analytically and empirically that intermediate levels of smoothing are optimal, which renders the new method superior to both PICE and direct cost-optimization. Expand
J an 2 01 5 A VARIATIONAL FORMULA FOR RISK-SENSITIVE REWARD
We derive a variational formula for the optimal growth rate of reward in the infinite horizon risk-sensitive control problem for discrete time Markov decision processes with compact metric state andExpand
...
1
2
3
...

References

SHOWING 1-10 OF 43 REFERENCES
Malliavin Calculus Method for Asymptotic Expansion of Dual Control Problems
  • M. Monoyios
  • Mathematics, Computer Science
  • SIAM J. Financial Math.
  • 2013
We develop a technique based on Malliavin-Bismut calculus ideas, for asymptotic expansion of dual control problems arising in connection with exponential indifference valuation of claims, and withExpand
A variational representation for certain functionals of Brownian motion
In this paper we show that the variational representation - log Ee -f(W) = inf E{1/2∫ 0 1 ∥v s ∥ 2 ds + f(w + ∫ 0 v s ds)} holds, where W is a standard d-dimensional Brownian motion, f is any boundedExpand
Meshless discretization of LQ-type stochastic control problems
We propose a novel Galerkin discretization scheme for stochastic optimal control problems on an indefinite time horizon. The control problems are linear-quadratic in the controls, but possiblyExpand
Malliavin calculus for Levy processes with applications to finance
Extending Gaussian Malliavin derivatives, to a finite moments Levy process framework, using chaos expansions has proven to be a successful approach. In this work the theory is extended by theExpand
A weak convergence approach to the theory of large deviations
Formulation of Large Deviation Theory in Terms of the Laplace Principle. First Example: Sanov's Theorem. Second Example: Mogulskii's Theorem. Representation Formulas for Other Stochastic Processes.Expand
A Unifying Framework for Linearly Solvable Control
TLDR
This work generalizes the recently developed risk-sensitive path-integral control formalism and demonstrates the usefulness of the framework on control problems with noise where different values of lead to qualitatively different control behaviors. Expand
Linearly-solvable Markov decision problems
TLDR
A class of MPDs which greatly simplify Reinforcement Learning, which have discrete state spaces and continuous control spaces and enable efficient approximations to traditional MDPs. Expand
Malliavin Calculus for Lévy Processes with Applications to Finance
The Continuous Case: Brownian Motion.- The Wiener-Ito Chaos Expansion.- The Skorohod Integral.- Malliavin Derivative via Chaos Expansion.- Integral Representations and the Clark-Ocone formula.- WhiteExpand
Controlled Markov processes and viscosity solutions
This book is intended as an introduction to optimal stochastic control for continuous time Markov processes and to the theory of viscosity solutions. The authors approach stochastic control problemsExpand
Logarithmic Transformations and Stochastic Control
Abstract : We are concerned with a class of problems described in a somewhat imprecise way as follows. Consider a linear operator of the form L + V(x), where L is the generator of a Markov process xExpand
...
1
2
3
4
5
...