An Optimal Control Derivation of Nonlinear Smoothing Equations

  title={An Optimal Control Derivation of Nonlinear Smoothing Equations},
  author={Jin W. Kim and Prashant G. Mehta},
  journal={arXiv: Optimization and Control},
  • J. Kim, P. Mehta
  • Published 2019
  • Mathematics
  • arXiv: Optimization and Control
The purpose of this paper is to review and highlight some connections between the problem of nonlinear smoothing and optimal control of the Liouville equation. The latter has been an active area of recent research interest owing to work in mean-field games and optimal transportation theory. The nonlinear smoothing problem is considered here for continuous-time Markov processes. The observation process is modeled as a nonlinear function of a hidden state with an additive Gaussian measurement… Expand
What is the Lagrangian for Nonlinear Filtering?
The classical duality result of Kalman-Bucy is shown to be a special case and a constructive proof technique to derive the Kalman filter equation from the optimal control solution. Expand
O C ] 2 7 M ar 2 01 9 What is the Lagrangian for Nonlinear Filtering ?
Duality between estimation and optimal control is a problem of rich historical significance. The first duality principle appears in the seminar paper of Kalman-Bucy where the problem of minimumExpand
An optimal control approach to particle filtering
A distinguishing feature of the proposed method is that it uses the measurements over a finite-length time window instead of a single measurement for the estimation at each time step, resembling the batch methods of filtering, and improving fault tolerance. Expand
Ensemble Kalman Filter (EnKF) for Reinforcement Learning (RL)
A novel simulation-based algorithm, namely an ensemble Kalman filter (EnKF), is introduced, used to obtain formulae for optimal control, expressed entirely in terms of the EnKF particles. Expand
A Dual Characterization of the Stability of the Wonham Filter
This paper revisits the classical question of the stability of the nonlinear Wonham filter. The novel contributions of this paper are two-fold: (i) definition of the stabilizability for theExpand
The Conditional Poincar\'e Inequality for Filter Stability
This paper is concerned with the problem of nonlinear filter stability of ergodic Markov processes. The main contribution is the conditional Poincaré inequality (PI) which is shown to yield filterExpand


What is the Lagrangian for Nonlinear Filtering?
The classical duality result of Kalman-Bucy is shown to be a special case and a constructive proof technique to derive the Kalman filter equation from the optimal control solution. Expand
A Variational Approach to Nonlinear Estimation
Regular conditional versions of the forward and inverse Bayes formula are shown to have dual variational characterizations involving the minimization of apparent information and the maximization of compatible information, according to which Bayes' formula and its inverse are optimal information processors. Expand
Filtering, Stability, and Robustness
The theory of nonlinear filtering concerns the optimal estimation of a Markov signal in noisy observations. Such estimates necessarily depend on the model that is chosen for the signal andExpand
On the Relation Between Optimal Transport and Schrödinger Bridges: A Stochastic Control Viewpoint
A new look at the relation between the optimal transport problem and the Schrödinger bridge problem from a stochastic control perspective is taken and a generalization of optimal mass transport in the form of a (fluid dynamic) problem of optimal transport with prior is considered. Expand
Particle Smoothing for Hidden Diffusion Processes: Adaptive Path Integral Smoother
  • H. Ruiz, H. Kappen
  • Computer Science, Mathematics
  • IEEE Transactions on Signal Processing
  • 2017
A novel algorithm based on path integral control theory is proposed to efficiently estimate the smoothing distribution of continuous-time diffusion processes from partial observations by using an adaptive importance sampling method to improve the effective sampling size of the posterior and the reliability of the estimation of the marginals. Expand
Variational and optimal control representations of conditioned and driven processes
These interpretations of the driven process generalize and unify many previous results on maximum entropy approaches to nonequilibrium systems, spectral characterizations of positive operators, and control approaches to large deviation theory and lead to new methods for analytically or numerically approximating large deviation functions. Expand
Optimal control and nonlinear filtering for nondegenerate diffusion processes
A linear parabolic partial differential equation describing the pathwise filter for a nondegenerate diffusion is changed, by an exponential substitution. into the dynamic programming equation of anExpand
Maximum-likelihood recursive nonlinear filtering
The basic model for the general nonlinear filtering problem consists of a nonlinear plant driven by noise followed by nonlinear observation with additive noise. The object is to estimate, at eachExpand
Adaptive Importance Sampling for Control and Inference
  • H. Kappen
  • Mathematics, Computer Science
  • ArXiv
  • 2015
This contribution reviews PI control theory in the finite horizon case and derives a gradient descent method that allows to learn feed-back controllers using an arbitrary parametrisation and demonstrates the PI control method as an accurate alternative to particle filtering. Expand
A Variational Approach to Path Estimation and Parameter Inference of Hidden Diffusion Processes
A variational method for approximating the hidden states of the signal process given the full set of observations is developed, which leads to systematic approximations of the smoothing densities ofThe signal process. Expand