System identification with information theoretic criteria

  title={System identification with information theoretic criteria},
  author={Anton A. Stoorvogel and Jan H. van Schuppen},
Attention is focused in this paper on the approximation problem of system identification with information theoretic criteria. For a class of problems it is shown that the criterion of mutual information rate is identical to the criterion of exponential-of-quadratic cost and to $H_{infty$ entropy. In addition the relation between the likelihood function and divergence is explored. As a consequence of these relations a parameter estimator is derived by four methods for the approximation of a… 

Information-theoretic system identification

  • K. Chernyshov
  • Computer Science
    2017 4th International Conference on Control, Decision and Information Technologies (CoDIT)
  • 2017
A constructive procedure of the model parameter identification is derived that possesses a high level of generality and does not involve unrealistic a priori assumptions that degenerate the entity of the initial identification problem statement like those presented in some referenced literature sources.

An Information Theoretic Approach to System Identification via Input / Output Signal Processing

The aim of the paper is to present a conceptual approach to identification of nonlinear stochastic systems based on information measures of dependence based on a parameterized description of the system model under study combined with minimum of the relative entropy method to derive the mutual information of the systems’ and model’s output variables.


A constructive procedure of the model parameter identification is derived that possesses a high level of generality and does not involve unreal a priori assumptions degenerating the entity of the initial identification problem statement like those ones presented in some referenced literature sources and revised in the present paper.

An information theoretic approach to the statistical linearization of MIMO stochastic systems

  • K. Chernyshov
  • Computer Science
    2009 IEEE Control Applications, (CCA) & Intelligent Control, (ISIC)
  • 2009
A constructive procedure of deriving a linear input/output model which is statistically equivalent to a multi input / multi output dynamic stochastic system driven by a white-noise Gaussian process is proposed.


Relations within entropy rate, mutual information rate and H ∞ entropy are discussed in both general control problem and classic tracking problem and give information theoretic interpretations for the minimum entropy H∞ control theory.

Information theoretic interpretations for H˞ entropy

Based on the studies on information transmission in discrete multivariable linear time invariant (LTI) system disturbed by stationary noise, relations within entropy rate, mutual information rate and

Using Informational Measures of Dependence in Statistical Linearization

A constructive procedure of creating the linear input-output model that is statistically equivalent to the nonlinear dynamic stochastic system with white-noise Gaussian input process was proposed.

Risk sensitive identification of linear stochastic systems

The main result of the paper is the minimization of the proposed new criterion with respect to the weight-matrix K over a feasible set EK, where the cost function is known to be finite.



Information Theory and an Extension of the Maximum Likelihood Principle

The classical maximum likelihood principle can be considered to be a method of asymptotic realization of an optimum estimate with respect to a very general information theoretic criterion to provide answers to many practical problems of statistical model fitting.

Optimization of stochastic linear systems with additive measurement and process noise using exponential performance criteria

The expected value of a multiplicative performance criterion, represented by the exponential of a quadratic function of the state and control variables, is minimized subject to a discrete stochastic

The statistical theory of linear systems

Publisher Summary The chapter discusses the development of a rather complete inferential theory for ARMAX models. The first problem in the development is the coordinatization of spaces of such

Risk-sensitive linear/quadratic/gaussian control

  • P. Whittle
  • Mathematics
    Advances in Applied Probability
  • 1981
The conventional linear/quadratic/Gaussian assumptions are modified in that minimisation of the expectation of cost G defined by (2) is replaced by minimisation of the criterion function (5). The

Recursive linear estimation in Krein spaces. I. Theory

We develop a self-contained theory for linear estimation in Krein spaces. The theory is based on simple concepts such as projections and matrix factorizations, and leads to an interesting connection


. A rigorous analysis is given of the asymptotic bias of the log maximum likelihood as an estimate of the expected log likelihood of the maximum likelihood model, when a linear model, such as an

On the Shannon theory of information transmission in the case of continuous signals

In addition to the scheduled program, the following two papers, by A. N. Kolmogorov and V. I. Siforov, were presented at the 1956 Symposium on Information Theory. However, the manuscripts were

Centralized and decentralized stochastic control problems with an exponential cost criterion

  • C. JaenschJ. Speyer
  • Mathematics
    Proceedings of the 27th IEEE Conference on Decision and Control
  • 1988
The authors consider stochastic control problems with an exponential cost criterion. In particular, the centralized linear-exponential-Gaussian (LEG) control problem is solved with the hypothesis