# System identification with information theoretic criteria

@inproceedings{Stoorvogel1995SystemIW, title={System identification with information theoretic criteria}, author={Anton A. Stoorvogel and Jan H. van Schuppen}, year={1995} }

Attention is focused in this paper on the approximation problem of system identification with information theoretic criteria. For a class of problems it is shown that the criterion of mutual information rate is identical to the criterion of exponential-of-quadratic cost and to $H_{infty$ entropy. In addition the relation between the likelihood function and divergence is explored. As a consequence of these relations a parameter estimator is derived by four methods for the approximation of a…

## 76 Citations

### Information-theoretic system identification

- Computer Science2017 4th International Conference on Control, Decision and Information Technologies (CoDIT)
- 2017

A constructive procedure of the model parameter identification is derived that possesses a high level of generality and does not involve unrealistic a priori assumptions that degenerate the entity of the initial identification problem statement like those presented in some referenced literature sources.

### An Information Theoretic Approach to System Identification via Input / Output Signal Processing

- Computer Science
- 2006

The aim of the paper is to present a conceptual approach to identification of nonlinear stochastic systems based on information measures of dependence based on a parameterized description of the system model under study combined with minimum of the relative entropy method to derive the mutual information of the systems’ and model’s output variables.

### INFORMATION THEORETIC IDENTIFICATION CRITERIA : APPROACHES AND ALTERNATIVES

- Computer Science, Mathematics
- 2005

A constructive procedure of the model parameter identification is derived that possesses a high level of generality and does not involve unreal a priori assumptions degenerating the entity of the initial identification problem statement like those ones presented in some referenced literature sources and revised in the present paper.

### An Information-Theoretic Approach to System Identification with Applying Tsallis Entropy

- Computer Science
- 2018

### An algebraic method for system reduction of stationary Gaussian systems

- Computer Science, Mathematics
- 2003

### An information theoretic approach to the statistical linearization of MIMO stochastic systems

- Computer Science2009 IEEE Control Applications, (CCA) & Intelligent Control, (ISIC)
- 2009

A constructive procedure of deriving a linear input/output model which is statistically equivalent to a multi input / multi output dynamic stochastic system driven by a white-noise Gaussian process is proposed.

### INFORMATION THEORETIC INTERPRETATIONS FOR H

- Computer Science
- 2005

Relations within entropy rate, mutual information rate and H ∞ entropy are discussed in both general control problem and classic tracking problem and give information theoretic interpretations for the minimum entropy H∞ control theory.

### Information theoretic interpretations for H˞ entropy

- Computer Science
- 2005

Based on the studies on information transmission in discrete multivariable linear time invariant (LTI) system disturbed by stationary noise, relations within entropy rate, mutual information rate and…

### Using Informational Measures of Dependence in Statistical Linearization

- Computer Science, Mathematics
- 2002

A constructive procedure of creating the linear input-output model that is statistically equivalent to the nonlinear dynamic stochastic system with white-noise Gaussian input process was proposed.

### Risk sensitive identification of linear stochastic systems

- Mathematics, Computer ScienceMath. Control. Signals Syst.
- 2005

The main result of the paper is the minimization of the proposed new criterion with respect to the weight-matrix K over a feasible set EK, where the cost function is known to be finite.

## References

SHOWING 1-10 OF 61 REFERENCES

### Information Theory and an Extension of the Maximum Likelihood Principle

- Mathematics
- 1973

The classical maximum likelihood principle can be considered to be a method of asymptotic realization of an optimum estimate with respect to a very general information theoretic criterion to provide answers to many practical problems of statistical model fitting.

### Optimization of stochastic linear systems with additive measurement and process noise using exponential performance criteria

- Mathematics
- 1974

The expected value of a multiplicative performance criterion, represented by the exponential of a quadratic function of the state and control variables, is minimized subject to a discrete stochastic…

### The statistical theory of linear systems

- Mathematics
- 1988

Publisher Summary The chapter discusses the development of a rather complete inferential theory for ARMAX models. The first problem in the development is the coordinatization of spaces of such…

### On the optimal control of stochastic systems with an exponential-of-integral performance index

- Mathematics, Computer Science
- 1981

### Risk-sensitive linear/quadratic/gaussian control

- MathematicsAdvances in Applied Probability
- 1981

The conventional linear/quadratic/Gaussian assumptions are modified in that minimisation of the expectation of cost G defined by (2) is replaced by minimisation of the criterion function (5). The…

### Canonical Correlation Analysis of Time Series and the Use of an Information Criterion

- Mathematics
- 1976

### Recursive linear estimation in Krein spaces. I. Theory

- MathematicsProceedings of 32nd IEEE Conference on Decision and Control
- 1993

We develop a self-contained theory for linear estimation in Krein spaces. The theory is based on simple concepts such as projections and matrix factorizations, and leads to an interesting connection…

### ON THE UNBIASEDNESS PROPERTY OF AIC FOR EXACT OR APPROXIMATING LINEAR STOCHASTIC TIME SERIES MODELS

- Mathematics
- 1985

. A rigorous analysis is given of the asymptotic bias of the log maximum likelihood as an estimate of the expected log likelihood of the maximum likelihood model, when a linear model, such as an…

### On the Shannon theory of information transmission in the case of continuous signals

- Computer ScienceIRE Trans. Inf. Theory
- 1956

In addition to the scheduled program, the following two papers, by A. N. Kolmogorov and V. I. Siforov, were presented at the 1956 Symposium on Information Theory. However, the manuscripts were…

### Centralized and decentralized stochastic control problems with an exponential cost criterion

- MathematicsProceedings of the 27th IEEE Conference on Decision and Control
- 1988

The authors consider stochastic control problems with an exponential cost criterion. In particular, the centralized linear-exponential-Gaussian (LEG) control problem is solved with the hypothesis…