# The role of likelihood and entropy in incomplete-data problems: Applications to estimating point-process intensities and toeplitz constrained covariances

@article{Miller1987TheRO, title={The role of likelihood and entropy in incomplete-data problems: Applications to estimating point-process intensities and toeplitz constrained covariances}, author={M. Miller and D. Snyder}, journal={Proceedings of the IEEE}, year={1987}, volume={75}, pages={892-907} }

The principle of maximum entropy has played an important role in the solution of problems in which the measurements correspond to moment constraints on some many-to-one mapping h(x). In this paper we explore its role in estimation problems in which the measured data are statistical observations and moment constraints on the observation function h(x) do not exist. We conclude that: 1) For the class of likelihood problems arising in a complete-incomplete data context in which the complete data x… Expand

#### 149 Citations

Application of Likelihood and Entropy for Toeplitz Constrained Covariance Estimation

- Mathematics
- 1988

For the class of likelihood problems resulting from a complete-incomplete data specification in which the complete-data x are nonuniquely determined by the measured incomplete-data y via some… Expand

Entropy, Information Theory, Information Geometry and Bayesian Inference in Data, Signal and Image Processing and Inverse Problems

- Mathematics, Computer Science
- Entropy
- 2015

The main inference tools using Bayes rule, the maximum entropy principle (MEP), information theory, relative entropy and the Kullback–Leibler (KL) divergence, Fisher information and its corresponding geometries are reviewed. Expand

Maximum likelihood estimation of Toeplitz-block-Toeplitz covariances in the presence of subspace interference

- Mathematics, Computer Science
- Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170)
- 1998

The EM algorithm is extended to the case of two-dimensional signals, where spatial stationarity enforces a Toeplitz-block-Toeplitzer structure on the covariance matrix, and it is shown that this situation is amenable to a missing data interpretation, and can be incorporated into the EM iteration with moderate ease. Expand

Gaussian regression and power spectral density estimation with missing data: The MICROSCOPE space mission as a case study,

- Physics
- 2016

We present a Gaussian regression method for time series with missing data and stationary residuals of unknown power spectral density (PSD). The missing data are efficiently estimated by their… Expand

Statistical signal processing using a class of iterative estimation algorithms

- Mathematics
- 1987

Abstract : Many Signal Processing problems may be posed as statistical parameter estimation problems. A desired solution for the statistical problem is obtained by maximizing the Likelihood(ML), the… Expand

ESTIMATION OF THE AUTOCORRELATION COEFFICIENTS OF BAND LIMITED SIGNALS. APPLICATION TO THE SOURCES LOCALIZATION PROBLEM

- Mathematics
- 1992

ABSTRACT: This paper deals with the problem of estimating the first autocorrelation coefficients of a discrete process which has a band-limited spectrum in a way that accounts for the bandwidth… Expand

The use of a stopping rule in iterative image reconstruction

- Mathematics
- 1991

During a study of the characteristics of the Maximum Likelihood Estimator (MLE) method of image reconstruction from Positron Emission Tomography (PET) data, we have found that the requirement that… Expand

Robust Estimation of Structured Covariance Matrix for Heavy-Tailed Elliptical Distributions

- Mathematics, Computer Science
- IEEE Transactions on Signal Processing
- 2016

This paper proposes incorporating the prior structure information into Tyler's M-estimator and formulating the problem as minimizing the cost function of Tyler's estimator under the prior structural constraint, showing that the proposed estimator achieves a smaller estimation error than the benchmark estimators at a lower computational cost. Expand

Progress in structured covariance estimation

- Mathematics
- Fourth Annual ASSP Workshop on Spectrum Estimation and Modeling
- 1988

The author summarizes recent work on the problem of computing maximum-likelihood estimates of structured covariance matrices, as it applies to problems in array processing and spectrum estimation.… Expand

Relative entropy and the multivariable multidimensional moment problem

- Mathematics, Computer Science
- IEEE Transactions on Information Theory
- 2006

The inverse problem of describing power spectra which are consistent with second-order statistics is discussed, which has been the main motivation behind the present work. Expand

#### References

SHOWING 1-10 OF 42 REFERENCES

On the rationale of maximum-entropy methods

- Mathematics
- Proceedings of the IEEE
- 1982

We discuss the relations between maximum-entropy (MAXENT) and other methods of spectral analysis such as the Schuster, Blackman-Tukey, maximum-likelihood, Bayesian, and Autoregressive (AR, ARMA, or… Expand

Nonparametric roughness penalties for probability densities

Given a number of observations xlt ...,xN, a nonparametric method is suggested for estimating the entire probability density curve. The method is to subtract a roughness penalty from the log… Expand

Iterative algorithms for optimal signal reconstruction and parameter identification given noisy and incomplete data

- Mathematics, Computer Science
- ICASSP
- 1982

We present a new approach to the problem of estimating multiple signal and parameter unknowns given noisy and incomplete data. Using cross-entropy, we fit a separable density to the given model… Expand

Maximum entropy and conditional probability

- Mathematics, Computer Science
- IEEE Trans. Inf. Theory
- 1981

The following theorem is offered, which states that the conditional distribution of a given random variable X is the (normalized) product of the maximum entropy distribution and the initial distribution. Expand

Restoring with maximum likelihood and maximum entropy.

- Mathematics, Medicine
- Journal of the Optical Society of America
- 1972

A communication-theory model for the process of image formation is used and it is found that the most likely object has a maximum entropy and is represented by a restoring formula that is positive and not band limited. Expand

Restoring with maximum entropy. III. Poisson sources and backgrounds

- Physics
- 1978

The maximum entropy (ME) restoring formalism has previously been derived under the assumptions of (i) zero background and (ii) additive noise in the image. However, the noise in the signals from many… Expand

$I$-Divergence Geometry of Probability Distributions and Minimization Problems

- Mathematics
- 1975

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and… Expand

Minimum cross-entropy spectral analysis

- Mathematics
- 1981

The principle of minimum cross-entropy (minimum directed divergence, minimum discrimination information, minimum relative entropy) is summarized, discussed, and applied to the classical problem of… Expand

Estimation of Structured Covariance Matrices a Generalization of the Burg Technique

- Mathematics
- 1983

Covariance matrices from stationary time series are Toeplitz. In this case and many other situations, one knows that the actual covariance matrix belongs to a particular sub-class of covariance… Expand

Maximum-likelihood estimation applied to electron microscopic autoradiography

- Mathematics
- 1985

A new method for analysis of electron microscope autoradiographs is described which is based on the maximum-likelihood method of statistics for estimating the intensities of radioactivity in… Expand