The role of likelihood and entropy in incomplete-data problems: Applications to estimating point-process intensities and toeplitz constrained covariances

  title={The role of likelihood and entropy in incomplete-data problems: Applications to estimating point-process intensities and toeplitz constrained covariances},
  author={M. Miller and D. Snyder},
  journal={Proceedings of the IEEE},
The principle of maximum entropy has played an important role in the solution of problems in which the measurements correspond to moment constraints on some many-to-one mapping h(x). In this paper we explore its role in estimation problems in which the measured data are statistical observations and moment constraints on the observation function h(x) do not exist. We conclude that: 1) For the class of likelihood problems arising in a complete-incomplete data context in which the complete data x… Expand
Application of Likelihood and Entropy for Toeplitz Constrained Covariance Estimation
For the class of likelihood problems resulting from a complete-incomplete data specification in which the complete-data x are nonuniquely determined by the measured incomplete-data y via someExpand
Entropy, Information Theory, Information Geometry and Bayesian Inference in Data, Signal and Image Processing and Inverse Problems
The main inference tools using Bayes rule, the maximum entropy principle (MEP), information theory, relative entropy and the Kullback–Leibler (KL) divergence, Fisher information and its corresponding geometries are reviewed. Expand
Maximum likelihood estimation of Toeplitz-block-Toeplitz covariances in the presence of subspace interference
  • F. Nicolls, G. D. Jager
  • Mathematics, Computer Science
  • Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170)
  • 1998
The EM algorithm is extended to the case of two-dimensional signals, where spatial stationarity enforces a Toeplitz-block-Toeplitzer structure on the covariance matrix, and it is shown that this situation is amenable to a missing data interpretation, and can be incorporated into the EM iteration with moderate ease. Expand
Gaussian regression and power spectral density estimation with missing data: The MICROSCOPE space mission as a case study,
We present a Gaussian regression method for time series with missing data and stationary residuals of unknown power spectral density (PSD). The missing data are efficiently estimated by theirExpand
Statistical signal processing using a class of iterative estimation algorithms
Abstract : Many Signal Processing problems may be posed as statistical parameter estimation problems. A desired solution for the statistical problem is obtained by maximizing the Likelihood(ML), theExpand
ABSTRACT: This paper deals with the problem of estimating the first autocorrelation coefficients of a discrete process which has a band-limited spectrum in a way that accounts for the bandwidthExpand
The use of a stopping rule in iterative image reconstruction
During a study of the characteristics of the Maximum Likelihood Estimator (MLE) method of image reconstruction from Positron Emission Tomography (PET) data, we have found that the requirement thatExpand
Robust Estimation of Structured Covariance Matrix for Heavy-Tailed Elliptical Distributions
This paper proposes incorporating the prior structure information into Tyler's M-estimator and formulating the problem as minimizing the cost function of Tyler's estimator under the prior structural constraint, showing that the proposed estimator achieves a smaller estimation error than the benchmark estimators at a lower computational cost. Expand
Progress in structured covariance estimation
  • D. Fuhrmann
  • Mathematics
  • Fourth Annual ASSP Workshop on Spectrum Estimation and Modeling
  • 1988
The author summarizes recent work on the problem of computing maximum-likelihood estimates of structured covariance matrices, as it applies to problems in array processing and spectrum estimation.Expand
Relative entropy and the multivariable multidimensional moment problem
  • T. Georgiou
  • Mathematics, Computer Science
  • IEEE Transactions on Information Theory
  • 2006
The inverse problem of describing power spectra which are consistent with second-order statistics is discussed, which has been the main motivation behind the present work. Expand


On the rationale of maximum-entropy methods
  • E. Jaynes
  • Mathematics
  • Proceedings of the IEEE
  • 1982
We discuss the relations between maximum-entropy (MAXENT) and other methods of spectral analysis such as the Schuster, Blackman-Tukey, maximum-likelihood, Bayesian, and Autoregressive (AR, ARMA, orExpand
Nonparametric roughness penalties for probability densities
Given a number of observations xlt ...,xN, a nonparametric method is suggested for estimating the entire probability density curve. The method is to subtract a roughness penalty from the logExpand
Iterative algorithms for optimal signal reconstruction and parameter identification given noisy and incomplete data
We present a new approach to the problem of estimating multiple signal and parameter unknowns given noisy and incomplete data. Using cross-entropy, we fit a separable density to the given modelExpand
Maximum entropy and conditional probability
The following theorem is offered, which states that the conditional distribution of a given random variable X is the (normalized) product of the maximum entropy distribution and the initial distribution. Expand
Restoring with maximum likelihood and maximum entropy.
  • B. Frieden
  • Mathematics, Medicine
  • Journal of the Optical Society of America
  • 1972
A communication-theory model for the process of image formation is used and it is found that the most likely object has a maximum entropy and is represented by a restoring formula that is positive and not band limited. Expand
Restoring with maximum entropy. III. Poisson sources and backgrounds
The maximum entropy (ME) restoring formalism has previously been derived under the assumptions of (i) zero background and (ii) additive noise in the image. However, the noise in the signals from manyExpand
$I$-Divergence Geometry of Probability Distributions and Minimization Problems
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology andExpand
Minimum cross-entropy spectral analysis
The principle of minimum cross-entropy (minimum directed divergence, minimum discrimination information, minimum relative entropy) is summarized, discussed, and applied to the classical problem ofExpand
Estimation of Structured Covariance Matrices a Generalization of the Burg Technique
Covariance matrices from stationary time series are Toeplitz. In this case and many other situations, one knows that the actual covariance matrix belongs to a particular sub-class of covarianceExpand
Maximum-likelihood estimation applied to electron microscopic autoradiography
A new method for analysis of electron microscope autoradiographs is described which is based on the maximum-likelihood method of statistics for estimating the intensities of radioactivity inExpand