Exponentially embedded families - new approaches to model order estimation

@article{Kay2005ExponentiallyEF,
  title={Exponentially embedded families - new approaches to model order estimation},
  author={Steven M. Kay},
  journal={IEEE Transactions on Aerospace and Electronic Systems},
  year={2005},
  volume={41},
  pages={333-345}
}
  • S. Kay
  • Published 4 April 2005
  • Mathematics
  • IEEE Transactions on Aerospace and Electronic Systems
The use of exponential embedding of two or more probability density functions (pdfs) is introduced. Termed the exponentially embedded family (EEF) of pdfs, its properties are first examined and then it is applied to the problem of model order estimation. The proposed estimator is compared with the minimum description length (MDL) and is found to be superior for cases of practical interest. Also, we point out there is a relationship between the embedded family model order estimator and the… 

Figures from this paper

A Geometrical Interpretation of Exponentially Embedded Families of Gaussian Probability Density Functions for Model Selection
  • R. Costa, S. Kay
  • Mathematics, Computer Science
    IEEE Transactions on Signal Processing
  • 2013
TLDR
In this correspondence, a geometrical interpretation of the EEF is given and the sensitivity ofThe EEF approach to the choice of model origin in a Gaussian hypothesis testing framework is studied and optimality conditions for which the E EF using I-center achieves optimal performance in the Gaussian hypotheses testing framework are derived.
On the Exponentially Embedded Family (EEF) Rule for Model Order Selection
  • P. Stoica, P. Babu
  • Mathematics, Computer Science
    IEEE Signal Processing Letters
  • 2012
TLDR
This letter presents a generalized likelihood ratio (GLR)-based derivation of the recently proposed EEF rule in an attempt to cast EEF in the main stream of model order selection approaches and provides further insights into its theoretical foundations.
High-SNR model order selection using exponentially embedded family and its applications to curve fitting and clustering
TLDR
Simulation results show that, with nuisance parameters, the new EEF outperforms the original EEF and Bayesian information criterion (BIC) at high SNR.
The penalty term of Exponentially Embedded Family is estimated mutual information
  • Zhenghan Zhu, S. Kay
  • Mathematics, Computer Science
    2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • 2017
TLDR
It is shown that the EEF penalty term can be viewed as estimated mutual information (MI) between unknown parameters and received data from Bayesian viewpoints, as a result of an important relationship between Kullback-Leibler Divergence, signal-to-noise ratio (SNR) and MI in estimation/detection of random signals.
Exponentially embedded families for multimodal sensor processing
  • S. Kay, Quan Ding
  • Mathematics, Computer Science
    2010 IEEE International Conference on Acoustics, Speech and Signal Processing
  • 2010
TLDR
It is shown that the approximated PDF is asymptotically the one that is the closest to the unknown PDF in Kullback-Leibler (KL) divergence.
Probability Density Function Estimation Using the EEF With Application to Subset/Feature Selection
TLDR
Applications to subset selection in the context of multipath estimation as well as linear regression for machine learning are used to illustrate the practical utility of the proposed estimator.
Model Estimation and Classification Via Model Structure Determination
  • S. Kay, Quan Ding
  • Mathematics, Computer Science
    IEEE Transactions on Signal Processing
  • 2013
TLDR
It is shown that the model structure determination (MSD) is equivalent to the exponentially embedded family (EEF) for model order selection under some conditions and simulation results show that it outperforms the pseudo-maximum-likelihood (pseudo-ML) rule.
Information geometric probability models in statistical signal processing
TLDR
It is shown that finding the extended Chernoff point can be guided by considering the orientation of the component PDFs, and that the use of this paradigm can lead to better ways to combine estimates in classical problems such as combining estimates of common means from separate Normal populations.
Estimating the order of multiple sinusoids model using exponentially embedded family rule: Large sample consistency
TLDR
The large sample asymptotic properties of the Exponentially Embedded Family (EEF) based method for estimation of the order of the multiple sinusoids model are studied to establish that the estimator of model order using the EEF rule is large sample consistent.
Order estimation of 2-dimensional complex superimposed exponential signal model using exponentially embedded family (EEF) rule: large sample consistency properties
TLDR
This paper uses the recently proposed exponentially embedded family (EEF) rule for estimating the order of the 2-dimensional signal model and proves that the EEF rule based estimator is consistent in large sample scenario.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 22 REFERENCES
Conditional model order estimation
  • S. Kay
  • Mathematics, Computer Science
    IEEE Trans. Signal Process.
  • 2001
TLDR
A new approach to model order selection that is able to discriminate between models by basing the decision on the part of the data that is independent of the model parameters and which can be shown to be consistent and to outperform the minimum description length estimator.
Asymptotic MAP criteria for model selection
  • P. Djurić
  • Mathematics, Computer Science
    IEEE Trans. Signal Process.
  • 1998
TLDR
This paper derives maximum a posteriori (MAP) rules for several different families of competing models and obtain forms that are similar to AIC and naive MDL, but for some families, however, it is found that the derived penalties are different.
Optimal Choice of AR and MA Parts in Autoregressive Moving Average Models
  • R. Kashyap
  • Computer Science, Medicine
    IEEE Transactions on Pattern Analysis and Machine Intelligence
  • 1982
TLDR
The Bayesian method of choosing the best model for a given one-dimensional series among a finite number of candidates belonging to autoregressive, moving average, AR, ARMA, and other families is dealt with.
Estimation, inference, and specification analysis
  • H. White
  • Computer Science, Mathematics
  • 1993
TLDR
The underlying motivation for maximum-likelihood estimation is explored, the interpretation of the MLE for misspecified probability models is treated, and the conditions under which parameters of interest can be consistently estimated despite misspecification are given.
A Method for Discriminating between Models
TLDR
Using a combined distribution containing the component models as special cases, statistics are developed for testing for departures from one model in the direction of another and for testing the hypothesis that all models fit the data equally well.
Linear Statistical Inference and Its Applications
"C. R. Rao would be found in almost any statistician's list of five outstanding workers in the world of Mathematical Statistics today. His book represents a comprehensive account of the main body of
The Advanced Theory of Statistics
THIS very handsomely produced volume is one which it will be a pleasure to any mathematical statistician to possess. Mr. Kendall is indeed to be congratulated on the energy and, unswerving
Modeling By Shortest Data Description*
The number of digits it takes to write down an observed sequence x1,...,xN of a time series depends on the model with its parameters that one assumes to have generated the observed data. Accordingly,
Problems and solutions in theoretical statistics
TLDR
Significance tests: simple null hypotheses, distribution-free and randomization tests, decision theory, and Bayesian methods.
Fundamentals of statistical signal processing: estimation theory
Minimum variance unbiased estimation Cramer-Rao lower bound linear models general minimum variance unbiased estimation best linear unbiased estimators maximum likelihood estimation least squares
...
1
2
3
...