# Exponentially embedded families - new approaches to model order estimation

@article{Kay2005ExponentiallyEF, title={Exponentially embedded families - new approaches to model order estimation}, author={Steven M. Kay}, journal={IEEE Transactions on Aerospace and Electronic Systems}, year={2005}, volume={41}, pages={333-345} }

The use of exponential embedding of two or more probability density functions (pdfs) is introduced. Termed the exponentially embedded family (EEF) of pdfs, its properties are first examined and then it is applied to the problem of model order estimation. The proposed estimator is compared with the minimum description length (MDL) and is found to be superior for cases of practical interest. Also, we point out there is a relationship between the embedded family model order estimator and the…

## 101 Citations

A Geometrical Interpretation of Exponentially Embedded Families of Gaussian Probability Density Functions for Model Selection

- Mathematics, Computer ScienceIEEE Transactions on Signal Processing
- 2013

In this correspondence, a geometrical interpretation of the EEF is given and the sensitivity ofThe EEF approach to the choice of model origin in a Gaussian hypothesis testing framework is studied and optimality conditions for which the E EF using I-center achieves optimal performance in the Gaussian hypotheses testing framework are derived.

On the Exponentially Embedded Family (EEF) Rule for Model Order Selection

- Mathematics, Computer ScienceIEEE Signal Processing Letters
- 2012

This letter presents a generalized likelihood ratio (GLR)-based derivation of the recently proposed EEF rule in an attempt to cast EEF in the main stream of model order selection approaches and provides further insights into its theoretical foundations.

High-SNR model order selection using exponentially embedded family and its applications to curve fitting and clustering

- Mathematics, Computer Science2014 IEEE Symposium on Computational Intelligence and Data Mining (CIDM)
- 2014

Simulation results show that, with nuisance parameters, the new EEF outperforms the original EEF and Bayesian information criterion (BIC) at high SNR.

The penalty term of Exponentially Embedded Family is estimated mutual information

- Mathematics, Computer Science2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- 2017

It is shown that the EEF penalty term can be viewed as estimated mutual information (MI) between unknown parameters and received data from Bayesian viewpoints, as a result of an important relationship between Kullback-Leibler Divergence, signal-to-noise ratio (SNR) and MI in estimation/detection of random signals.

Exponentially embedded families for multimodal sensor processing

- Mathematics, Computer Science2010 IEEE International Conference on Acoustics, Speech and Signal Processing
- 2010

It is shown that the approximated PDF is asymptotically the one that is the closest to the unknown PDF in Kullback-Leibler (KL) divergence.

Probability Density Function Estimation Using the EEF With Application to Subset/Feature Selection

- Mathematics, Computer ScienceIEEE Transactions on Signal Processing
- 2016

Applications to subset selection in the context of multipath estimation as well as linear regression for machine learning are used to illustrate the practical utility of the proposed estimator.

Model Estimation and Classification Via Model Structure Determination

- Mathematics, Computer ScienceIEEE Transactions on Signal Processing
- 2013

It is shown that the model structure determination (MSD) is equivalent to the exponentially embedded family (EEF) for model order selection under some conditions and simulation results show that it outperforms the pseudo-maximum-likelihood (pseudo-ML) rule.

Information geometric probability models in statistical signal processing

- Computer Science
- 2016

It is shown that finding the extended Chernoff point can be guided by considering the orientation of the component PDFs, and that the use of this paradigm can lead to better ways to combine estimates in classical problems such as combining estimates of common means from separate Normal populations.

Estimating the order of multiple sinusoids model using exponentially embedded family rule: Large sample consistency

- Computer Science, MathematicsSignal Process.
- 2018

The large sample asymptotic properties of the Exponentially Embedded Family (EEF) based method for estimation of the order of the multiple sinusoids model are studied to establish that the estimator of model order using the EEF rule is large sample consistent.

Order estimation of 2-dimensional complex superimposed exponential signal model using exponentially embedded family (EEF) rule: large sample consistency properties

- Computer Science, MathematicsMultidimens. Syst. Signal Process.
- 2019

This paper uses the recently proposed exponentially embedded family (EEF) rule for estimating the order of the 2-dimensional signal model and proves that the EEF rule based estimator is consistent in large sample scenario.

## References

SHOWING 1-10 OF 22 REFERENCES

Conditional model order estimation

- Mathematics, Computer ScienceIEEE Trans. Signal Process.
- 2001

A new approach to model order selection that is able to discriminate between models by basing the decision on the part of the data that is independent of the model parameters and which can be shown to be consistent and to outperform the minimum description length estimator.

Asymptotic MAP criteria for model selection

- Mathematics, Computer ScienceIEEE Trans. Signal Process.
- 1998

This paper derives maximum a posteriori (MAP) rules for several different families of competing models and obtain forms that are similar to AIC and naive MDL, but for some families, however, it is found that the derived penalties are different.

Optimal Choice of AR and MA Parts in Autoregressive Moving Average Models

- Computer Science, MedicineIEEE Transactions on Pattern Analysis and Machine Intelligence
- 1982

The Bayesian method of choosing the best model for a given one-dimensional series among a finite number of candidates belonging to autoregressive, moving average, AR, ARMA, and other families is dealt with.

Estimation, inference, and specification analysis

- Computer Science, Mathematics
- 1993

The underlying motivation for maximum-likelihood estimation is explored, the interpretation of the MLE for misspecified probability models is treated, and the conditions under which parameters of interest can be consistently estimated despite misspecification are given.

A Method for Discriminating between Models

- Computer Science
- 1970

Using a combined distribution containing the component models as special cases, statistics are developed for testing for departures from one model in the direction of another and for testing the hypothesis that all models fit the data equally well.

Linear Statistical Inference and Its Applications

- Mathematics
- 1966

"C. R. Rao would be found in almost any statistician's list of five outstanding workers in the world of Mathematical Statistics today. His book represents a comprehensive account of the main body of…

The Advanced Theory of Statistics

- SociologyNature
- 1943

THIS very handsomely produced volume is one which it will be a pleasure to any mathematical statistician to possess. Mr. Kendall is indeed to be congratulated on the energy and, unswerving…

Modeling By Shortest Data Description*

- Computer ScienceAutom.
- 1978

The number of digits it takes to write down an observed sequence x1,...,xN of a time series depends on the model with its parameters that one assumes to have generated the observed data. Accordingly,…

Problems and solutions in theoretical statistics

- Mathematics, Computer Science
- 1978

Significance tests: simple null hypotheses, distribution-free and randomization tests, decision theory, and Bayesian methods.

Fundamentals of statistical signal processing: estimation theory

- Mathematics
- 1993

Minimum variance unbiased estimation Cramer-Rao lower bound linear models general minimum variance unbiased estimation best linear unbiased estimators maximum likelihood estimation least squares…