# Estimation of mutual information by the fuzzy histogram

@article{Haeri2014EstimationOM, title={Estimation of mutual information by the fuzzy histogram}, author={Maryam Amir Haeri and Mohammad Mehdi Ebadzadeh}, journal={Fuzzy Optimization and Decision Making}, year={2014}, volume={13}, pages={287-318} }

Mutual Information (MI) is an important dependency measure between random variables, due to its tight connection with information theory. It has numerous applications, both in theory and practice. However, when employed in practice, it is often necessary to estimate the MI from available data. There are several methods to approximate the MI, but arguably one of the simplest and most widespread techniques is the histogram-based approach. This paper suggests the use of fuzzy partitioning for the…

## 12 Citations

### IC-FNN: A Novel Fuzzy Neural Network With Interpretable, Intuitive, and Correlated-Contours Fuzzy Rules for Function Approximation

- Computer ScienceIEEE Transactions on Fuzzy Systems
- 2018

A novel fuzzy neural network with intuitive, interpretable, and correlated-contours fuzzy rules (IC-FNN), for function approximation, is presented and could construct more parsimonious structures with higher accuracy, in comparison to the existing methods.

### A robust estimator of mutual information for deep learning interpretability

- Computer ScienceArXiv
- 2022

GMM-MI (pronounced “Jimmie”), an algorithm based on Gaussian mixture models that can be applied to both discrete and continuous settings that quantifies both the level of disentanglement between the latent variables, and their association with relevant physical quantities, thus unlocking the interpretability of the latent representation.

### Fuzzy Evidential Network and Its Application as Medical Prognosis and Diagnosis Models

- Computer ScienceJ. Biomed. Informatics
- 2017

### NONLINEAR FUNCTION APPROXIMATION

- Computer Science
- 2021

The proposed model, based on a general Gaussian model able to construct different shapes, is successfully applied to real-world time-series predictions, regression problems, and nonlinear system identification and outperforms other methods with a more parsimonious structure.

### Independent component analysis approach using higher orders of Non-Gaussianity

- Computer Science2017 Iranian Conference on Electrical Engineering (ICEE)
- 2017

A novel algorithm has been proposed to handle cases with unknown probability distributions led to the vagueness of nonlinearity as a priori and manifested the highest accuracy in test data over other well-known BSS methods.

### IT2CFNN: An Interval Type-2 Correlation-Aware Fuzzy Neural Network to Construct Non-Separable Fuzzy Rules with Uncertain and Adaptive Shapes for Nonlinear Function Approximation

- Computer ScienceAppl. Soft Comput.
- 2022

### Semantic schema theory for genetic programming

- Computer ScienceApplied Intelligence
- 2015

This study proposes a schema theory which could be a more realistic model for GP and could be potentially employed for improving GP in practice, and introduces the concept of semantic schema, which partitions the search space according to semantics of trees, regardless of their syntactic variety.

### An improved semantic schema modeling for genetic programming

- Computer ScienceSoft Comput.
- 2018

An improved approach for developing the semantic schema in genetic programming by incorporating semantic awareness in schema theory is presented and new insight is provided on the relation between syntactic and semantic spaces.

### Semantic schema modeling for genetic programming using clustering of building blocks

- Computer ScienceApplied Intelligence
- 2017

This paper first defines the notion of semantics for a tree based on the mutual information between its output vector and the target and introduces semantic building blocks to facilitate the modeling of semantic schema, and proposes information based clustering to cluster the building blocks.

### Incipient Fault Detection for Chemical Processes Using Two-Dimensional Weighted SLKPCA

- EngineeringIndustrial & Engineering Chemistry Research
- 2019

An enhanced SLKPCA method, referred to as two-dimensional weighted SLK PCs, is proposed by integrating the sample and component weighting strategies, which puts large weights on the samples with strong fault information.

## References

SHOWING 1-10 OF 19 REFERENCES

### Interval-valued probability density estimation based on quasi-continuous histograms: Proof of the conjecture

- Computer ScienceFuzzy Sets Syst.
- 2011

### Fuzzy Histograms and Density Estimation

- MathematicsSMPS
- 2006

Histogram is the oldest and most widely used density estimator for presentation and exploration of observed univariate data and is said to be uniform or regular.

### Estimating mutual information.

- Computer SciencePhysical review. E, Statistical, nonlinear, and soft matter physics
- 2004

Two classes of improved estimators for mutual information M(X,Y), from samples of random points distributed according to some joint probability density mu(x,y), based on entropy estimates from k -nearest neighbor distances are presented.

### The mutual information: Detecting and evaluating dependencies between variables

- Computer ScienceECCB
- 2002

The findings show that the algorithms used so far may be quite substantially improved upon when dealing with small datasets, finite sample effects and other sources of potentially misleading results have to be taken into account.

### Canonical dependency analysis based on squared-loss mutual information

- Computer ScienceNeural Networks
- 2012

### On estimation of entropy and mutual information of continuous distributions

- Computer Science
- 1989

### Estimation of the Information by an Adaptive Partitioning of the Observation Space

- MathematicsIEEE Trans. Inf. Theory
- 1999

We demonstrate that it is possible to approximate the mutual information arbitrarily closely in probability by calculating the relative frequencies on appropriate partitions and achieving conditional…

### Justification and numerical realization of the uniform method for finding point estimates of interval elicited scaling constants

- MathematicsFuzzy Optim. Decis. Mak.
- 2008

An analytical justification and a numerical realization of the uniform method that finds point estimates of interval scaling constants that are uniformly distributed in their uncertainty intervals are proposed.