Energetic Natural Gradient Descent

@inproceedings{Thomas2016EnergeticNG,
  title={Energetic Natural Gradient Descent},
  author={Philip S. Thomas and Bruno Castro da Silva and Christoph Dann and Emma Brunskill},
  booktitle={ICML},
  year={2016}
}
We propose a new class of algorithms for minimizing or maximizing functions of parametric probabilistic models. These new algorithms are natural gradient algorithms that leverage more information than prior methods by using a new metric tensor in place of the commonly used Fisher information matrix. This new metric tensor is derived by computing directions of steepest ascent where the distance between distributions is measured using an approximation of energy distance (as opposed to Kullback… CONTINUE READING