Fast Exact Multiplication by the Hessian

@article{Pearlmutter1994FastEM,
  title={Fast Exact Multiplication by the Hessian},
  author={Barak A. Pearlmutter},
  journal={Neural Computation},
  year={1994},
  volume={6},
  pages={147-160}
}
Just storing the Hessian H (the matrix of second derivatives 2E/wiwj of the error E with respect to each pair of weights) of a large neural network is difficult. Since a common use of a large matrix like H is to compute its product with various vectors, we derive a technique that directly calculates Hv, where v is an arbitrary vector. To calculate Hv, we first define a differential operator Rv{f(w)} = (/r)f(w rv)|r=0, note that Rv{w} = Hv and Rv{w} = v, and then apply Rv{} to the equations used… CONTINUE READING
Highly Influential
This paper has highly influenced 34 other papers. REVIEW HIGHLY INFLUENTIAL CITATIONS
Highly Cited
This paper has 472 citations. REVIEW CITATIONS

Citations

Publications citing this paper.

472 Citations

050'92'97'03'09'15
Citations per Year
Semantic Scholar estimates that this publication has 472 citations based on the available data.

See our FAQ for additional information.

Similar Papers

Loading similar papers…