# An extended collection of matrix derivative results for forward and reverse mode algorithmic dieren tiation

@inproceedings{Giles2008AnEC, title={An extended collection of matrix derivative results for forward and reverse mode algorithmic dieren tiation}, author={Michael B. Giles}, year={2008} }

This paper collects together a number of matrix derivative results which are very useful in forward and reverse mode algorithmic differentiation (AD). It highlights in particular the remarkable contribution of a 1948 paper by Dwyer and Macphail which derives the linear and adjoint sensitivities of a matrix product, inverse and determinant, and a number of related results motivated by applications in multivariate analysis in statistics.
This is an extended version of a paper which will appear…

## 60 Citations

Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation

- Mathematics
- 2008

This paper collects together a number of matrix derivative results which are very useful in forward and reverse mode algorithmic differentiation. It highlights in particular the remarkable…

On evaluating higher-order derivatives of the QR decomposition of tall matrices with full column rank in forward and reverse mode algorithmic differentiation

- Mathematics, Computer ScienceOptim. Methods Softw.
- 2012

We address the task of higher-order derivative evaluation of computer programs that contain QR decompositions of tall matrices with full column rank. The approach is a combination of univariate…

Algorithmic Differentiation of Linear Algebra Functions with Application in Optimum Experimental Design (Extended Version)

- Mathematics, Computer ScienceArXiv
- 2010

We derive algorithms for higher order derivative computation of the rectangular QR and eigenvalue decomposition of symmetric matrices with distinct eigenvalues in the forward and reverse mode of…

Derivatives of partial eigendecomposition of a real symmetric matrix for degenerate cases

- Physics, Computer ScienceArXiv
- 2020

The forward and backward derivatives of partial eigendecomposition, i.e. where it only obtains some of the eigenpairs, of a real symmetric matrix for degenerate cases are presented.

Algorithmic differentiation in Python with AlgoPy

- Computer ScienceJ. Comput. Sci.
- 2013

AlgoPy provides the means to compute derivatives of arbitrary order and Taylor approximations of such programs as NumPy, based on a combination of univariate Taylor polynomial arithmetic and matrix calculus in the (combined) forward/reverse mode of Algorithmic Differentiation (AD).

The Kähler Mean of Block-Toeplitz Matrices with Toeplitz Structured Blocks

- Mathematics, Computer ScienceSIAM J. Matrix Anal. Appl.
- 2016

The generalized barycenter is derived, or generalized Kahler mean, and a greedy approximation is shown to be close to the generalized mean with a significantly lower computational cost.

N A ] 4 S ep 2 01 9 Automatic Differentiation for Complex Valued SVD

- 2019

Automatic differentiation(AD) evaluates derivatives or gradients of any functions specified by computer programs[1]. It is implemented by propagating derivatives of primitive operations via chain…

The Bordering Method of the Cholesky Decomposition and its Backward Differentiation

- Computer Science
- 2017

The backward differentiation of the Cholesky decomposition by the bordering method is described and it is found that the resulting algorithm can be adapted to vector processing, as is also true of the algorithms developed from the inner product form and outer product form.

Differentiation of the Cholesky decomposition

- Mathematics, Computer ScienceArXiv
- 2016

New `blocked' algorithms, based on differentiating the Cholesky algorithm DPOTRF in the LAPACK library, are recommended, which uses `Level 3' matrix-matrix operations from BLAS, and so is cache-friendly and easy to parallelize.

A review of automatic differentiation and its efficient implementation

- Computer Science, MathematicsWiley Interdiscip. Rev. Data Min. Knowl. Discov.
- 2019

Automatic differentiation is a powerful tool to automate the calculation of derivatives and is preferable to more traditional methods, especially when differentiating complex algorithms and mathematical functions.

## References

SHOWING 1-10 OF 19 REFERENCES

Some Applications of Matrix Derivatives in Multivariate Analysis

- Mathematics
- 1967

Abstract It is claimed that the reasons for using matrices of derivatives, in appropriate situations, are as compelling as those for using matrices. This paper provides basic material for such use.…

Old and New Matrix Algebra Useful for Statistics

- Mathematics
- 2000

The partials with respect to the numerator are laid out according to the shape of Y while the partials with respect to the denominator are laid out according to the transpose of X. For example, dy/dx…

Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition

- Computer ScienceFrontiers in applied mathematics
- 2000

This second edition has been updated and expanded to cover recent developments in applications and theory, including an elegant NP completeness argument by Uwe Naumann and a brief introduction to scarcity, a generalization of sparsity.

An efficient overloaded implementation of forward mode automatic differentiation in MATLAB

- Mathematics, Computer ScienceTOMS
- 2006

The Mad package described here facilitates the evaluation of first derivatives of multidimensional functions that are defined by computer codes written in MATLAB through the separation of the linear combination of derivative vectors into a separate derivative vector class derivvec.

Matrix inversion algorithms by means of automatic differentiation

- Mathematics
- 1994

Abstract There are many matrix inversion algorithms, some being widely known and others not as widely known. We will show that some of known elaborate formulas for matrix inversion can be derived by…

ADMAT : Automatic differentiation in MATLAB using object oriented methods ∗

- 2007

Differentiation is one of the fundamental problems in numerical mathematics. The solution of many optimization problems and other applications require knowledge of the gradient, the Jacobian matrix,…

ADMIT-1: automatic differentiation and MATLAB interface toolbox

- Computer ScienceTOMS
- 2000

This article provides an introduction to the design and usage of ADMIT-1, a generic automatic differentiation tool that enables the computation of sparse Jacobian and Hessian matrices from a MATLAB environment.

The Scaling and Squaring Method for the Matrix Exponential Revisited

- Mathematics
- 2009

The scaling and squaring method is the most widely used method for computing the matrix exponential, not least because it is the method implemented in the MATLAB function expm. The method scales the…

Combining source transformation and operator overloading techniques to compute derivatives for MATLAB programs

- Computer ScienceProceedings. Second IEEE International Workshop on Source Code Analysis and Manipulation
- 2002

A novel software tool is proposed to automatically transform a given MATLAB program into another MATLab program capable of computing not only the original function but also user-specified derivatives of that function.

An Introduction to Multivariate Statistics

- Computer Science, MedicineCanadian journal of psychiatry. Revue canadienne de psychiatrie
- 1993

There are costs associated with these benefits, such as increased complexity, decreased power, multiple ways of answering the same question, and ambiguity in the allocation of shared variance.