• Corpus ID: 17431500

An extended collection of matrix derivative results for forward and reverse mode algorithmic dieren tiation

  title={An extended collection of matrix derivative results for forward and reverse mode algorithmic dieren tiation},
  author={Michael B. Giles},
This paper collects together a number of matrix derivative results which are very useful in forward and reverse mode algorithmic differentiation (AD). It highlights in particular the remarkable contribution of a 1948 paper by Dwyer and Macphail which derives the linear and adjoint sensitivities of a matrix product, inverse and determinant, and a number of related results motivated by applications in multivariate analysis in statistics. This is an extended version of a paper which will appear… 
Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation
This paper collects together a number of matrix derivative results which are very useful in forward and reverse mode algorithmic differentiation. It highlights in particular the remarkable
On evaluating higher-order derivatives of the QR decomposition of tall matrices with full column rank in forward and reverse mode algorithmic differentiation
We address the task of higher-order derivative evaluation of computer programs that contain QR decompositions of tall matrices with full column rank. The approach is a combination of univariate
Algorithmic Differentiation of Linear Algebra Functions with Application in Optimum Experimental Design (Extended Version)
We derive algorithms for higher order derivative computation of the rectangular QR and eigenvalue decomposition of symmetric matrices with distinct eigenvalues in the forward and reverse mode of
Derivatives of partial eigendecomposition of a real symmetric matrix for degenerate cases
  • M. Kasim
  • Physics, Computer Science
  • 2020
The forward and backward derivatives of partial eigendecomposition, i.e. where it only obtains some of the eigenpairs, of a real symmetric matrix for degenerate cases are presented.
Algorithmic differentiation in Python with AlgoPy
AlgoPy provides the means to compute derivatives of arbitrary order and Taylor approximations of such programs as NumPy, based on a combination of univariate Taylor polynomial arithmetic and matrix calculus in the (combined) forward/reverse mode of Algorithmic Differentiation (AD).
The Kähler Mean of Block-Toeplitz Matrices with Toeplitz Structured Blocks
The generalized barycenter is derived, or generalized Kahler mean, and a greedy approximation is shown to be close to the generalized mean with a significantly lower computational cost.
N A ] 4 S ep 2 01 9 Automatic Differentiation for Complex Valued SVD
Automatic differentiation(AD) evaluates derivatives or gradients of any functions specified by computer programs[1]. It is implemented by propagating derivatives of primitive operations via chain
The Bordering Method of the Cholesky Decomposition and its Backward Differentiation
The backward differentiation of the Cholesky decomposition by the bordering method is described and it is found that the resulting algorithm can be adapted to vector processing, as is also true of the algorithms developed from the inner product form and outer product form.
Differentiation of the Cholesky decomposition
New `blocked' algorithms, based on differentiating the Cholesky algorithm DPOTRF in the LAPACK library, are recommended, which uses `Level 3' matrix-matrix operations from BLAS, and so is cache-friendly and easy to parallelize.
A review of automatic differentiation and its efficient implementation
  • C. Margossian
  • Computer Science, Mathematics
    Wiley Interdiscip. Rev. Data Min. Knowl. Discov.
  • 2019
Automatic differentiation is a powerful tool to automate the calculation of derivatives and is preferable to more traditional methods, especially when differentiating complex algorithms and mathematical functions.


Some Applications of Matrix Derivatives in Multivariate Analysis
Abstract It is claimed that the reasons for using matrices of derivatives, in appropriate situations, are as compelling as those for using matrices. This paper provides basic material for such use.
Old and New Matrix Algebra Useful for Statistics
The partials with respect to the numerator are laid out according to the shape of Y while the partials with respect to the denominator are laid out according to the transpose of X. For example, dy/dx
Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition
This second edition has been updated and expanded to cover recent developments in applications and theory, including an elegant NP completeness argument by Uwe Naumann and a brief introduction to scarcity, a generalization of sparsity.
An efficient overloaded implementation of forward mode automatic differentiation in MATLAB
  • S. Forth
  • Mathematics, Computer Science
  • 2006
The Mad package described here facilitates the evaluation of first derivatives of multidimensional functions that are defined by computer codes written in MATLAB through the separation of the linear combination of derivative vectors into a separate derivative vector class derivvec.
Matrix inversion algorithms by means of automatic differentiation
Abstract There are many matrix inversion algorithms, some being widely known and others not as widely known. We will show that some of known elaborate formulas for matrix inversion can be derived by
ADMAT : Automatic differentiation in MATLAB using object oriented methods ∗
Differentiation is one of the fundamental problems in numerical mathematics. The solution of many optimization problems and other applications require knowledge of the gradient, the Jacobian matrix,
ADMIT-1: automatic differentiation and MATLAB interface toolbox
This article provides an introduction to the design and usage of ADMIT-1, a generic automatic differentiation tool that enables the computation of sparse Jacobian and Hessian matrices from a MATLAB environment.
The Scaling and Squaring Method for the Matrix Exponential Revisited
The scaling and squaring method is the most widely used method for computing the matrix exponential, not least because it is the method implemented in the MATLAB function expm. The method scales the
Combining source transformation and operator overloading techniques to compute derivatives for MATLAB programs
A novel software tool is proposed to automatically transform a given MATLAB program into another MATLab program capable of computing not only the original function but also user-specified derivatives of that function.
An Introduction to Multivariate Statistics
  • D. Streiner
  • Computer Science, Medicine
    Canadian journal of psychiatry. Revue canadienne de psychiatrie
  • 1993
There are costs associated with these benefits, such as increased complexity, decreased power, multiple ways of answering the same question, and ambiguity in the allocation of shared variance.