Functional differentiation of computer programs

@inproceedings{Karczmarczuk1998FunctionalDO,
  title={Functional differentiation of computer programs},
  author={Jerzy Karczmarczuk},
  booktitle={ICFP '98},
  year={1998}
}
We present two purely functional implementations of the computational differentiation tools -- the well known numeric (not symbolic!) techniques which permit to compute pointwise derivatives of functions defined by computer programs economically and exactly. We show how the co-recursive (lazy) algorithm formulation permits to construct in a transparent and elegant manner the entire infinite tower of derivatives of higher order for any expressions present in the program, and we present a purely… 
Functional Differentiation of Computer Programs
TLDR
It is shown how the use of lazy evaluation permits a transparent and elegant construction of the entire infinite tower of derivatives of higher order for any expressions present in the program.
Scientific computation and functional programming
TLDR
This article shows how to perform simple abstract computations on state vectors and discusses the construction of lazy algorithms that enormously simplify manipulation of potentially infinite data structures or iterative processes.
Functional Coding of Differential Forms
TLDR
This paper generalizes the one-dimensional algorithmic differentiation formalism in functional sauce presented elsewhere, and shows how to implement in a simple way the domain of differential forms with the p-vector algebra, Hodge “star” operator, and the differentiation.
Lazy Codes For Scientific Computations
The lazy evaluation protocol defers the evaluation of a function arguments until the moment, when the function uses them. If such a function constructs composite data structures, it is possible to
Compiling to categories
TLDR
An implementation of the simply typed lambda-calculus as a collection of meaning-preserving transformations added to an existing compiler, such as GHC for Haskell, which appears to provide a compelling alternative to deeply embedded domain-specific languages.
Higher Order Automatic Differentiation of Higher Order Functions
TLDR
This work characterises a forward-mode AD method on a higher order language with algebraic data types as the unique structure preserving macro given a choice of derivatives for basic operations, and describes a rich semantics for differentiable programming, based on diffeological spaces.
Lazy multivariate higher-order forward-mode AD
A method is presented for computing all higher-order partial derivatives of a multivariate function R n → R. This method works by evaluating the function under a nonstandard interpretation, lifting
Functorial String Diagrams for Reverse-Mode Automatic Differentiation
TLDR
To give an efficient yet principled implementation of the AD algorithm, a sound and complete representation of hierarchical string diagrams is defined as a class of hierarchical hypergraphs the authors call hypernets.
Principles of Automatic Differentiation
TLDR
Different descriptions of the Forward Mode of AD, like the matrix-vector product based approach, the idea of lifting functions to the algebra of dual numbers, the method of Taylor series expansion on dual numbers and the application of the push-forward operator are summarised.
Provably correct, asymptotically efficient, higher-order reverse-mode automatic differentiation
In this paper, we give a simple and efficient implementation of reverse-mode automatic differentiation, which both extends easily to higher-order functions, and has run time and memory consumption
...
1
2
3
4
...

References

SHOWING 1-10 OF 57 REFERENCES
Functional Programming and Mathematical Objects
TLDR
It is shown how the lazy evaluation paradigms simplify the construction and the presentation of several algorithms dealing with iterative data types, such as power series or formal Pade expansion.
Functional Coding of Differential Forms
TLDR
This paper generalizes the one-dimensional algorithmic differentiation formalism in functional sauce presented elsewhere, and shows how to implement in a simple way the domain of differential forms with the p-vector algebra, Hodge “star” operator, and the differentiation.
Algorithm 755: ADOL-C: a package for the automatic differentiation of algorithms written in C/C++
The C++ package ADOL-C described here facilitates the evaluation of first and higher derivatives of vector functions that are defined by computer programs written in C or C++. The resulting
Generating Power of Lazy Semantics
Automatic differentiation bibliography
TLDR
This is a bibliography of work related to automatic differentiation, a technique for the fast, accurate propagation of derivative values using the chain rule, which is a fundamental tool for scientific computation with applications in optimization, nonlinear equations, non linear least squares approximation, stiff ordinary differential equation, partial differential equations, continuation methods, and sensitivity analysis.
Efficient Derivative Codes through Automatic Differentiation and Interface Contraction: An Application in Biostatistics
TLDR
This paper focuses on how to take advantage of the fact that the number of variables passed between subroutines frequently is small compared with the numberof variables with respect to which one wishes to differentiate, resulting in much more efficient code for the computation of derivatives.
Recipes for adjoint code construction
TLDR
The described method is based on a few basic principles, which permits the establishment of simple construction rules for adjoint statements and complete adjoint subprograms and is an implementation of the tangent linear and adjoint model compiler (TAMC).
Concrete mathematics - a foundation for computer science
From the Publisher: This book introduces the mathematics that supports advanced computer programming and the analysis of algorithms. The primary aim of its well-known authors is to provide a solid
Computational differentiation : techniques, applications, and tools
This volume goes beyond the first volume published in 1991 (SIAM) in that it encompasses both the automatic transformation of computer programs as well as the methodologies for the efficient
On the LambertW function
TLDR
A new discussion of the complex branches of W, an asymptotic expansion valid for all branches, an efficient numerical procedure for evaluating the function to arbitrary precision, and a method for the symbolic integration of expressions containing W are presented.
...
1
2
3
4
5
...