# Functional differentiation of computer programs

@inproceedings{Karczmarczuk1998FunctionalDO, title={Functional differentiation of computer programs}, author={Jerzy Karczmarczuk}, booktitle={ICFP '98}, year={1998} }

We present two purely functional implementations of the computational differentiation tools -- the well known numeric (not symbolic!) techniques which permit to compute pointwise derivatives of functions defined by computer programs economically and exactly. We show how the co-recursive (lazy) algorithm formulation permits to construct in a transparent and elegant manner the entire infinite tower of derivatives of higher order for any expressions present in the program, and we present a purely…

## Topics from this paper

## 32 Citations

Functional Differentiation of Computer Programs

- Computer ScienceHigh. Order Symb. Comput.
- 2001

It is shown how the use of lazy evaluation permits a transparent and elegant construction of the entire infinite tower of derivatives of higher order for any expressions present in the program.

Scientific computation and functional programming

- Computer ScienceComput. Sci. Eng.
- 1999

This article shows how to perform simple abstract computations on state vectors and discusses the construction of lazy algorithms that enormously simplify manipulation of potentially infinite data structures or iterative processes.

Functional Coding of Differential Forms

- Computer Science
- 1999

This paper generalizes the one-dimensional algorithmic differentiation formalism in functional sauce presented elsewhere, and shows how to implement in a simple way the domain of differential forms with the p-vector algebra, Hodge “star” operator, and the differentiation.

Lazy Codes For Scientific Computations

- Mathematics
- 2000

The lazy evaluation protocol defers the evaluation of a function arguments until the moment, when the function uses them. If such a function constructs composite data structures, it is possible to…

Compiling to categories

- Computer ScienceProc. ACM Program. Lang.
- 2017

An implementation of the simply typed lambda-calculus as a collection of meaning-preserving transformations added to an existing compiler, such as GHC for Haskell, which appears to provide a compelling alternative to deeply embedded domain-specific languages.

Higher Order Automatic Differentiation of Higher Order Functions

- Computer ScienceArXiv
- 2021

This work characterises a forward-mode AD method on a higher order language with algebraic data types as the unique structure preserving macro given a choice of derivatives for basic operations, and describes a rich semantics for differentiable programming, based on diffeological spaces.

Lazy multivariate higher-order forward-mode AD

- Computer SciencePOPL '07
- 2007

A method is presented for computing all higher-order partial derivatives of a multivariate function R n → R. This method works by evaluating the function under a nonstandard interpretation, lifting…

Functorial String Diagrams for Reverse-Mode Automatic Differentiation

- Computer ScienceArXiv
- 2021

To give an efficient yet principled implementation of the AD algorithm, a sound and complete representation of hierarchical string diagrams is defined as a class of hierarchical hypergraphs the authors call hypernets.

Principles of Automatic Differentiation

- Computer Science
- 2014

Different descriptions of the Forward Mode of AD, like the matrix-vector product based approach, the idea of lifting functions to the algebra of dual numbers, the method of Taylor series expansion on dual numbers and the application of the push-forward operator are summarised.

Provably correct, asymptotically efficient, higher-order reverse-mode automatic differentiation

- Proceedings of the ACM on Programming Languages
- 2022

In this paper, we give a simple and efficient implementation of reverse-mode automatic differentiation, which both extends easily to higher-order functions, and has run time and memory consumption…

## References

SHOWING 1-10 OF 57 REFERENCES

Functional Programming and Mathematical Objects

- Mathematics, Computer ScienceFPLE
- 1995

It is shown how the lazy evaluation paradigms simplify the construction and the presentation of several algorithms dealing with iterative data types, such as power series or formal Pade expansion.

Functional Coding of Differential Forms

- Computer Science
- 1999

This paper generalizes the one-dimensional algorithmic differentiation formalism in functional sauce presented elsewhere, and shows how to implement in a simple way the domain of differential forms with the p-vector algebra, Hodge “star” operator, and the differentiation.

Algorithm 755: ADOL-C: a package for the automatic differentiation of algorithms written in C/C++

- Mathematics, Computer ScienceTOMS
- 1996

The C++ package ADOL-C described here facilitates the evaluation of first and higher derivatives of vector functions that are defined by computer programs written in C or C++. The resulting…

Automatic differentiation bibliography

- Computer Science
- 1992

This is a bibliography of work related to automatic differentiation, a technique for the fast, accurate propagation of derivative values using the chain rule, which is a fundamental tool for scientific computation with applications in optimization, nonlinear equations, non linear least squares approximation, stiff ordinary differential equation, partial differential equations, continuation methods, and sensitivity analysis.

Efficient Derivative Codes through Automatic Differentiation and Interface Contraction: An Application in Biostatistics

- Computer ScienceSIAM J. Sci. Comput.
- 1997

This paper focuses on how to take advantage of the fact that the number of variables passed between subroutines frequently is small compared with the numberof variables with respect to which one wishes to differentiate, resulting in much more efficient code for the computation of derivatives.

Recipes for adjoint code construction

- Mathematics, Computer ScienceTOMS
- 1998

The described method is based on a few basic principles, which permits the establishment of simple construction rules for adjoint statements and complete adjoint subprograms and is an implementation of the tangent linear and adjoint model compiler (TAMC).

Concrete mathematics - a foundation for computer science

- Computer Science, Mathematics
- 1989

From the Publisher:
This book introduces the mathematics that supports advanced computer programming and the analysis of algorithms. The primary aim of its well-known authors is to provide a solid…

Computational differentiation : techniques, applications, and tools

- Computer Science
- 1996

This volume goes beyond the first volume published in 1991 (SIAM) in that it encompasses both the automatic transformation of computer programs as well as the methodologies for the efficient…

On the LambertW function

- Mathematics, Computer ScienceAdv. Comput. Math.
- 1996

A new discussion of the complex branches of W, an asymptotic expansion valid for all branches, an efficient numerical procedure for evaluating the function to arbitrary precision, and a method for the symbolic integration of expressions containing W are presented.