Optimal Jacobian accumulation is NP-complete

  title={Optimal Jacobian accumulation is NP-complete},
  author={Uwe Naumann},
  journal={Mathematical Programming},
  • U. Naumann
  • Published 30 November 2007
  • Mathematics
  • Mathematical Programming
We show that the problem of accumulating Jacobian matrices by using a minimal number of floating-point operations is NP-complete by reduction from Ensemble Computation. The proof makes use of the fact that, deviating from the state-of-the-art assumption, algebraic dependences can exist between the local partial derivatives. It follows immediately that the same problem for directional derivatives, adjoints, and higher derivatives is NP-complete, too. 

On the Computational Complexity of the Chain Rule of Differential Calculus

This work forms Chain Rule Differentiation and it is proved to be NP-complete, the fundamental prerequisite for computing accurate derivatives of composite functions which perform a potentially very large number of elemental function evaluations.

Derivation of Augmented Arithmetic for Computing Gradient, Hessian and Jacobian through Forward Mode AD Using Dual Numbers

The chain rule based differentiation arithmetic is used to find the Gradient and Hessian and Jacobian is evaluated using Gradient arithmetic by computing Gradient for components and is arranged in matrix form to give Jacobian value.

Optimization of Generalized Jacobian Chain Products without Memory Constraints

A dynamic programming algorithm is proposed for the minimization of the computational cost of such generalized Jacobian chain products without considering constraints on the available persistent system memory.

Hessian Chain Bracketing

This work proposes a dynamic programming formulation for the solution of Hessian Accumulation over a sub-search space and yields improvements by factors of ten and higher over the state of the art based on second-order tangent and adjoint algorithmic differentiation.

On lower bounds for optimal Jacobian accumulation

A new approach for computing the lower bounds for the optimal solution of the and problems, both of which are conjectured to be NP-hard.

A Dynamic Programming Heuristic for Dense Hessian Chain Bracketing

Motivated by second-order parameter sensitivity analysis of surrogate numerical models obtained through training and pruning of deep neural networks this paper focusses on bracketing of dense Hessian chain products with the aim of minimizing the total number of floating-point operations to be performed.

An Integer Programming Approach to Optimal Derivative Accumulation

This paper proposes to tackle the problem of vertex elimination with an integer programming technique, and develops an IP formulation for it, and has developed several bound-tightening and symmetry-breaking constraints to strengthen the basic IP formulation.

Algorithmic Differentiation of Numerical Methods : Tangent-Linear and Adjoint Direct Solvers for Systems of Linear Equations

The Algorithmic Differentiation of numerical simulation programs that contain calls to direct solvers for systems of n linear equations is considered and mathematical insight yields a reduction of the local computational complexity to O(n).

A Simple and Efficient Tensor Calculus

It is shown that using Ricci notation is not necessary for an efficient tensor calculus and developed an equally efficient method for the simpler Einstein notation, and it turns out that turning to Einstein notation enables further improvements that lead to even better efficiency.

From Semantics to Computer Science: Reversal strategies for adjoint algorithms

This paper makes a survey of the reversal strategies employed in recent tools and describes some of the more abstract formalizations used to justify these strategies.




A specialized constant folding algorithm is presented to decrease the size of the combinatorial problem in order to increase the efcienc y of heuristics for its solution and it is shown that this algorithm preserves optimality in the sense that an optimal solution for the reduced problem yields an objective value no worse than that of an optimal solutions for the original problem.

On the calculation of Jacobian matrices by the Markowitz rule

This work examines this vertex elimination problem and solves it approximately, using a greedy heuristic, the resulting Markowitz scheme for Jacobian evaluation to be more efficient than column by column or row by row evaluation using the forward or the reverse mode.

The Complexity of Partial Derivatives

Elimination techniques for cheap Jacobians

It is shown that vertex elimination in general does not lead to optimal Jacobian code, and two new elimination methods are introduced that demonstrate their superiority over the vertex elimination approach.

Optimal accumulation of Jacobian matrices by elimination methods on the dual computational graph

  • U. Naumann
  • Computer Science, Mathematics
    Math. Program.
  • 2004
Face elimination is introduced as the basic technique for accumulating Jacobian matrices by using a minimal number of arithmetic operations and its superiority over both edge and vertex elimination methods is shown.

Cheaper Jacobians by Simulated Annealing

Simulated annealing as a method for generating nearly optimal Jacobian code for combinatorial optimization problem based on vertex elimination in computational graphs.

Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition

This second edition has been updated and expanded to cover recent developments in applications and theory, including an elegant NP completeness argument by Uwe Naumann and a brief introduction to scarcity, a generalization of sparsity.

Compiling fast partial derivatives of functions given by algorithms

The system Jake described produces gradients significantly faster than numerical differencing for n > 8 and can handle algorithms Af with arbitrary flow of control.

Automatic differentiation : applications, theory, and implementations

Automatic Differentiation: A Tool for Variational Data Assimilation and Adjoint Sensitivity Analysis for Flood Modeling.

Computational differentiation : techniques, applications, and tools

This volume goes beyond the first volume published in 1991 (SIAM) in that it encompasses both the automatic transformation of computer programs as well as the methodologies for the efficient