• DOI:10.32614/RJ-2021-089
• Corpus ID: 221534385

Robust and Efficient Optimization Using a Marquardt-Levenberg Algorithm with R Package marqLevAlg

@article{Philipps2021RobustAE,
  title={Robust and Efficient Optimization Using a Marquardt-Levenberg Algorithm with R Package marqLevAlg},
  author={Viviane Philipps and Boris P. Hejblum and M{\'e}lanie Prague and Daniel Commenges and C{\'e}cile Proust-Lima},
  journal={R J.},
  year={2021},
  volume={13},
  pages={273}
}

• V. Philipps, B. Hejblum, +2 authors C. Proust-Lima
• Published 8 September 2020
• Computer Science
• R J.

Optimization is an essential task in many computational problems. In statistical modelling for instance, in the absence of analytical solution, maximum likelihood estimators are often retrieved using iterative optimization algorithms. R software already includes a variety of optimizers from general-purpose optimization algorithms to more specific ones. Among Newton-like methods which have good convergence properties, the Marquardt-Levenberg algorithm (MLA) provides a particularly robust algorithm for solving optimization problems. Newton-like methods generally have two major limitations: (i) convergence criteria that are a little too loose, and do not ensure convergence towards a maximum, (ii) a calculation time that is often too long, which makes them unusable in complex problems. We propose in the marqLevAlg package an efficient and general implementation of a modified MLA combined with strict convergence criteria and parallel computations. Convergence to saddle points is avoided by using the relative distance to minimum/maximum criterion (RDM) in addition to the stability of the parameters and of the objective function. RDM exploits the first and second derivatives to compute the distance to a true local maximum. The independent multiple evaluations of the objective function at each iteration used for computing either first or second derivatives are called in parallel to allow a theoretical speed up to the square of the number of parameters. We show through the estimation of 7 relatively complex statistical models how parallel implementation can largely reduce computational time. We also show through the estimation of the same model using 3 different algorithms (BFGS of optim routine, an E-M, and MLA) the superior efficiency of MLA to correctly and consistently reach the maximum.