Levenberg-Marquardt algorithm
On Wed, 10 Jan 2001, Dermot MacSweeney wrote:
Hi All, Is the Levenberg-Marquardt algorithm available in R. This method combines the steepest descent algorithm and Newton's method.
My books have it as a modified Gauss-Newton method for non-linear least squares problems, and although its some years since I read the papers, I am pretty sure that is what the author's tackled. Another way to look at it is that it uses ridge regression on the local linearizations to find the step. 1) R does not specialised software for non-linear least squares: nls is Gauss-Newton inside, I believe. (I could not find that documented, except for the S version.) But NLS is a specialized problem and not that common in my experience. 2) Levenberg-Marquardt is hardly competitive these days. It probably was not by the time of Marquardt(1963) in statistical problems, which are normally in optimization parlance `large residual problems'. That is, the best fit is not nearly exact. For such problems general optimization algorithms (e.g. those in optim) are often at least as good as specialized NLS methods. So I did not think it worth implementing the current crop of specialized NLS methods. The Nocedal and Wright (1999) reference in ?optim is an excellent overview: I had page 267 open as I wrote this. `Performance of the Gauss-Newton and Levenberg-Marquardt algorithms is usual poor in the large-residual case.'
Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272860 (secr) Oxford OX1 3TG, UK Fax: +44 1865 272595 -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._