Skip to content

Levenberg-Marquardt algorithm

2 messages · Dermot MacSweeney, Brian Ripley

Original
Dermot MacSweeney
# 
Hi All,

Is the Levenberg-Marquardt algorithm available in R. This method combines the 
steepest descent algorithm and Newton's method.

Thanks in Advance,
Dermot MacSweeney.


**************************************************************

Dermot MacSweeney			NMRC, 
Email: dsweeney at nmrc.ucc.ie		Lee Maltings, 
Tel: +353 21 904178			Prospect Row, 
Fax: +353 21 270271			Cork, 
WWW: http://nmrc.ucc.ie			Ireland

**************************************************************


-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Brian Ripley
#
On Wed, 10 Jan 2001, Dermot MacSweeney wrote:

            

      
      
      
    

        
My books have it as a modified Gauss-Newton method for non-linear least
squares problems, and although its some years since I read the papers,
I am pretty sure that is what the author's tackled.  Another way to look
at it is that it uses ridge regression on the local linearizations
to find the step.

1) R does not specialised software for non-linear least squares: nls is
Gauss-Newton inside, I believe.  (I could not find that documented, except
for the S version.)  But NLS is a specialized problem and not that common
in my experience.

2) Levenberg-Marquardt is hardly competitive these days. It probably
was not by the time of Marquardt(1963) in statistical problems, which are
normally in optimization parlance `large residual problems'.  That is, the
best fit is not nearly exact.

For such problems general optimization algorithms (e.g. those in optim)
are often at least as good as specialized NLS methods.  So I did not
think it worth implementing the current crop of specialized NLS methods.

The Nocedal and Wright (1999) reference in ?optim is an excellent
overview: I had page 267 open as I wrote this.

`Performance of the Gauss-Newton and Levenberg-Marquardt algorithms is
usual poor in the large-residual case.'