Optimize question

Hi, 

Nelder and Mead is slow to converge but has the advantage that objective function derivatives need not be calculated. 

Sometimes speed to convergence is dependent on the problem in hand, so I suggest you set up a test optimisation and try each method in turn to benchmark each method. 


Regards

Wayne



-----Original Message-----
From: r-sig-finance-bounces at stat.math.ethz.ch
[mailto:r-sig-finance-bounces at stat.math.ethz.ch]On Behalf Of
Matthias.Koberstein at hsbctrinkaus.de
Sent: 16 July 2008 15:49
To: r-sig-finance at stat.math.ethz.ch
Subject: [R-SIG-Finance] Optimize question



Hi,

I use the command optim and optimize in a function.
Unfortunatley the standard method needs a lot of time and in accordance to
the manual is the slowest (Nelder-Maed).
A more "dirty" optimization qould be sufficient for my purposes as long as
it is faster. The function provides 4 other methods ("BFGS", "CG",
"L-BFGS-B", "SANN")
but which one is the fastest? Does anyone have eperience with that?

Thank you very much in advance

Matthias

**** Ressourcen schonen, weniger drucken - Think before you print! ****

---------------------------------------------------------------------
Diese E-Mail sowie eventuelle Anh?nge enthalten vertrauliche und / oder
rechtlich gesch?tzte Informationen. Wenn Sie nicht der richtige Adressat
sind oder diese E-Mail irrt?mlich erhalten haben, informieren Sie bitte
sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren
oder Speichern sowie die unbefugte Weitergabe dieser E-Mail sind nicht
gestattet.

This e-mail and any attachments may contain confidential...{{dropped:18}}

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

Also note that there is convergence, and convergence
to an answer that is good enough.


Patrick Burns
patrick at burns-stat.com
+44 (0)20 8525 0696
http://www.burns-stat.com
(home of S Poetry and "A Guide for the Unwilling S User")

Hi, 

Nelder and Mead is slow to converge but has the advantage that objective function derivatives need not be calculated. 

Sometimes speed to convergence is dependent on the problem in hand, so I suggest you set up a test optimisation and try each method in turn to benchmark each method. 


Regards

Wayne



-----Original Message-----
From: r-sig-finance-bounces at stat.math.ethz.ch
[mailto:r-sig-finance-bounces at stat.math.ethz.ch]On Behalf Of
Matthias.Koberstein at hsbctrinkaus.de
Sent: 16 July 2008 15:49
To: r-sig-finance at stat.math.ethz.ch
Subject: [R-SIG-Finance] Optimize question



Hi,

I use the command optim and optimize in a function.
Unfortunatley the standard method needs a lot of time and in accordance to
the manual is the slowest (Nelder-Maed).
A more "dirty" optimization qould be sufficient for my purposes as long as
it is faster. The function provides 4 other methods ("BFGS", "CG",
"L-BFGS-B", "SANN")
but which one is the fastest? Does anyone have eperience with that?

Thank you very much in advance

Matthias

**** Ressourcen schonen, weniger drucken - Think before you print! ****

---------------------------------------------------------------------
Diese E-Mail sowie eventuelle Anh?nge enthalten vertrauliche und / oder
rechtlich gesch?tzte Informationen. Wenn Sie nicht der richtige Adressat
sind oder diese E-Mail irrt?mlich erhalten haben, informieren Sie bitte
sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren
oder Speichern sowie die unbefugte Weitergabe dieser E-Mail sind nicht
gestattet.

This e-mail and any attachments may contain confidential...{{dropped:18}}

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

This may depend on your problem: how many dimensions do you have, is it a constrained or unconstrained optimization, does your function have smooth first (and second) derivatives, can you compute them analytically, etc.?

From: Matthias.Koberstein at hsbctrinkaus.de <Matthias.Koberstein at hsbctrinkaus.de>
Subject: [R-SIG-Finance] Optimize question
To: r-sig-finance at stat.math.ethz.ch
Received: Thursday, 17 July, 2008, 12:48 AM
Hi,

I use the command optim and optimize in a function.
Unfortunatley the standard method needs a lot of time and
in accordance to
the manual is the slowest (Nelder-Maed).
A more "dirty" optimization qould be sufficient
for my purposes as long as
it is faster. The function provides 4 other methods
("BFGS", "CG",
"L-BFGS-B", "SANN")
but which one is the fastest? Does anyone have eperience
with that?

Thank you very much in advance

Matthias

**** Ressourcen schonen, weniger drucken - Think before you
print! ****

---------------------------------------------------------------------
Diese E-Mail sowie eventuelle Anh?nge enthalten
vertrauliche und / oder
rechtlich gesch?tzte Informationen. Wenn Sie nicht der
richtige Adressat
sind oder diese E-Mail irrt?mlich erhalten haben,
informieren Sie bitte
sofort den Absender und vernichten Sie diese Mail. Das
unerlaubte Kopieren
oder Speichern sowie die unbefugte Weitergabe dieser E-Mail
sind nicht
gestattet.

This e-mail and any attachments may contain
confidential...{{dropped:18}}

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

Dear all,

thank you for your help regarding the optim speed question.
I just wanted to share my results with you.
After testing all optimization methods, the BFGS algorithm proved the
fastest by quite a bit.
The algorithms performances on a three dimensional problem with ~2200
observations were

BFGS 0:44 min.
L-BFGS -B 1:06 min
NM: 1:28 minutes
CS >3 min.
SANN >3 min.

(all with default convergence)

Matthias S. Koberstein
__________________________________
HSBC Trinkaus
Structured Solutions Group
K?nigsalle 21/23, 40212 D?sseldorf

Phone: +49 211 910 4412
e-mail: matthias.koberstein at hsbctrinkaus.de



**** Ressourcen schonen, weniger drucken - Think before you print! ****

---------------------------------------------------------------------
Diese E-Mail sowie eventuelle Anh?nge enthalten vertrauliche und / oder
rechtlich gesch?tzte Informationen. Wenn Sie nicht der richtige Adressat
sind oder diese E-Mail irrt?mlich erhalten haben, informieren Sie bitte
sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren
oder Speichern sowie die unbefugte Weitergabe dieser E-Mail sind nicht
gestattet.

This e-mail and any attachments may contain confidential and / or
privileged information. If you are not the intended recipient or have
received this e-mail in error, please notify the sender immediately and
destroy this e-mail . Any unauthorized copying, storing, disclosure or
distribution of the contents of this e-mail is strictly forbidden.

---------------------------------------------------------------------
HSBC Trinkaus & Burkhardt AG
Sitz: D?sseldorf, K?nigsallee 21/23, 40212 D?sseldorf, Handelsregister:
Amtsgericht D?sseldorf HRB 54447
Mitglieder des Vorstands: Andreas Schmitz (Sprecher), Paul Hagen, Dr. Olaf
Huth, Carola Gr?fin v. Schmettow
Vorsitzender des Aufsichtsrats: Dr. Sieghardt Rometsch


                                                                           
             Moshe Olshansky                                               
             <m_olshansky at yaho                                             
             o.com>                                                     An 
             Gesendet von:               r-sig-finance at stat.math.ethz.ch,  
             r-sig-finance-bou           Matthias.Koberstein at hsbctrinkaus. 
             nces at stat.math.et           de                                
             hz.ch                                                   Kopie 
                                                                           
             Fax-Deckblatt:                                          Thema 
             HSBCTuB                     Re: [R-SIG-Finance] Optimize      
             17.07.2008 01:35            question                          
                                                                           
              Bitte antworten                                              
                    an                                                     
             m_olshansky at yahoo                                             
                   .com                                                    
                                                                           
                                                                           




This may depend on your problem: how many dimensions do you have, is it a
constrained or unconstrained optimization, does your function have smooth
first (and second) derivatives, can you compute them analytically, etc.?


--- On Thu, 17/7/08, Matthias.Koberstein at hsbctrinkaus.de

From: Matthias.Koberstein at hsbctrinkaus.de

<Matthias.Koberstein at hsbctrinkaus.de>

Subject: [R-SIG-Finance] Optimize question
To: r-sig-finance at stat.math.ethz.ch
Received: Thursday, 17 July, 2008, 12:48 AM
Hi,

I use the command optim and optimize in a function.
Unfortunatley the standard method needs a lot of time and
in accordance to
the manual is the slowest (Nelder-Maed).
A more "dirty" optimization qould be sufficient
for my purposes as long as
it is faster. The function provides 4 other methods
("BFGS", "CG",
"L-BFGS-B", "SANN")
but which one is the fastest? Does anyone have eperience
with that?

Thank you very much in advance

Matthias

**** Ressourcen schonen, weniger drucken - Think before you
print! ****

---------------------------------------------------------------------
Diese E-Mail sowie eventuelle Anh?nge enthalten
vertrauliche und / oder
rechtlich gesch?tzte Informationen. Wenn Sie nicht der
richtige Adressat
sind oder diese E-Mail irrt?mlich erhalten haben,
informieren Sie bitte
sofort den Absender und vernichten Sie diese Mail. Das
unerlaubte Kopieren
oder Speichern sowie die unbefugte Weitergabe dieser E-Mail
sind nicht
gestattet.

This e-mail and any attachments may contain
confidential...{{dropped:18}}

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

Thanks for the feedback,

Interesting to see the difference in the methods. 

If you play with the convergence criteria then you may well improve on these times. Sometimes the optim algortithms search hard for a very small change in objective value and effectiveley only change the final parameter estimates by miniscule amounts. 

Regards

Wayne





-----Original Message-----
From: r-sig-finance-bounces at stat.math.ethz.ch
[mailto:r-sig-finance-bounces at stat.math.ethz.ch]On Behalf Of
Matthias.Koberstein at hsbctrinkaus.de
Sent: 17 July 2008 13:41
To: m_olshansky at yahoo.com
Cc: r-sig-finance at stat.math.ethz.ch;
r-sig-finance-bounces at stat.math.ethz.ch
Subject: [R-SIG-Finance] Antwort: Re: Optimize question


Dear all,

thank you for your help regarding the optim speed question.
I just wanted to share my results with you.
After testing all optimization methods, the BFGS algorithm proved the
fastest by quite a bit.
The algorithms performances on a three dimensional problem with ~2200
observations were

BFGS 0:44 min.
L-BFGS -B 1:06 min
NM: 1:28 minutes
CS >3 min.
SANN >3 min.

(all with default convergence)

Matthias S. Koberstein
__________________________________
HSBC Trinkaus
Structured Solutions Group
K?nigsalle 21/23, 40212 D?sseldorf

Phone: +49 211 910 4412
e-mail: matthias.koberstein at hsbctrinkaus.de



**** Ressourcen schonen, weniger drucken - Think before you print! ****

---------------------------------------------------------------------
Diese E-Mail sowie eventuelle Anh?nge enthalten vertrauliche und / oder
rechtlich gesch?tzte Informationen. Wenn Sie nicht der richtige Adressat
sind oder diese E-Mail irrt?mlich erhalten haben, informieren Sie bitte
sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren
oder Speichern sowie die unbefugte Weitergabe dieser E-Mail sind nicht
gestattet.

This e-mail and any attachments may contain confidential and / or
privileged information. If you are not the intended recipient or have
received this e-mail in error, please notify the sender immediately and
destroy this e-mail . Any unauthorized copying, storing, disclosure or
distribution of the contents of this e-mail is strictly forbidden.

---------------------------------------------------------------------
HSBC Trinkaus & Burkhardt AG
Sitz: D?sseldorf, K?nigsallee 21/23, 40212 D?sseldorf, Handelsregister:
Amtsgericht D?sseldorf HRB 54447
Mitglieder des Vorstands: Andreas Schmitz (Sprecher), Paul Hagen, Dr. Olaf
Huth, Carola Gr?fin v. Schmettow
Vorsitzender des Aufsichtsrats: Dr. Sieghardt Rometsch


                                                                           
             Moshe Olshansky                                               
             <m_olshansky at yaho                                             
             o.com>                                                     An 
             Gesendet von:               r-sig-finance at stat.math.ethz.ch,  
             r-sig-finance-bou           Matthias.Koberstein at hsbctrinkaus. 
             nces at stat.math.et           de                                
             hz.ch                                                   Kopie 
                                                                           
             Fax-Deckblatt:                                          Thema 
             HSBCTuB                     Re: [R-SIG-Finance] Optimize      
             17.07.2008 01:35            question                          
                                                                           
              Bitte antworten                                              
                    an                                                     
             m_olshansky at yahoo                                             
                   .com                                                    
                                                                           
                                                                           




This may depend on your problem: how many dimensions do you have, is it a
constrained or unconstrained optimization, does your function have smooth
first (and second) derivatives, can you compute them analytically, etc.?


--- On Thu, 17/7/08, Matthias.Koberstein at hsbctrinkaus.de

From: Matthias.Koberstein at hsbctrinkaus.de

<Matthias.Koberstein at hsbctrinkaus.de>

Subject: [R-SIG-Finance] Optimize question
To: r-sig-finance at stat.math.ethz.ch
Received: Thursday, 17 July, 2008, 12:48 AM
Hi,

I use the command optim and optimize in a function.
Unfortunatley the standard method needs a lot of time and
in accordance to
the manual is the slowest (Nelder-Maed).
A more "dirty" optimization qould be sufficient
for my purposes as long as
it is faster. The function provides 4 other methods
("BFGS", "CG",
"L-BFGS-B", "SANN")
but which one is the fastest? Does anyone have eperience
with that?

Thank you very much in advance

Matthias

**** Ressourcen schonen, weniger drucken - Think before you
print! ****

---------------------------------------------------------------------
Diese E-Mail sowie eventuelle Anh?nge enthalten
vertrauliche und / oder
rechtlich gesch?tzte Informationen. Wenn Sie nicht der
richtige Adressat
sind oder diese E-Mail irrt?mlich erhalten haben,
informieren Sie bitte
sofort den Absender und vernichten Sie diese Mail. Das
unerlaubte Kopieren
oder Speichern sowie die unbefugte Weitergabe dieser E-Mail
sind nicht
gestattet.

This e-mail and any attachments may contain
confidential...{{dropped:18}}

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

_______________________________________________
R-SIG-Finance at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.