any more direct-search optimization method in R

Mathematica may well have good optimization routines; I know MATLAB
does (e.g. the optimization toolbox, 
http://www.mathworks.com/products/optimization/?BB=1 , has a 
general-constraints nonlinear optimizer) -- I also think
more people develop optimization code in MATLAB because of its use in
engineering circles.  (The DIRECT algorithm is implemented in
TOMLAB, a MATLAB add-on package from a third party.)
(S+NuOPT is also available from Insightful,
although it doesn't look like it does global stochastic stuff;
our friends at Otter Research have ADMB, although I don't
know how well it handles noisy objective functions.)

    Drawbacks: (1) switching platforms is a pain, (2) most of
these alternatives are
expensive [although if you have the money it may well be
worth it], (3) Mathematica is notoriously bad about documenting
its methods and giving references to the peer-reviewed literature.
If I were regularly doing really difficult optimization
problems I might switch to MATLAB+add-ons.

   I would like to see more optimization choices implemented in
R, but haven't gotten around to doing so myself yet.  For hard
optimization problems, however, it is nearly always true that
you have to know something about your problem and tune methods
accordingly (see Spencer Graves' message in this thread).

   Ben

-------- Original Message --------
Subject: Re: [R] any more direct-search optimization method in R
Date: Tue, 28 Feb 2006 09:55:15 -0800 (PST)
From: Greg Tarpinian <sasprog474474 at yahoo.com>
To: Ben Bolker <bolker at ufl.edu>

This may not be the most helpful advice, but Mathematica is a wonderful
platform having many more built-in optimization routines than R.  I have
found the simulated annealing facility to be robust and much easier to
use than R's own facility.  This may be a case where switching to another
platform would be the easiest solution.

Kind regards,

     Greg

Ingmar Visser <I.Visser <at> uva.nl> writes:

If you have only boundary constraints on parameters you can use method
L-BFGS in optim.
Hth, ingmar

From: Weijie Cai <wcai11 <at> hotmail.com>

I am dealing with a noisy function (gradient,hessian not available) with
simple boundary constraints (x_i>0). I've tried constrOptim() using

nelder

mead to minimize it but it is way too slow and the returned results are

not

satisfying. simulated annealing is so hard to tune and it always crashes

R

program in my case. I wonder if there are any packages or functions can

do

direct search optimization?

   Noisy functions are really challenging to optimize; (1) there is no
"best" method (despite all the papers doing comparisons of stochastic
global optimizers on various sets of test functions); (2) the fancier
methods are hard to program [and existing implementations tend have more
restricted licenses]; (3) they tend to be slow (thousands of function
evaluations).   Packages on CRAN that *might* be helpful are
genalg, DEoptim.
   A typical "poor man's" approach to boundary constraints is to
add a quadratic penalty (perhaps not even trying to evaluate the
objective function -- e.g. substituting the value at the closest
boundary point) for parameters outside the constraints into
the objective function.
   With more information (number of parameters, time to compute a
single function evaluation, kind of noise) we might be able to help
more.
  
  Ben Bolker

______________________________________________
R-help at stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html

__________________________________________________

620B Bartram Hall                            bolker at zoo.ufl.edu
Zoology Department, University of Florida    http://www.zoo.ufl.edu/bolker
Box 118525                                   (ph)  352-392-5697
Gainesville, FL 32611-8525                   (fax) 352-392-3704