Mathematica may well have good optimization routines; I know MATLAB does (e.g. the optimization toolbox, http://www.mathworks.com/products/optimization/?BB=1 , has a general-constraints nonlinear optimizer) -- I also think more people develop optimization code in MATLAB because of its use in engineering circles. (The DIRECT algorithm is implemented in TOMLAB, a MATLAB add-on package from a third party.) (S+NuOPT is also available from Insightful, although it doesn't look like it does global stochastic stuff; our friends at Otter Research have ADMB, although I don't know how well it handles noisy objective functions.) Drawbacks: (1) switching platforms is a pain, (2) most of these alternatives are expensive [although if you have the money it may well be worth it], (3) Mathematica is notoriously bad about documenting its methods and giving references to the peer-reviewed literature. If I were regularly doing really difficult optimization problems I might switch to MATLAB+add-ons. I would like to see more optimization choices implemented in R, but haven't gotten around to doing so myself yet. For hard optimization problems, however, it is nearly always true that you have to know something about your problem and tune methods accordingly (see Spencer Graves' message in this thread). Ben -------- Original Message -------- Subject: Re: [R] any more direct-search optimization method in R Date: Tue, 28 Feb 2006 09:55:15 -0800 (PST) From: Greg Tarpinian <sasprog474474 at yahoo.com> To: Ben Bolker <bolker at ufl.edu> This may not be the most helpful advice, but Mathematica is a wonderful platform having many more built-in optimization routines than R. I have found the simulated annealing facility to be robust and much easier to use than R's own facility. This may be a case where switching to another platform would be the easiest solution. Kind regards, Greg
--- Ben Bolker <bolker at ufl.edu> wrote:
Ingmar Visser <I.Visser <at> uva.nl> writes:
If you have only boundary constraints on parameters you can use method L-BFGS in optim. Hth, ingmar
From: Weijie Cai <wcai11 <at> hotmail.com>
I am dealing with a noisy function (gradient,hessian not available) with simple boundary constraints (x_i>0). I've tried constrOptim() using
nelder
mead to minimize it but it is way too slow and the returned results are
not
satisfying. simulated annealing is so hard to tune and it always crashes
R
program in my case. I wonder if there are any packages or functions can
do
direct search optimization?
Noisy functions are really challenging to optimize; (1) there is no "best" method (despite all the papers doing comparisons of stochastic global optimizers on various sets of test functions); (2) the fancier methods are hard to program [and existing implementations tend have more restricted licenses]; (3) they tend to be slow (thousands of function evaluations). Packages on CRAN that *might* be helpful are genalg, DEoptim. A typical "poor man's" approach to boundary constraints is to add a quadratic penalty (perhaps not even trying to evaluate the objective function -- e.g. substituting the value at the closest boundary point) for parameters outside the constraints into the objective function. With more information (number of parameters, time to compute a single function evaluation, kind of noise) we might be able to help more. Ben Bolker
______________________________________________ R-help at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
__________________________________________________
620B Bartram Hall bolker at zoo.ufl.edu Zoology Department, University of Florida http://www.zoo.ufl.edu/bolker Box 118525 (ph) 352-392-5697 Gainesville, FL 32611-8525 (fax) 352-392-3704