Dear all, Firstly, I do apologize if my question is simple and posted in the wrong place but I had no reply from the R-help mailing list (maybe it is too simple!). I was wondering why parscale is set to 20 in the "wild" function example used in ?optim. This function has only one parameter and if we set parscale equal to 1 then the solution near the global minimum is not found. I would use parscale only in cases the object function has more than one parameter to be optimised, shouldn't I? Many thanks in advance to all of you and kind regards, Tom __________________________________________________
"wild" function example in optim
4 messages · Werner Bier, Thomas Lumley, Brian Ripley +1 more
On Tue, 26 Apr 2005, Werner Bier wrote:
Dear all, Firstly, I do apologize if my question is simple and posted in the wrong place but I had no reply from the R-help mailing list (maybe it is too simple!). I was wondering why parscale is set to 20 in the "wild" function example used in ?optim. This function has only one parameter and if we set parscale equal to 1 then the solution near the global minimum is not found. I would use parscale only in cases the object function has more than one parameter to be optimised, shouldn't I?
parscale is more important in cases with more than one parameter (and with one parameter you could set fnscale instead of parscale to get the same effect) However, a sufficiently badly scaled one-d problem can still benefit from fnscale or parscale.
f
function(x) 1e-10*x^2
g
function(x) 2e-10*x
optim(7,f,g,method="CG")$par
[1] 7
optim(7,f,g,method="CG",control=list(parscale=1e5))$par
[1] 1.209735e-14
optim(7,f,g,method="CG",control=list(fnscale=1e-10))$par
[1] 1.673141e-15 -thomas -thomas
On Tue, 26 Apr 2005, Thomas Lumley wrote:
On Tue, 26 Apr 2005, Werner Bier wrote:
Dear all, Firstly, I do apologize if my question is simple and posted in the wrong place but I had no reply from the R-help mailing list (maybe it is too simple!). I was wondering why parscale is set to 20 in the "wild" function example used in ?optim. This function has only one parameter and if we set parscale equal to 1 then the solution near the global minimum is not found.
Note that there the method is "SANN". That makes assumptions about step sizes, in fact using a spherical Gaussian distribution of fixed size. So parscale=20 is telling it to make initial steps large enough to explore the `blobs'. In particular, parscale is not set for the BFGS call in that example.
I would use parscale only in cases the object function has more than one parameter to be optimised, shouldn't I?
parscale is more important in cases with more than one parameter (and with one parameter you could set fnscale instead of parscale to get the same effect)
Not necessarily. The finite-differencing is done in units rescaled by parscale. So a unit change in a single parameter needs to be a reasonably-sized step. One can always set fnscale and neps, but it is easier to set parscale.
However, a sufficiently badly scaled one-d problem can still benefit from fnscale or parscale.
f
function(x) 1e-10*x^2
g
function(x) 2e-10*x
optim(7,f,g,method="CG")$par
[1] 7
optim(7,f,g,method="CG",control=list(parscale=1e5))$par
[1] 1.209735e-14
optim(7,f,g,method="CG",control=list(fnscale=1e-10))$par
[1] 1.673141e-15
but without g
optim(7,f,method="CG",control=list(parscale=1e5))$par
[1] 1.209735e-14
optim(7,f,method="CG",control=list(fnscale=1e-10))$par
[1] 1.997947e-11
Brian D. Ripley, ripley@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
Thomas Lumley <tlumley@u.washington.edu> writes:
On Tue, 26 Apr 2005, Werner Bier wrote:
Dear all, Firstly, I do apologize if my question is simple and posted in the wrong place but I had no reply from the R-help mailing list (maybe it is too simple!). I was wondering why parscale is set to 20 in the "wild" function example used in ?optim. This function has only one parameter and if we set parscale equal to 1 then the solution near the global minimum is not found. I would use parscale only in cases the object function has more than one parameter to be optimised, shouldn't I?
parscale is more important in cases with more than one parameter (and with one parameter you could set fnscale instead of parscale to get the same effect) However, a sufficiently badly scaled one-d problem can still benefit from fnscale or parscale.
f
function(x) 1e-10*x^2
g
function(x) 2e-10*x
optim(7,f,g,method="CG")$par
[1] 7
optim(7,f,g,method="CG",control=list(parscale=1e5))$par
[1] 1.209735e-14
optim(7,f,g,method="CG",control=list(fnscale=1e-10))$par
[1] 1.673141e-15
It also depends on the optimizer. The SANN optimizer basically jumps haphazardly (well, a bit more intelligently than that) back and forth along the x axis and then "cools down" in order to settle in the "best" local minimum. The parscale plays a role in setting the scale of those jumps and if it is too low it might not wander far enough to get near the true minimum. For further information, you really need to do your own reading. References are given on the help page.
O__ ---- Peter Dalgaard Blegdamsvej 3 c/ /'_ --- Dept. of Biostatistics 2200 Cph. N (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~~~~~~~~~ - (p.dalgaard@biostat.ku.dk) FAX: (+45) 35327907