optim-Bug (PR#6720)

kestler@neuro.informatik.uni-ulm.de writes:

Full_Name: Dr. Hans A. Kestler
Version: 1.8.1.
OS: Linux, Win, Mac OSX
Submission from: (NULL) (134.60.73.116)


The code below produces after a different number of iterations i the following
error:

Error in optim(par = rep(0.5, length(edges)), loglik, method = "L-BFGS-B",  :
        non-finite value supplied by optim

This was reproducible on different machines (Mac G4 OSX, AMD Opteron Linux SUSE
9.0, Intel P4 Suse 9.0, P4 Windows XP Prof), only with different i.

The non-deterministic behaviour made us recompile R with debug options and use
valgrind for memory-leak checking. The result was horrible. We are now in the
process of going through the f2c code of lbfgsb.c. The problem seems to be a
access violation of wn1 at run time. 

I have this down to the use of R_alloc in the vect() function (line
40, optim.c). Replacing with S_alloc (which zeros memory) removes the
issue for me on this machine and on the Opteron. Could you please
verify on other platforms? And if anyone actually understands the code
could you verify that it makes sense to require the workspace to be
initialized?

On 21 Apr 2004, Peter Dalgaard wrote:

I have this down to the use of R_alloc in the vect() function (line
40, optim.c). Replacing with S_alloc (which zeros memory) removes the
issue for me on this machine and on the Opteron. Could you please
verify on other platforms? And if anyone actually understands the code
could you verify that it makes sense to require the workspace to be
initialized?

Zeroing the workspace is not a requirement of the original L-BFGS-B code 
that I can see.  Given that it was originally in Fortran, and Fortran 
often does zero it seems a likely symptom, but it does mean that a 
variable is being used uninitialized somewhere in the code (converted to 
C).  It would be better to leave vect alone and to zero the workspace with 
a memset call in lbfgsb.  (Incidentally, I don't know why S_alloc does not 
use memset -- we do require standard C and use in seeral other places.)

Zeroing the workspace is not a requirement of the original L-BFGS-B code 
that I can see.  Given that it was originally in Fortran, and Fortran 
often does zero it seems a likely symptom, but it does mean that a 
variable is being used uninitialized somewhere in the code (converted to 
C).  It would be better to leave vect alone and to zero the workspace with 
a memset call in lbfgsb.  (Incidentally, I don't know why S_alloc does not 
use memset -- we do require standard C and use in seeral other places.)

Prof Brian Ripley <ripley@stats.ox.ac.uk> writes:
...

Zeroing the workspace is not a requirement of the original L-BFGS-B code 
that I can see.  Given that it was originally in Fortran, and Fortran 
often does zero it seems a likely symptom, but it does mean that a 
variable is being used uninitialized somewhere in the code (converted to 
C).  It would be better to leave vect alone and to zero the workspace with 
a memset call in lbfgsb.  (Incidentally, I don't know why S_alloc does not 
use memset -- we do require standard C and use in several other places.)

I had planned to suggest that we use memset more widely and perhaps
add a macro Memset, defined like the current Memcpy, to R_ext/RS.h.  I
felt that memset was likely to be more efficient for zeroing large
areas of memory than running a loop would be.  Recently Peter Dalgaard
and Andy Liaw and I discussed this with regard to a test to see if
vectors using more than 4 GB of memory could be allocated on an
Opteron.  Andy installed my suggested modification to do_makevector to
use memset for zeroing freshly allocated vectors and found that the
running time for a test on an Opteron/Linux system actually increased.
It appears that, at least on that system, memset runs a loop at the
level of characters.

I still think it would be a good idea to use memset but we would want
to do some timing comparisons to make sure we don't slow things down.