optimization with Sparse matrices

Dube, Jean-Pierre <Jean-Pierre.Dube <at> chicagobooth.edu> writes:

To whom it may concern,

I am trying to maximize a log-likelihood function using optim. 
This is a simple problem with only 18
parameters.  To conserve memory, I am using sparse matrices 
(SLAM) for some of the data matrices used in the
computation of the likelihood.  However, optim appears to
convert the sparse matrix back to regular data
format.  This causes me to run out of memory as R tries 
to create an 8GB matrix.  In short, it does not look as
though "optim" is compatible with sparse matrices. 
Does anyone have a suggestion for how I can maximize a
function in R using sparse matrices for some of the data inputs?

I wouldn't guess that optim() itself would do any conversion
of the data values (are these passed as additional parameters
or used as global variables?)  The only thing optim() does with
parameters in '...' is pass them along to the objective
function as follows:

    fn1 <- function(par) fn(par, ...)

and I can't see a reason that would convert them in any way.
It seems more likely to me (?) that there is some step in 
your objective function where you do an operation that accidentally
converts the sparse back to non-sparse matrices ...

  Can you construct a minimal example that shows this happening?