Symmetric Matrix classes

Scanning for 'Matrix' in old R-help e-mail, I found

"GA" == Gad Abraham <gabraham at csse.unimelb.edu.au>
? ? on Fri, 27 Nov 2009 13:45:00 +1100 writes:

? ?GA> Hi,
? ?GA> I'd like to store large covariance matrices using Matrix classes.

? ?GA> dsyMatrix seems like the right one, but I want to specify just the
? ?GA> upper/lower triangle and diagonal and not have to instantiate a huge
? ?GA> n^2 vector just for the sake of having half of it ignored:

? ?GA> Dumb example:
? ?GA> M <- new("dsyMatrix", uplo="U", x=rnorm(1e4), Dim=as.integer(c(100, 100)))
? ?GA> diag(M) <- 1

? ?GA> This doesn't work:
? ?GA> M <- new("dsyMatrix", uplo="U", x=0, Dim=as.integer(c(100, 100)))
? ?GA> Error in validObject(.Object) :
? ?GA> invalid class "dsyMatrix" object: length of x slot != prod(Dim)

? ?GA> Is there an easier way of doing this?

I think you want a ?"dspMatrix" ?("sp" == "symmetric packed")
instead.

Before going into details: Is this topic still interesting to
those involved almost two months ago?

Regards,
Martin

8 * (20000 * 20000)/2^30  # size in gigabytes for full array

4 * (20000 * 20001)/2^30  # size in gigabytes for sp array

Scanning for 'Matrix' in old R-help e-mail, I found

"GA" == Gad Abraham <gabraham at csse.unimelb.edu.au>
? ? on Fri, 27 Nov 2009 13:45:00 +1100 writes:

? ?GA> Hi,
? ?GA> I'd like to store large covariance matrices using Matrix classes.

? ?GA> dsyMatrix seems like the right one, but I want to specify just the
? ?GA> upper/lower triangle and diagonal and not have to instantiate a huge
? ?GA> n^2 vector just for the sake of having half of it ignored:

? ?GA> Dumb example:
? ?GA> M <- new("dsyMatrix", uplo="U", x=rnorm(1e4), Dim=as.integer(c(100, 100)))
? ?GA> diag(M) <- 1

? ?GA> This doesn't work:
? ?GA> M <- new("dsyMatrix", uplo="U", x=0, Dim=as.integer(c(100, 100)))
? ?GA> Error in validObject(.Object) :
? ?GA> invalid class "dsyMatrix" object: length of x slot != prod(Dim)

? ?GA> Is there an easier way of doing this?

I think you want a ?"dspMatrix" ?("sp" == "symmetric packed")
instead.

Before going into details: Is this topic still interesting to
those involved almost two months ago?

On Tue, Jan 19, 2010 at 9:26 AM, Martin Maechler
<maechler at stat.math.ethz.ch> wrote:

Scanning for 'Matrix' in old R-help e-mail, I found

"GA" == Gad Abraham <gabraham at csse.unimelb.edu.au>
? ? on Fri, 27 Nov 2009 13:45:00 +1100 writes:

? ?GA> Hi,
? ?GA> I'd like to store large covariance matrices using Matrix classes.

? ?GA> dsyMatrix seems like the right one, but I want to specify just the
? ?GA> upper/lower triangle and diagonal and not have to instantiate a huge
? ?GA> n^2 vector just for the sake of having half of it ignored:

? ?GA> Dumb example:
? ?GA> M <- new("dsyMatrix", uplo="U", x=rnorm(1e4), Dim=as.integer(c(100, 100)))
? ?GA> diag(M) <- 1

? ?GA> This doesn't work:
? ?GA> M <- new("dsyMatrix", uplo="U", x=0, Dim=as.integer(c(100, 100)))
? ?GA> Error in validObject(.Object) :
? ?GA> invalid class "dsyMatrix" object: length of x slot != prod(Dim)

? ?GA> Is there an easier way of doing this?

I think you want a ?"dspMatrix" ?("sp" == "symmetric packed")
instead.

Before going into details: Is this topic still interesting to
those involved almost two months ago?

Regards,
Martin

Also, I fail to understand the advantage of allocating storage for
roughly half the elements in the matrix instead of the whole matrix
when the matrix is so large. ?If dense storage of a symmetric matrix
of size 20000 takes up too much memory (approx 3 GB for each copy) it
is unlikely that the packed symmetric storage form, using about 1.5 GB
per copy, is going to save the day.

8 * (20000 * 20000)/2^30 ?# size in gigabytes for full array

[1] 2.980232

4 * (20000 * 20001)/2^30 ?# size in gigabytes for sp array

[1] 1.490191

If the matrix is sparse, the dsCMatrix class may help.

And there is also the issue of what exactly do you want to do with the
matrix once you have stored it?