Construction of a large sparse matrix
The dense blocks are too big as Reid has already written -- for smaller instances of this sort of thing I would suggest that the the kronecker product %x% operator in SparseM, would be more convenient. url: www.econ.uiuc.edu/~roger Roger Koenker email rkoenker at uiuc.edu Department of Economics vox: 217-333-4558 University of Illinois fax: 217-244-6678 Champaign, IL 61820
On Apr 18, 2005, at 3:54 PM, Doran, Harold wrote:
Dear List: I'm working to construct a very large sparse matrix and have found relief using the SparseM package. I have encountered an issue that is confusing to me and wonder if anyone may be able to suggest a smarter solution. The matrix I'm creating is a covariance matrix for a larger research problem that is subsequently used in a simulation. Below is the latex form of the matrix if anyone wants to see the pattern I am trying to create. The core of my problem seems to localize to the last line of the following portion of code. n<-sample.size*4 k<-n/4 vl.mat <- as.matrix.csr(0, n, n) block <- 1:k #each submatrix size for(i in 1:3) vl.mat[i *k + block, i*k + block] <- LE When the variable LE is 0, the matrix is easily created. For example, when sample.size = 10,000 this matrix was created on my machine in about 1 second. Here is the object size.
object.size(vl.mat)
[1] 160692
However, when LE is any number other than 0, the code generates an
error. For example, when I try LE <- 2 I get
Error: cannot allocate vector of size 781250 Kb
In addition: Warning message:
Reached total allocation of 1024Mb: see help(memory.size)
Error in as.matrix.coo(as.matrix.csr(value, nrow = length(rw), ncol =
length(cl))) :
Unable to find the argument "x" in selecting a method for
function "as.matrix.coo"
I'm guessing that single digit integers should occupy the same amount
of
memory. So, I'm thinking that the matrix is "less sparse" and the
problem is related to the introduction of a non-zero element (seems
obvious). However, the matrix still retains a very large proportion of
zeros. In fact, there are still more zeros than non-zero elements.
Can anyone suggest a reason why I am not able to create this matrix?
I'm
at the limit of my experience and could use a pointer if anyone is able
to provide one.
Many thanks,
Harold
P.S. The matrix above is added to another matrix to create the
covariance matrix below. The code above is designed to create the
portion of the matrix \sigma^2_{vle}\bm{J} .
\begin{equation}
\label{vert:cov}
\bm{\Phi} = var
\left [
\begin{array}{c}
Y^*_{1}\\
Y^*_{2}\\
Y^*_{3}\\
Y^*_{4}\\
\end{array}
\right ]
=
\left [
\begin{array}{cccc}
\sigma^2_{\epsilon}\bm{I}& \sigma^2_{\epsilon}\rho\bm{I} & \bm{0} &
\bm{0}\\
\sigma^2_{\epsilon}\rho\bm{I} &
\sigma^2_{\epsilon}\bm{I}+\sigma^2_{vle}\bm{J} &
\sigma^2_{\epsilon}\rho^2\bm{I} & \bm{0}\\
\bm{0} & \sigma^2_{\epsilon}\rho^2\bm{I} &
\sigma^2_{\epsilon}\bm{I}+\sigma^2_{vle}\bm{J}&
\sigma^2_{\epsilon}\rho^3\bm{I}\\
\bm{0} & \bm{0} & \sigma^2_{\epsilon}\rho^3\bm{I}&
\sigma^2_{\epsilon}\bm{I}+\sigma^2_{vle}\bm{J} \\
\end{array}
\right]
\end{equation}
where $\bm{I}$ is the identity matrix, $\bm{J}$ is the unity matrix,
and
$\rho$ is the autocorrelation.
[[alternative HTML version deleted]]
______________________________________________ R-help at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html