Fast way to populate a sparse matrix

Convert your 'targets' matrix into a 2 column matrix with the 1st
column representing the row and the 2nd the column where you want your
values, then change the values to a single vector and you can just use
the targets matrix as the subsetting in 1 step without (explicit)
looping, for example:

library(Matrix)

adjM <- Matrix(0,nrow=10,ncol=10)

locs <- cbind( sample(1:10), sample(1:10) )
vals <- rnorm(10)

adjM[ locs ] <- vals

I would expect this to be faster than looping (but have not tested).

I need to generate a sparse matrix. Currently I have the data held in two
regular matrices. One 'targets' holds the column subscripts while the other
'scores' holds the values. I have written a 'toy' sample below. Using this
approach takes about 90 seconds to populate a 30000 x 30000 element matrix.
I'm going to need to scale this up by a factor of about 1000 so I really
need a faster way of populating the sparse matrix.
Any advice received gratefully.

# toy code starts here

require('Matrix')
set.seed(0)

adjM<-Matrix(0,nrow=10,ncol=10)

#generate the scores for the sparse matrix, with the target locations
targets<-matrix(nrow=10,ncol=5)
scores<-matrix(nrow=10,ncol=5)
for(iloc in 1:10)
  {
  targets[iloc,]<-sample(1:10,5,replace=FALSE)
  scores[iloc,]<-rnorm(5)
  }

#populate the sparse matrix
for(iloc in 1:10)
  {
  adjM[iloc,targets[iloc,!is.na(targets[iloc,])]]<-scores[iloc,!is.na
(targets[iloc,])]
  }

        [[alternative HTML version deleted]]

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Gregory (Greg) L. Snow Ph.D.
538280 at gmail.com

library(Matrix)

adjM <- Matrix(0,nrow=10,ncol=10)

locs <- cbind( sample(1:10), sample(1:10) )
vals <- rnorm(10)

adjM[ locs ] <- vals

... and once you've got your data in this format, why not construct the sparse matrix directly?

	adjM <- sparseMatrix(i = locs[,1], j = locs[,2], x = vals)

I've found this to be very efficient and have used it with sparse matrices containing up to around 100 million nonzero entries.

Hope this helps,
Stefan

Stefan Evert <stefanML at collocations.de>
    on Fri, 25 Apr 2014 09:09:31 +0200 writes:

>> library(Matrix)
    >> 
    >> adjM <- Matrix(0,nrow=10,ncol=10)
    >> 
    >> locs <- cbind( sample(1:10), sample(1:10) )
    >> vals <- rnorm(10)
    >> 
    >> adjM[ locs ] <- vals

    > ... and once you've got your data in this format, why not construct the sparse matrix directly?

    > adjM <- sparseMatrix(i = locs[,1], j = locs[,2], x = vals)

    > I've found this to be very efficient and have used it with sparse matrices containing up to around 100 million nonzero entries.

Yes, indeed,  thank you Stefan!

Let me reiterate :

__  Unless you can use special constructors such as  
__
__   Diagonal()      # diagonal matrices
__   bdiag()         # block diagonal matrices
__   bandSparse()    # "banded diagonal" matrices
__   kronecker(a, b) # where a or b are sparse
__
__   (and maybe another one I forgot)
__
__  *the* way to efficiently construct large sparse matrices is
__    sparseMatrix()
__    or sometimes its alternative / precursor  spMatrix().
__
__    Matrix(d, ...., sparse=TRUE)
__
__  is nice and fine only for relatively *small* matrices,
__  as it really works from a dense original 'd' (directly or via replication)


    > Hope this helps,

I do hope, too.
I'm very happy for suggestions on  how  we as Matrix authors
could make this better known.

Recently, someone proposed to make the 'rsparseMatrix()'
utility function from  help(sparseMatrix)
into an "official" Matrix package function.  If I did that,
I could start using  rsparseMatrix() in typical examples rather
than the current often use of  Matrix()  or
as(<traditional matrix>, "sparseMatrix")
both of which are perfect for the small examples that are
typical for help files.

Martin Maechler,
ETH Zurich

An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20140429/be595682/attachment.pl>