Hi Nevil (cc'ing r-sig-geo): How are they sparse, exactly? Is this something where you could be using a vector-like format instead (x,y,category time 1, category time 2), etc? It might be looking towards the spacetime data models for some inspiration, since they have a lot of different sparse and semi-sparse formats. I'm not quite following why a compressed format wouldn't work in terms of storage space -- raster, and the functions like getValueBlock (for instance) don't care what raster format the files are in -- in your example, those 160 layers could be MUCH smaller (on disk) if they are truly sparse (lots of 0s or similar values), and still provide the read/write capabilities just like an uncompressed TIFF would. --j
On Fri, Jul 11, 2014 at 3:37 AM, nevil amos <nevil.amos at gmail.com> wrote:
Thanks for the suggestion. I am compiling a raster stack with ~160 layers of 8000x5000 categorical rasters I then need to do an apply() to get unique value combinations for each corresponding cell in the rasters, so reading from compressed format ( as I might do if I were plotting them woes not work. On Fri, Jul 11, 2014 at 7:35 AM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
Nevil: Have you considered using an alternative backing format that has compression, like an LZW Tiff? Raster can wrap these with no problem but you benefit from a significantly smaller footprint. --j On Fri, Jul 4, 2014 at 11:00 AM, nevil amos <nevil.amos at gmail.com> wrote:
I note mention of RasterLayerSparse in the raster package documentation e.g. under setValues() Is it possible to setValues from a sparse matrix? can the values be stored in the raster object in sparse form to save memory? Trying to create "sparse raster" the sparse matrix is not accepted: library(raster) library(Matrix) m<-matrix(c(0,0,1),3,3) M<-Matrix(m)
M
3 x 3 sparse Matrix of class "dtCMatrix"
[1,] . . .
[2,] . . .
[3,] 1 1 1
ext<-extent(c(0,3,0,3))
R<-raster(ext,3,3)
setValues(R,M)
Error in setValues(R, M) : values must be a vector
[[alternative HTML version deleted]]
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
-- Jonathan A. Greenberg, PhD Assistant Professor Global Environmental Analysis and Remote Sensing (GEARS) Laboratory Department of Geography and Geographic Information Science University of Illinois at Urbana-Champaign 259 Computing Applications Building, MC-150 605 East Springfield Avenue Champaign, IL 61820-6371 Phone: 217-300-1924 http://www.geog.illinois.edu/~jgrn/ AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
Jonathan A. Greenberg, PhD Assistant Professor Global Environmental Analysis and Remote Sensing (GEARS) Laboratory Department of Geography and Geographic Information Science University of Illinois at Urbana-Champaign 259 Computing Applications Building, MC-150 605 East Springfield Avenue Champaign, IL 61820-6371 Phone: 217-300-1924 http://www.geog.illinois.edu/~jgrn/ AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007