stack many files without loading into memory
On Thu Feb 05 2015 at 7:41:33 AM Dominik Schneider <
Dominik.Schneider at colorado.edu> wrote:
I think you are correct. s=stack(fn,quick=T) writeRaster(s,'localpath/2012data')
Ugh, sorry yes that's me reading too fast. I should have suggested the next step to writeRaster, I'm not sure why you don't include the file extension here though? Why not writeRaster(s,'localpath/2012data.grd') would get the data local. I guess the trade off is that the file size is an
order of magnitude bigger than if I saved them in an .RData file but much quicker to access.
You might achieve similar compression if you choose GeoTIFF, with the right
options (and you need rgdal). Try a test with a single layer, e.g.
s=stack(fn,quick = TRUE)
require(rgdal)
writeRaster(s[[1]],'localpath/2012data_temp01.tif', options =
c("COMPRESS=LZW", "TILED=YES")
Does the file size of "2012data_temp01.tif" look promising?
The native "rasterfile" format does not support compression as far as I
know. Tiling may be of help or hindrance, depending on the dimensions and
the extra margin added by the tiles if they need to extend beyond the
margins - you can control tile size with BLOCKX/YSIZE if needed:
http://www.gdal.org/frmt_gtiff.html
(NetCDF4 - with ncdf4 package - can also compress and tile natively, but I
haven't tried that via raster myself).
Cheers, Mike.
ds On Wed, Feb 4, 2015 at 12:51 PM, Dominik Schneider < dominik.schneider at colorado.edu> wrote:
Wouldn't that keep the link to the server on which they are stored now?
On Wed, Feb 4, 2015 at 12:50 PM, Michael Sumner <mdsumner at gmail.com>
wrote:
Why not stack(fn) ? On Thu, 5 Feb 2015 06:41 Dominik Schneider < Dominik.Schneider at colorado.edu> wrote:
Hi -
I have some data on a server but would like to bring them local in a
somewhat compressed format that is still easy to access.
/Volumes/hD/2012 -> 100 geotiffs
~/project/data/ -> store those geotiffs here without needing server
access.
untested, I think I could do something like:
s=stack()
writeRaster(s,'2012stack')
fn=list.files('/Volumes/hD/2012',pattern='*.tif',full.names=T)
lapply(fn,function(f){
s=stack('2012stack')
r=raster(f)
names(r)=gsub(pattern='.tif',replacement='',basename(f))
s=addLayer(s,r)
writeRaster(s,'2012stack')
})
Or is it better to save to a .RData?
Is there a better way that doesn't require me to loop through each
geotiff
since I can't load it all into memory.
Thanks
Dominik
[[alternative HTML version deleted]]
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo