Alternative to zonal for large images
Oscar, Thanks for illustrating the large speed gain of using data.table for aggregating (instead of tapply / aggregate). I am going to use this in the raster package for the zonal function and a few others. Robert On Wed, Feb 13, 2013 at 11:00 PM, Oscar Perpi??n Lamigueiro
<oscar.perpinan at gmail.com> wrote:
Oscar, I find your idea ingenious, however I don't see how this method could still work with out of memory raster (since you load the whole raster using getValues()). data.table storage is more efficient, so maybe it's the fact that you can shrink the space taken by the two layers that allow to process rasters in memory ? Or is canProcessInMemory() too conservative ? I think it's using a 10% buffer.
Hello, My last code focused only on speed improvement. We should add several lines of code to check memory usage. On the other hand, I am still learning to use the amazing data.table package, so I cannot give a solution for out of memory problems. However, it seems that data.table copes with them quite well: http://stackoverflow.com/questions/11564775/exceeding-memory-limit-in-r-even-with-24gb-ram Besides, I have just discovered the ":=" operator which could be useful for working with large datasets. http://stackoverflow.com/questions/9508118/out-of-memory-when-modifying-a-big-r-dataframe http://stackoverflow.com/questions/7029944/when-should-i-use-the-operator-in-data-table Best, Oscar.
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo