memory issues w/ raster processing
Marcia,
If the problem persists you can try to force it to write to disk if it does not.
setOptions(todisk=TRUE)
If it still fails:
setOptions(chunksize=100000)
I would normally not recommend to use these options.
The below worked for me (also with your number of rows and columns,
but then it takes a while!)
library(raster)
s <- list()
r <- raster(ncol=26, nrow=29)
for (i in 1:10) {
s[i] <- setValues(r, round(runif(ncell(r)+0.5) * 3))
}
s <- stack(s)
f.232 = function(x){
# adapted from code by Robert Hijmans
# changes 232 to 222, but does not touch 233 or 223
a <- which(x == 3)
a <- subset(a, a > 1 & a < length(x))
b <- which(x[a-1] == 2 & x[a+1] == 2)
x[a[b]] <- 2
return(x)
}
x <- calc(s, f.232)
On Wed, Dec 29, 2010 at 10:00 AM, Marcia Macedo
<marcia.n.macedo at gmail.com> wrote:
Dear List,
I have been trying to run a simple function on a raster stack (dimensions
are roughly 75 MB with 2600 samples x 2900 lines x 10 layers). The function
(adapted from code that Robert posted last week) is as follows:
f.232 = function(x){
? ?# adapted from code by Robert Hijmans
? ?# changes 232 ?to 222, but does not touch 233 or 223
? ?a <- which(x == 3)
? ?a <- subset(a, a > 1 & a < length(x))
? ?b <- which(x[a-1] == 2 & x[a+1] == 2)
? ?x[a[b]] <- 2
? ?return(x)
}
The problem is that the function takes a very long time to run and often
crashes or gives me memory warnings like this one:
Warning messages:
1: In dimnames(data) <- dimnames :
?Reached total allocation of 7854Mb: see help(memory.size)
2: In dimnames(data) <- dimnames :
?Reached total allocation of 7854Mb: see help(memory.size)
I am confused because this data cube is not that large and I think my system
should be able to handle it (see specs. below).
*System specs:*
R version 2.12.1 (2010-12-16)
Platform: x86_64-pc-mingw32/x64 (64-bit)
Installed memory (RAM): 8 GB
Processor: Intel Core i3, 2.13 GHz
I must admit that I don't really understand how the raster package manages
memory, but I was imagining that it would be more efficient to write the
results to file iteratively (in blocks) while running the calculations
(rather than writing all results to memory and then writing to
file)...However, my attempts to adapt the writeValues code in the package
documentation have not worked (i.e. the output file gets created but never
written to and the process crashes as described above).
Any thoughts about what might be going on with the memory? I am very new to
raster processing in R (was previously working with ENVI+IDL) so forgive me
if this is a naive question!
Thanks,
Marcia
? ? ? ?[[alternative HTML version deleted]]
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo