Hi Lo?c,
thank you for your reply and the suggestion. Unfortunately, using the
cluster functions in raster is not an option, because I am developing a
package which may run with raster data but not necessarily. Therefore, I
wanted to parallelize the stuff in the function of the package and not the
loop over the raster image.
Cheers
Lukas
Am Freitag, 26. August 2016, 13:02:33 schrieb Lo?c Dutrieux:
Hi Lukas,
What does your function do? Can you use the built-in parallel
functionalities of the raster package instead? See ?beginCluster
Cheers,
Lo?c
On 26/08/2016 12:23, Lukas Lehnert via R-sig-Geo wrote:
Dear list members,
I tried to process a large GeoTIFF file in a blockwise manner (following
the vignette "Writing functions for large raster files").
I then combined the approach with a "foreeach"-loop to parallelize the
calculation on the raster data:
library(raster)
library(doMC)
library(foreach)
registerDoMC(3)
ra <- raster("dem.tif")
tr <- blockSize(ra)
outfile <- "test_2.tif"
res <- writeStart(raster(ra), outfile, overwrite = TRUE)
for (i in 1:tr$n)
{
v <- getValuesBlock(ra, row=tr$row[i], nrows=tr$nrows[i])
v2 <- foreach(i=1:length(v), .combine = 'c') %dopar% {
i*(-1)
}
res <- writeValues(res, v2, tr$row[i])
cat(round(i/tr$n*100, 2), "%\n")
}
res <- writeStop(res)
However, I get errors and warnings like the following ones:
ERROR 1: /tmp/Rtmpz6qEOE/xortest_2.tif:Failed to allocate memory for to
read TIFF directory (0 elements of 12 bytes each)
ERROR 1: TIFFReadDirectory:Failed to read directory at offset 4639232
Warning 1: TIFFFetchNormalTag:ASCII value for tag "GeoASCIIParams"
contains
null byte in value; value incorrectly truncated during reading due to
implementation limitations
The errors are different each time I restart the process. If I do not
use
foreach but a common for-loop, the errors do not occur. I tried several
raster files (large ones and small ones) and two different computers,
but
the problem remains.
Does somebody has any idea, how to solve this problem?
Thank you in advance
Lukas