gstat memory
Dear Murray and list, repairing the memory leaks turned out to be a more fun job than I thought it would be. Memory leakage mainly took place in the neighbourhood search algorithm (nsearch.c) which uses a quadtree for 2 and octtree (octree?) for 3-dimensional data. As you can guess, the code is full with recursion, and was contributed by Steve Joyce, someone much cleverder than I am, back in 1997, so that was a nice puzzle. But the leaks should be gone now. gstat_0.9-64 now propagates through CRAN. In my enthousiasm, I removed any other memory leak I could. This meant several code changes at the c level that looked as if back then I made them to resolve a bug -- of course we didn't have valgrind then. In other words, I might have introduced new problems. If you encounter anything unexpected, then please let me know. Whenever this stabilizes, I will start calling increase gstat's version number to 1.0_0 Best regards, -- Edzer
Murray Richardson wrote:
Hello R-SIG-GEO list,
I know this has come up before but I am having an ongoing memory
problem with the gstat package (gstat out of dynamic memory) that I
can't seem to solve.
I am using R to interpolate DEMs from LiDAR xyz point files and mosaic
them together via RSAGA. The script uses a loop to load in each xyz
point file, and interpolate over a regular lattice of points using idw
from gstat. Although large, the computational requirements within
each iteration of the loop should be well within my system's ability
so it seems like it is a cumulative effect (note I can restart the
process at the last loop that triggered the error and the iteration is
successful). I am removing temp objects and running gc() at the end
of every loop.
It proceeds normally for about 5-10 iterations and physical memory use
on my system (VISTA 64 bit, 12GB RAM) gradually increases over time
until I get the "gstat out of memory" error. Here is the relevant
portion of the loop:
...
path<-"E:/LidarData/1_Ground_First_Return/UTM17/"
filen<-paste(path, centre_tile$TILECODE, ".xyz", sep="")
xyzi<-read.table(filen, sep=",", header=F)
names(xyzi)<-c("x","y","z","i")
for(j in 1:length(tile_group_names)){
filen<-paste(path, tile_group_names[j], ".xyz", sep="")
tmp_xyzi<-read.table(filen, sep=",", header=F)
names(tmp_xyzi)<-c("x","y","z","i")
xyzi<-rbind(xyzi,tmp_xyzi)
}
coordinates(xyzi)=~x+y
# note grid_coords is just the regular lattice that is created from
the current tile coordinates
coordinates(grid_coords)=~x+y
# do the interpolation
interp<-idw(z~1, xyzi, grid_coords, nmax=4,maxdist=2, idp=1.0)
names(interp)<-c("z","var")
slot(interp,"data")<-data.frame(slot(interp,"data"))
finalSPntsDF<-SpatialPointsDataFrame(interp, data.frame(interp$z),
proj4string = CRS(as.character(NA)), match.ID = TRUE)
finalSPDF<-SpatialPixelsDataFrame(finalSPntsDF,
data.frame(interp$z), tolerance = sqrt(.Machine$double.eps),
proj4string = CRS(as.character(NA)))
... On a related note - I have tried using the 64 bit version of
Revolution but unfortunately the gstat package has not been ported.
Has anyone contemplated or begun a 64 bit port of this package?
Thanks
Murray
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at stat.math.ethz.ch https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Edzer Pebesma Institute for Geoinformatics (ifgi), University of M?nster Weseler Stra?e 253, 48151 M?nster, Germany. Phone: +49 251 8333081, Fax: +49 251 8339763 http://ifgi.uni-muenster.de/ http://www.springer.com/978-0-387-78170-9 e.pebesma at wwu.de