readGDAL does not close connections
On Thu, 6 Mar 2008, Paul Hiemstra wrote:
Dear r-sig-geo list,
I'm struggeling with readGDAL (rgdal package) not closing its file
connections (I suspect). I am trying to construct a timeseries of
rainfall using the following loop (conceptual code):
pt = SpatialPoint()
rainfall = numeric(length = length(listofiles))
for(i in 1:length(listofiles)) {
grid = readGDAL(listofiles[i])
rainfall[i] = grd$value[overlay(grid, pt)]
}
There are more than 35000 grids (HDF5) that I loop over (2007 - 15
minutes interval). After reading about 1000 grids my program fails:
HDF5:"radar/2007/01/11/RAD_NL21_PCP_NA_200701111515.h5"://image1/image_data
has GDAL driver HDF5Image
and has 256 rows and 256 columns
Error in gzfile(file, "r") : unable to open connection
In addition: Warning message:
In gzfile(file, "r") :
cannot open compressed file '/home/hiemstra/lib/R/rgdal/DESCRIPTION'
After an error like this, please also report what traceback() says. A connection is not an open file. I have just run a loop of 10000 readGDAL() without any connections left open - showConnections() shows open connections - and indeed as far as I know, readGDAL() does not itself use any connections mechanisms. In your case, something is trying to read: /home/hiemstra/lib/R/rgdal/DESCRIPTION using a connections mechanism, but without traceback(), we don't know what. Roger
I suspect that readGDAL opens connections to the files, but does not close them and the program fails because too many files have been openened simultaneously. I do not get messages like "Closing GDAL dataset handle 0x8ff7900... destroyed ... done." when I run the script. Does anybody on the r-sig-geo list have similar experiences? kind regards, Paul ps Additional information on my session
sessionInfo()
R version 2.6.1 (2007-11-26) i486-pc-linux-gnu locale: LC_CTYPE=en_US.UTF-8;LC_NUMERIC=C;LC_TIME=en_US.UTF-8;LC_COLLATE=en_US.UTF-8;LC_MONETARY=en_US.UTF-8;LC_MESSAGES=en_US.UTF-8;LC_PAPER=en_US.UTF-8;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_US.UTF-8;LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] rgdal_0.5-24 maptools_0.7-7 foreign_0.8-23 automap_0.4-3 classInt_0.1-9 [6] e1071_1.5-17 class_7.2-41 gstat_0.9-43 sp_0.9-23 loaded via a namespace (and not attached): [1] grid_2.6.1 lattice_0.17-6 rcompgen_0.1-17 I run Debian Lenny
Roger Bivand Economic Geography Section, Department of Economics, Norwegian School of Economics and Business Administration, Helleveien 30, N-5045 Bergen, Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43 e-mail: Roger.Bivand at nhh.no