Hello everyone
My problem is the following : basically I need to pick up daily
precipitation data from a netcdf file (50000 days) to create 50000 pcraster
maps (one for each day).
I am therefore doing a loop using RGDAL that looks like:
/for (d in 1:50000){
tget<-readGDAL("R:/mypath/target.map") ##here I load my .map target file
tget at data<-new_data[d] #I change the data (replacing with the on efrom
Netcdf)
writeGDAL(tget, fname=name_out,
drivername='PCRaster',type="Float32",mvFlag=-9999) #export to .map files
}
/
This works well until d=509 (and outputs are perfectly fine) and then it
crashes, and I got the error message
/Error in .local(.Object, ...) : Dataset copy failed
Calls: writeGDAL ... saveDataset -> new -> initialize -> initialize ->
.local -> .Call
/
The point is that if I start the loop at e.g. d=100, it crashes at 509+100.
So, it does not seem to be a problem of syntax.
So I suspect it may be a problem of memory (but I have enough disk space
left)?.maybe I should close the connection everytime but GDAL.close(tget)
does not work
/Error in slot(dataset, "handle") :
no slot of name "handle" for this object of class "SpatialGridDataFrame"
/
The very weird thing is that it happens when I run it on the cluster. If
doing it locally (windows), I have no problem when d=509?
Do you have any idea?
Best,
--
View this message in context: http://r-sig-geo.2731867.n2.nabble.com/RGDAL-Error-with-writeGDAL-to-pcraster-maps-in-a-loop-tp7586213.html
Sent from the R-sig-geo mailing list archive at Nabble.com.
[RGDAL] Error with writeGDAL to pcraster maps (in a loop)
3 messages · Zack Holden, Philippe R
Hi Phillipe, I've had problems in the past reading many NetCDF files with having too many open files. I don't know if this is related to the issue you're seeing, but I'm attaching a link to to my solution for that problem. You could have an upper limit set on your machine to the number of files that can be opened. http://r-sig-geo.2731867.n2.nabble.com/solved-raster-ncdf4-error-too-many-open-files-td7583723.html Also, it might help others diagnose your problem if you tell us what OS you are using and provide information about your R session. I hope this helps, Zack On Tue, Apr 15, 2014 at 3:12 AM, Philippe R
<philippe.roudier at jrc.ec.europa.eu> wrote:
Hello everyone
My problem is the following : basically I need to pick up daily
precipitation data from a netcdf file (50000 days) to create 50000 pcraster
maps (one for each day).
I am therefore doing a loop using RGDAL that looks like:
/for (d in 1:50000){
tget<-readGDAL("R:/mypath/target.map") ##here I load my .map target file
tget at data<-new_data[d] #I change the data (replacing with the on efrom
Netcdf)
writeGDAL(tget, fname=name_out,
drivername='PCRaster',type="Float32",mvFlag=-9999) #export to .map files
}
/
This works well until d=509 (and outputs are perfectly fine) and then it
crashes, and I got the error message
/Error in .local(.Object, ...) : Dataset copy failed
Calls: writeGDAL ... saveDataset -> new -> initialize -> initialize ->
.local -> .Call
/
The point is that if I start the loop at e.g. d=100, it crashes at 509+100.
So, it does not seem to be a problem of syntax.
So I suspect it may be a problem of memory (but I have enough disk space
left)?.maybe I should close the connection everytime but GDAL.close(tget)
does not work
/Error in slot(dataset, "handle") :
no slot of name "handle" for this object of class "SpatialGridDataFrame"
/
The very weird thing is that it happens when I run it on the cluster. If
doing it locally (windows), I have no problem when d=509?
Do you have any idea?
Best,
--
View this message in context: http://r-sig-geo.2731867.n2.nabble.com/RGDAL-Error-with-writeGDAL-to-pcraster-maps-in-a-loop-tp7586213.html
Sent from the R-sig-geo mailing list archive at Nabble.com.
_______________________________________________ R-sig-Geo mailing list R-sig-Geo at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
1 day later
Considering what i have read on several forums, it may come from a problem in the GDAL.close function in a previous version of rgdal package ( I am using rgdal 0.8-11). So updating the package would normally solve the problem... -- View this message in context: http://r-sig-geo.2731867.n2.nabble.com/RGDAL-Error-with-writeGDAL-to-pcraster-maps-in-a-loop-tp7586213p7586237.html Sent from the R-sig-geo mailing list archive at Nabble.com.