Hi all,
After reading this help post:
https://stat.ethz.ch/pipermail/r-help/2011-November/297141.html
I think I have the
"issue in the Windows 64-bit NetCDF external dependency"
and would like to know whether it is possible to fix it, or whether I
should change computer to solve my problem (which could take some time,
given I have a lot of files to transfer and process).
Here is my problem :
- I have to read a large ncdf file (3.8 gigas)
- I have 20 variables in it, each variable's dimension is 250 (lon ) * 270 (lat) * 730 (time)
- My computer uses windows 7 64 bit
- I have installed R version 2.15.2 (64 bits)
- I am using the ncdf library, the open.ncdf function works fine.
- I use function get.var.ncdf to retrieve the variables.
- when I read the ncdf one time step at a time, it reads ok from time steps 1 to 418
- R crashes whenever I try to read any of the variables for a time step
>= 419 (no matter how much I "zoom" the spatial dimensions, and even
if I try to read only one time step over a small spatial extent)
with the following message :
"Microsoft Visual C++ Runtime Library:
This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information"
On another computer (runing linux) :
- ferret is able to retrieve an entire variable
- matlab is able to retrieve an entire variable
- R is able to retrieve an entire variable (not a problem of memory)
- if we cut the ncdf to include only the 418 first time steps, the new ncdf is then... 2.0 gigas
On my computer:
- if I use a ncdf that has been "cut" spatially on another computer, I
can extract any entire variable, over the 730 time steps (the new ncdf is then 1.8 Gigas instead of
3.8 gigas)
- if I use a ncdf that keeps all the spatial information, and all the
time steps, but with only one variable in it, I can read this entire
variable (the new ncdf is then 800 megas)
So it looks as if R were having trouble extracting variables from ncdf files of more than 2 gigas when runing under windows.
- Has anyone got any explanation for this (I thought the big advantage of working wth ncdf files was the possibility to handle huge files, because R does not have to read the entire file, to read time step 419, is R putting all the previous information into memory ?)
- Is there a way to fix this ? (or should I process all my files on another computer ?)
Thank you in advance for all your answers.
Magali