Unexpected values obtained when reading in data using ncdf and ncdf4

On Apr 22, 2016, at 8:08 AM, David W. Pierce <dpierce at ucsd.edu> wrote:

On Fri, Apr 22, 2016 at 1:32 AM, Louise Mair <louise.mair at slu.se> wrote:

Dear R Users,

I am encountering a problem when reading nc files into R using the 
ncdf and ncdf4 libraries. The nc files are too large to attach an 
example (but if someone is interested in helping out I could send a 
file privately via an online drive), but the code is basic:

?[...]?


?Hi Louise,

I'm the author of the ncdf and ncdf4 libraries. What are the details 
-- what operating system are you running on, what version of R and the 
netcdf library are you using?

If you make the files available to me I can take a look.

Regards,

--Dave Pierce
?

for(i in 1:length(thesenames[,1])){
  data <- nc_open(paste(INDIR, thesenames[i,c("wholename")], sep=""),
write=F)
  d.vars <- names(data$var)
  d.size <- (data$var[[length(d.vars)]])$size

  # Obtaining longitude and latitude values
  d.lon <- as.vector(ncvar_get(data, varid="lon", start=c(1,1),
count=c(d.size[1],d.size[2])))
  d.lat <- as.vector(ncvar_get(data, varid="lat", start=c(1,1),
count=c(d.size[1],d.size[2])))

  # Obtaining climate data values
  df.clim <- data.frame(rn=seq(1:length(d.lon)))
  for(y in 1:d.size[3]){
    df.clim[,1+y] <- as.vector(ncvar_get(data, 
varid=d.vars[length(d.vars)], start=c(1,1,y),
count=c(d.size[1],d.size[2],1)))
     names(df.clim)[1+y] <- paste("y",y,sep="")  }
  tosummarise[,,i] <- as.matrix(df.clim[,-1]) }

The data are temperature or precipitation, across space and time.

For most of the >250 files I have, there are no problems, but for 
around 8 of these files, I get strange values. The data should be 
within a relatively narrow range, yet I get values such as 
-8.246508e+07  or 7.659506e+11. The particularly strange part is that 
these kind of values occur at regularly spaced intervals across the 
data, usually within a single time step.

I have the same problem (including the exact same strange values) 
when using ArcMap, yet the data provider assures me that the data 
look normal when using CDO (climate data operators) to view them, and 
that there are no missing values.

I realise this is very difficult to diagnose without the nc files 
themselves, so my questions are (1) Has anyone encountered something 
like this before?, (2) Is there something I am failing to specify in 
the code when reading in?, and (3) Is anyone interested in digging 
into this and willing to play around with the nc files if I make them available privately?

Thanks very much in advance!
Louise





       [[alternative HTML version deleted]]

--
David W. Pierce
Division of Climate, Atmospheric Science, and Physical Oceanography 
Scripps Institution of Oceanography, La Jolla, California, USA
(858) 534-8276 (voice)  /  (858) 534-8561 (fax)    dpierce at ucsd.edu

	[[alternative HTML version deleted]]