Request for help about running a loop and reading .NC files

Hi,

Rolf is correct that you might consider enclosing your nc_open(...) with
try(nc_open(...)), and then testing if the value returned by try() inherits
from the "try-error" class. If there is no problem opening the NCDF file,
then it will return an "ncdf4" class object.

Also, you might isolate the file extraction stuff into a function to make
your life easier. Something like this (obviously not tested)...


### START
# extract data from a NCDF file
# @param filename the path and name of the file
# @return either NULL (for an exception) or a data.frame
extract_from_nc <- function(filename){

  nc <- try(nc_open(filename))
  if (inherits(nc, "try-error")){
    # if it isn't a healthy NCDF file you will be here
    x <- NULL
  } else {
    # otherwise you will be here and you can run your extraction
    mean1 <- ...
     ...
    x <- as.data.frame(mean_chl_df)
    write.csv(x, ... other stuff here ...)
    nc_close(nc)
  }
  return(x)
}

flist <- list.files(path ="NCFiles/", pattern = "^.*\\.(nc|NC|Nc|Nc)$",
full.names = TRUE)

xx <- lapply(flist, extract_from_nc)
ix <- sapply(xx, is.null)                    # find the NULLS
xx <- xx[!ix]                                # remove them (if any)
if (length(xx) > 0) x <- do.call(rbind, xx)  # row bind what remains into
one data frame (in case that is helpful)
### END

I hope that helps.

Cheers,
Ben





On Mon, Mar 23, 2020 at 5:53 PM Bhaskar Mitra <bhaskar.kolkata at gmail.com>
wrote:

Hello Everyone,

I have written a loop which reads hundreds of .nc files
and extract information from each .nc file and
exports that corresponding information as a csv file.

The loop works fine until it encounters a .nc file which it cannot read
and the loop stops. I would appreciate if anyone can suggest how can I
modify
the loop, whereby the loop will continue to run by bypassing those
particular
files which it cannot read.

In the end I was also hoping to modify the loop such that it
will generate a report which will inform me which
files were not read by the loop.  The codes are given below

Thanks,
Bhaskar Mitra
NAU

#_------------------------------------------------------------------


library(ncdf4)
library(reshape2)
library(dplyr)
library(stringr)


setwd("directory path")

Output <- "directory path"

flist <- list.files(path ="NCFiles/", pattern = "^.*\\.(nc|NC|Nc|Nc)$")

for (i in 1: length(flist))
{

  nc <- nc_open(paste0("NCFiles/",flist[i]))
  mean1 <- ncvar_get(nc,attributes(nc$dim)$names[3])
  nc_close(nc)

  mean_chl_df <- melt(mean1)
  trial.table.df <-as.data.frame(mean_chl_df)


write.csv(trial.table.df,paste0(Output,"/",tools::file_path_sans_ext(flist[i]),".csv"))

}

        [[alternative HTML version deleted]]

--
Ben Tupper
Bigelow Laboratory for Ocean Science
East Boothbay, Maine
http://www.bigelow.org/
https://eco.bigelow.org