Skip to content

still struggling with FTP download on MODIS

2 messages · Alexandre VILLERS, Tomislav Hengl

#
Dear all,

I'm still not sure that this list is the right place for that. So if 
those questions bother you, please give me the name of the list I should 
contact.

Here a version of the script I'm using to download MODIS data from NSIDC 
(weekly snow cover)

Tsnow <- 
"ftp://anonymous:test at n4ftl01u.ecs.nasa.gov/SAN/MOST/MOD10A2.005/"  
###server of NSIDC with directories / id and password
myfolder<-paste(tempdir(),"\\" , sep="") #the folder to download data

#####-getting the list of the folders in Tsnow (thanks to Barry Rowlingson)
resp <-getURL(Tsnow)
  items <- strsplit(resp,"\r\n")[[1]]
  folderLines <- items[substr(items,1,1)=='d']
  lastBit <- function(x){x[length(x)]}
  dates <- unlist(lapply(strsplit(folderLines," "),lastBit))
  datesK<-data.frame(dates=dates, keep=rep(NA,length(dates)))
  
##### select months for which I want the data
months<-c("01","02","03","04","05","06","10", "11", "12") #months for 
which I want the data
        for (i in months){ #loop to keep only the dates I'm interested in
              garde<-grep(datesK$dates, pattern=paste(".",i,".", sep=""))
              datesK$keep[garde]<-1
              }
datesK<-subset(datesK, datesK$keep==1) # keep the dates of interest

for (i in 1:length(datesK[,1])){    ## loop to download the tiles of 
interest
        getlist1 <- strsplit(getURL(paste(Tsnow, datesK[i,][[1]], "/", 
sep=""),.opts=curlOptions(ftplistonly=TRUE)), "\r\n")[[1]] #get the 
files in the folder i
        BLOCKtemp <- getlist1[grep(getlist1, 
pattern="MOD10A2.********.h18v04.005.*************.hdf")[1]] #select the 
tile of interest (in my case h18v04.005)
        download.file(paste(Tsnow, datesK[i,][[1]], "/", 
BLOCKtemp,sep=""), destfile=paste(myfolder, BLOCKtemp, sep=""), 
mode='wb', method='wget') #download the tile
        }

So here is my problem. I can't get into some specifc folders and receive 
the error message.
Error in curlPerform(curl = curl, .opts = opts, .encoding = .encoding) :
  server did not report OK, got 550
For example, i=4 (folder "2000.03.13"). And this is always true, 
suggesting it's not a problem of connection. The error appears as soon 
as getURL is called in the loop.
I succeed in downloading for some values of i (manually) but when 
looping, it stops on those "black" folders. I've checked directly from 
filezilla or my web browser: I can get into those folder and download 
the tiles.

The tiles are pretty small (150 to 450 Ko). So if someone could try from 
his place to see if he gets the same error message for the same value of 
i (4, 28,31, etc.).
Since I have 550 tiles to download and process, I would prefer not to do 
that manually (and I can't get to use the tool proposed by the NSIDC for 
the moment).

Thanks for the help

Alex

P.S.: my next question will deal with some projection problem !!!
#
The ftp error code "550" means: "Requested action not taken. File unavailable (e.g., file not found,
no access).", which means that the ftp server is not allowing you access.

Did you try checking your wget tool? I remember I had headaches before I manage to get the wget
running without problems. Make sure you set:
wget.exe you need to put in your Windows directory folder (Note: make sure you disable your
antivirus tools such as Norton or McAfee otherwise it might block wget from running);
T. Hengl
http://spatial-analyst.net