Skip to content

Class mode text isopen can read can write - too many open connections

4 messages · Santosh Srinivas, Joshua Wiley

Original
Santosh Srinivas
# 
I am downloading data files using RCurl and everything works except till
some limit is hit and says too many connections open 
It is a simple download using URL and I am writing the status in a tryCatch
block to a log file.


showConnections()
     description class mode text isopen can read can write
description class      mode text   isopen   can read can write
0 "stdin"     "terminal" "r"  "text" "opened" "yes"    "no"     
1 "stdout"    "terminal" "w"  "text" "opened" "no"     "yes"    
2 "stderr"    "terminal" "w"  "text" "opened" "no"     "yes"    

I tried closeAllConnections() but of no use.

When I try to delete a downloaded file . It says cannot be done because
"folder is open in R GUI front end"

Any idea how to resolve this?
R version 2.11.1 (2010-05-31) 
i386-pc-mingw32 

locale:
[1] LC_COLLATE=English_United Kingdom.1252  LC_CTYPE=English_United
Kingdom.1252    LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C
LC_TIME=English_United Kingdom.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] RCurl_1.4-4.1  bitops_1.0-4.1 zoo_1.6-4     

loaded via a namespace (and not attached):
[1] grid_2.11.1     lattice_0.19-11 tools_2.11.1
Joshua Wiley
# 
Hi,

Is it a public URL (i.e., that we can try downloading from too)?  Do
you get the same error now matter where/what you download or just from
that one place?  Finally, if you are using Windows > XP, are you
running R as an administrator (or very sure that the log file or
whatever else you are creating is not being written to some place that
Windows will try to restrict such as in Programs)?

Cheers,

Josh

On Sun, Oct 17, 2010 at 8:57 AM, Santosh Srinivas
<santosh.srinivas at gmail.com> wrote:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Santosh Srinivas
    

    

      
      #
    

  


  

      
I'm basically doing using code: .... NOTE THIS IS A RECURSIVE DOWNLOAD ... SO CHANGE TO A DIRECTORY THAT YOU WANT TO JUNK ...

After the download ... try deleting the data without closing R and it says the file is currently held "open" by R ... Not sure how I can close that connection ..
Please let me know any "release" operations that I ened to add in the code too.

library(zoo)
library("RCurl")

x <- seq(as.Date("01-Jan-2010",format="%d-%b-%Y"), Sys.Date(), by=1) #to generate series of dates
#sDate <- x[6]



cmDownFun <- function (sDate)
{
	sMonth <- casefold( as.character(sDate,format="%b"),upper=T) #Get the month
	sYear <- casefold( as.character(sDate,format="%Y"),upper=T) #Get the month
	sDate1 <- casefold( as.character(sDate, format="%d%b%Y"), upper =T) #Get the date
	sURL <- paste("http://www.nseindia.com/content/historical/EQUITIES/",sYear,"/",sMonth,"/cm",sDate1,"bhav.csv.zip", sep="")
	
	tryCatch(
		{
			download.file(sURL,paste("CM",sDate1,".zip",sep="")) #download the file
			print (paste("Successfully downloaded:", paste("CM",sDate1,".zip",sep="")))
			write(paste("Successfully downloaded:", paste("CM",sDate1,".zip",sep="")),file = "Success-Log.txt",append=TRUE,sep="\n")
			closeAllConnections()
			
		},
		warning = function (ex){
				print(paste("Failed to download:", paste("CM",sDate1,".zip",sep="")),file = "Failure-Log.txt",append=TRUE,sep="\n")
				write(paste("Failed to download:", paste("CM",sDate1,".zip",sep="")),file = "Failure-Log.txt",append=TRUE,sep="\n")
				closeAllConnections()
			})
}

#lapply(x, function(x) try(cmDownFun(x),silent = TRUE))
lapply(x, cmDownFun)




-----Original Message-----
From: Joshua Wiley [mailto:jwiley.psych at gmail.com] 
Sent: 17 October 2010 21:39
To: Santosh Srinivas
Cc: r-help at r-project.org
Subject: Re: [R] Class mode text isopen can read can write - too many open connections

Hi,

Is it a public URL (i.e., that we can try downloading from too)?  Do
you get the same error now matter where/what you download or just from
that one place?  Finally, if you are using Windows > XP, are you
running R as an administrator (or very sure that the log file or
whatever else you are creating is not being written to some place that
Windows will try to restrict such as in Programs)?

Cheers,

Josh

On Sun, Oct 17, 2010 at 8:57 AM, Santosh Srinivas

        
<santosh.srinivas at gmail.com> wrote:

            

      
      
      
    

        

      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Joshua Wiley
    

    

      
      #
    

  


  

      
Hi,

I was able to reproduce your problem (I changed the date sequence to
starting "01-Oct-2010", but that does not really matter).  The
interesting thing is that I could delete some files, and they were not
random.  Whenever Windows showed one of the zip folders' size as 0KB,
then I could not delete, but I could delete all the other ones.  It
makes me think there might have been an error in writing those
particular ones which lead both to them not having anything in them
and the connection not shutting down properly.  If the connections
were not shutting down, this would also explain your problem with R
having too many open connections.

I do not really have a solution though (other than shutting down R),
the details of connections and file writing are really beyond my
experience.

Best regards,

Josh

On Sun, Oct 17, 2010 at 7:15 PM, Santosh Srinivas

        
<santosh.srinivas at gmail.com> wrote: