On 17/07/2009 7:57 PM, Marilyn & Rich Short wrote:
Hello,
I'm having a problem in R with repeated use of the "load", "save", and
"close" commands. I'm getting an error message that reads, "Too many
open files". I'm opening files and closing them (and unlinking them),
but when I go through that process 509 times, the program halts and I
get this error message: "cannot open the connection" with warning
messages: "Too many open files". I've been working on this problem for
a couple of weeks and have gleaned a bit of info from different internet
threads, but no solutions yet.
I'm using Windows XP, SP3 and R 2.9.1.
Here is my session info:
R version 2.9.1 (2009-06-26)
i286-pc-mingw32
locale:
LC_COLLATE=English_United States.1252;LC_CTYPE=English_United
States. 1252;LC_MONETARY=English_United
States. 1252;LC_NUMERIC=C;LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
I'm using the vanilla load of R, with nothing added after booting up.
The problem also occurs on my Vista machine as well.
The program below will induce the problem.
------------------------------------------------------------
junk = 1
PathA = tempdir()
conX = paste(PathA,"junk",sep="\\")
outJunk = file(conX, open="wb")
save(junk, file=outJunk)
close(outJunk)
for(i in 1:4000){
outMIA = file(conX, open="rb")
load(file=outMIA)
close(outMIA)
closeAllConnections()
unlink(conX)
rm(outMIA)
rm(junk)
cat(" i = ",i,sep=" ")
gc()
zzz = showConnections(all=FALSE)
cat(" zzz = ",zzz,"\n",sep=" ")
}
--------------------------------------------------------------------
There is some talk on the internet that some windows systems have a
limit of 512 files that can be open at one time. Even though I'm closing
my files each time, something is keeping track of how many times I've
opened and closed a file in a session. I've talked to Microsoft and run
a test program in Visual Studio C#, and, at the moment, it looks like
the problem does not lie in the Microsoft arena. The C# program
performed a similar task 10,000 times without a problem. I'm not totally
convinced, but the current evidence says to look elsewhere.
I've attached a script that will induce the problem. However, be warned
that, if you use it, you will have to get out of R after you run it. R
will no longer be able to access files such as help or sessionInfo().
This can be solved by getting out of the R GUI and back in.
R E-mails from as far back as 2006 ask for help on the issue. There have
been several suggestions, but no confirmed solution that I can find. You
will see my attempt at these suggestions in the script [ rm(outMIA);
rm(junk); closeAllConnections(); and gc(); after close(outMIA); and
unlink(conX);]. For me, this becomes important because it limits the
total number if iterations in nested do-loops. This is where I ran into
the problem. The program above will allow you to reproduce the problem.
Any suggestion would be greatly appreciated.
This looks like a bug to me, and so I've posted it to the bugs list, but
your script doesn't make sense for normal use. Before the loop you
create a file named according to conX containing the junk variable, and
then within the loop you load that file, *then delete it*, using
unlink(). However, if you examine the value of unlink(), you'll see
that it fails every time.
Moreover, the use of connections is unnecessary: load() can work with a
filename. So you can skip the file() and close() calls.
If I simplify your script by removing the unnecessary stuff, then it
works. That is, this works:
junk = 1
PathA = tempdir()
conX = paste(PathA,"junk",sep="\\")
save(junk, file=conX)
for(i in 1:4000){
load(file=conX)
rm(junk)
cat(" i = ",i,sep=" ")
gc()
zzz = showConnections(all=FALSE)
cat(" zzz = ",zzz,"\n",sep=" ")
}
So a suggestion for a workaround is simply to follow the pattern above,
rather than that of your demonstration code.
I'll spend a bit more time and see if I can find what is causing the
open file leak.
Duncan Murdoch