Raw Message

Message-ID: <39B5ED61E7BFC24FA8277B6DE92A9A3F041541B6@fkimlki01.enterprise.afmc.ds.af.mil>
Date: 2011-10-03T18:26:55Z
From: Cable, Sam B Civ USAF AFMC AFRL/RVBXI
Subject: file input with readLines

More on my previous question ...

I have put in timing statements to try to get a better idea of where the
problem is, like so:

conn<-file('filename','r')

for (chunk in 1:100000) {
   print(paste('begin read at',date()))
   Lines<-readLines(conn,n=25)
   print(paste('begin processing at',date()))
  # process "Lines"
   print(paste('end loop at',date()))
}

Every time I go through the loop, all the date() functions return
*exactly* the same time!  It *looks like* it runs through each iteration
very quickly and then takes longer and longer to simply start the next
iteration.  I don't believe this.  I think R must be doing some kind of
latency trick or something.  But, anyway, the point is that I was
assuming the problem was in the I/O, and now I don't know if it's I/O or
processing.  Either way, I don't understand it and would really
appreciate some wisdom from you guys.

Thanks.