Skip to content
Prev 274758 / 398506 Next

How to read data sequentially into R (line by line)?

johannes rara 
Thanks Jim,

I tried to convert this solution into my situation (.txt file as an input);

zz <- file("myfile.txt", "r")

fileNo <- 1  # used for file name
buffer <- NULL
repeat{
   input <- read.csv(zz, as.is=T, nrows=1000000, sep='!',
row.names=NULL, na.strings="")
   if (length(input) == 0) break  # done
   buffer <- c(buffer, input)
   # find separator
   repeat{
       indx <- which(grepl("^GG!KK!KK!", buffer))[1]
       if (is.na(indx)) break  # not found yet; read more
       writeLines(buffer[1:(indx - 1L)]
           , sprintf("newFile%04d.txt", fileNo)
           )
       buffer <- buffer[-c(1:indx)]  # remove data
       fileNo <- fileNo + 1
   }
}

but it gives me an error

Error in read.table(file = file, header = header, sep = sep, quote = quote,  :
  no lines available in input
Do you know a reason for this?

-J

2011/10/18 jim holtman <jholtman at gmail.com>:

Thread (8 messages)

johannes rara How to read data sequentially into R (line by line)? Oct 18 jim holtman How to read data sequentially into R (line by line)? Oct 18 johannes rara How to read data sequentially into R (line by line)? Oct 18 jim holtman How to read data sequentially into R (line by line)? Oct 18 johannes rara How to read data sequentially into R (line by line)? Oct 18 jim holtman How to read data sequentially into R (line by line)? Oct 18 johannes rara How to read data sequentially into R (line by line)? Oct 18 jim holtman How to read data sequentially into R (line by line)? Oct 18