Skip to content

textConnection taking a long time to open a big string

2 messages · james.holtman@convergys.com, Thomas W Blackwell

james.holtman@convergys.com
# 
I was using 'textConnection' to read in a file with about 11,000 lines so I
could detect lines with incomplete data and delete them and then read them
in with 'scan'.  I am using 1.7.0 on Windows.  Here is the output from the
script and it was using 51 seconds just to do the textConnection.

Is there a limit on how large a text object can be to be used with
'textConnection'?

########   script output    ################
Read 11299 items
chr [1:11299] "8.3155  32   71   4 1907   122    0 1130  105  167  216
0  3686   32  13  37  18" ...
[1] 51.96  0.01 53.20    NA    NA
[1] 944525
[1] 0.14 0.00 0.14   NA   NA
x.c
    3     6    17
    1     1 11297
_
platform i386-pc-mingw32
arch     i386
os       mingw32
system   i386, mingw32
status
major    1
minor    7.0
year     2003
month    04
day      16
language R
--
"NOTICE:  The information contained in this electronic mail tran... {{dropped}}
Thomas W Blackwell
# 
Two alternate ways to the same result:

x.1 <- scan(file=, what=rep(list(0),17), fill=T, multi.line=F)
incomplete.lines <- seq(length(x.1[[17]]))[ is.na(x.1[[17]] ]

x.1 <- scan(file=, what='')
x.2 <- strsplit(x.1, "[\\t ]")
incomplete.lines <- seq(length(x.1))[ unlist(lapply(x.2, length)) < 17 ]

Please read the help for these functions.

HTH  -  tom blackwell  -  u michigan medical school  -  ann arbor  -
On Wed, 30 Apr 2003 james.holtman at convergys.com wrote: