read data into R with some constraints
On Thu, 11 Jan 2001, Yu-Ling Wu wrote:
Hi,
I have a big data file (over 30,000 records) looks
like this:
100, 20, 46, 70
103, 0, 22, 45
117, -1, 34, 65
120, 15, 0, 25
113, 0, -1, 32
142, -1, -1, 55
.....
I want to read only those records having positive
values in all of the four
columns. That is, I don't want to read record # 3, 5,
and 6 into R. However,
when I type:
read.csv("data.csv", sep=",") -> rawdata
Um, read.csv uses sep =",", and you need header=FALSE.
it reads the whole thing into R including those records I don't want. Could anyone tell me how I can read only those records I want?
You can't! Until you have read the record, you cannot tell if all the
entries are positive.
Is this really a problem? You only have around 120k numbers, and I just
did it very easily.
rawdata <- read.csv("data.csv", header=F)
Perhaps better is to use a matrix and scan():
rawdata <- matrix(scan("data.csv", sep=","), , 4, byrow=TRUE)
keep <- (rawdata <= 0) %*% rep(1,4) == 0
rawdata[keep, ]
Takes a few seconds and a few Mb.
Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272860 (secr) Oxford OX1 3TG, UK Fax: +44 1865 272595 -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._