Problem reading large tables - R-help

Tue, Jan 6, 2004 11:03 AM #

Hi, 
	I'm trying to read in a fairly large (92 observations by 3680 variables)
table into R from a space-delimited text file (attached) using the command: d8
<- read.table('d8.r', header=T). The function call runs to completion, and I
get back a valid table object. However, starting at column 999, the table
records the value TRUE when it should record T (T's in columns 998 and earlier
are fine). I've looked at the data file, and I can see no difference between
(e.g.) the T at position 998 in row 1 and the T in position 999 in row 1, yet
998 is recorded as T and 999 as TRUE. 

I know I could just update the table in R to change all TRUEs to Ts, but I'm
worried there may be some underlying limit I'm running up against. I've tried
this on both R-1.7.1 and 1.8.1 on Linux/IA-32. Can anyone help?

Thanks,
Daniel

Daniel Myers
Laboratory of Dr. Michael P. Cummings
Center for Bioinformatics and Computational Biology
University of Maryland
Agri/LFSc Surge Building #296
College Park, MD 20742-3360
dmyers at umiacs.umd.edu
301.405.1262 work
http://serine.umiacs.umd.edu/personnel/myers_daniel/
-
"Think of your breed; for brutish ignorance / Your mettle was not made; you 
were made men / To follow after knowledge and excellence."
As if I also was hearing for the first time: like the blast of a trumpet, 
like the voice of God. For a moment I forget who I am and where I am.
--Primo Levi, Survival in Auschwitz

Duncan Murdoch

Tue, Jan 6, 2004 11:43 AM #

On Tue, 6 Jan 2004 14:03:47 -0500, Daniel Sumers Myers
<dmyers at umiacs.umd.edu> wrote :

The special-looking value 999 is probably just a coincidence.  Likely
what happened is that column 999 was the first column that looked to
the type.convert function like   a purely logical column (because all
values are T there?).  You can tell R not to automatically convert
values by using the colClasses argument to read.table, e.g. colClasses
= "character" forces everything to stay as a character.

Duncan Murdoch

P.S. You can't send attachments to the mailing list, so I didn't see
your data file.

Daniel Sumers Myers

Tue, Jan 6, 2004 11:51 AM #

Thanks to all, it was a conversion problem (now fixed).

Daniel

On Tue, Jan 06, 2004 at 02:43:22PM -0500, Duncan Murdoch wrote:

Daniel Myers
Laboratory of Dr. Michael P. Cummings
Center for Bioinformatics and Computational Biology
University of Maryland
Agri/LFSc Surge Building #296
College Park, MD 20742-3360
dmyers at umiacs.umd.edu
301.405.1262 work
http://serine.umiacs.umd.edu/personnel/myers_daniel/
-
"Think of your breed; for brutish ignorance / Your mettle was not made; you 
were made men / To follow after knowledge and excellence."
As if I also was hearing for the first time: like the blast of a trumpet, 
like the voice of God. For a moment I forget who I am and where I am.
--Primo Levi, Survival in Auschwitz

Martin Maechler

Wed, Jan 7, 2004 5:11 AM #

Duncan> On Tue, 6 Jan 2004 14:03:47 -0500, Daniel Sumers
    Duncan> Myers <dmyers at umiacs.umd.edu> wrote :

    ..........

    Duncan> P.S. You can't send attachments to the mailing list,
yes, you can, if you use the proper "Content-Type", e.g.,
'text/plain' is ok, see below.

    Duncan> so I didn't see your data file.

He forgot it, and then tried to resend it.  That second try
failed because the full length message became over 500k bytes;
and these currently are completely filtered irrespectively of
content (one needs a limit for large messages: these could bog
down the machine(s) that do the mail filtering).

On the topic of e-mail attachments,
http://www.R-project.org/mail.html#instructions  says

 >> Furthermore, most binary e-mail attachments are not accepted,
 >> i.e., they are removed from the posting completely. As an
 >> exception, we allow application/pdf, application/postscript,
 >> and image/png (and x-tar and gzip on R-devel). You can use
 >> text/plain as well, or simply paste text into your message instead.

Regards,
Martin Maechler <maechler at stat.math.ethz.ch>	http://stat.ethz.ch/~maechler/
Seminar fuer Statistik, ETH-Zentrum  LEO C16	Leonhardstr. 27
ETH (Federal Inst. Technology)	8092 Zurich	SWITZERLAND
phone: x-41-1-632-3408		fax: ...-1228			<><