Representation of data in libraries

At present the example data sets in R libraries are to be given as
expressions that can be read directly into R.  For example, the acid.R 
file in the main library looks like
 acid <- data.frame(
  carb  = c(0.1, 0.3, 0.5, 0.6, 0.7, 0.9),
  optden = c(0.086, 0.269, 0.446, 0.538, 0.626, 0.782), row.names = paste(1:6))

This is great when you have only a few observations.  I have one
example data set with over 9000 rows and 17 variables.  Even when I
set -v 40, I exhaust the available memory trying to read it in as a
data.frame. 

You need to specify -n some_large_number to read in large data sets,
specifying -v is not enough.  You can see this by using gcinfo(T) to
report heap and cons cell usage at each garbage collection.

Are there alternatives that would cause less memory usage?  In
S/S-PLUS the data.dump/data.restore functions use a portable
representation that can be parsed without exponential memory growth.

The R save() format is portable, at least among Unices. You could have the
data.R file contain the command
	eval(load("data.Rdata"),.GlobalEnv)
where "data.Rdata" is the saved file. There is an ascii=T option, which
might make it more portable to other operating systems.

I haven't checked, but I assume that this format can be read more
efficiently than sourcing R code.


	-thomas

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._