Huge data frames?
RODBC definitely uses more memory than read.table. One day I'll rewrite RODBC to use the more efficient internals of the current read.table.
On Wed, 28 Aug 2002 chr.schulz at email.de wrote:
..perhaps you have more access
when you use the Rodbc Package and MySQL as
Data-Store !?
channel <- odbcConnect("db_name","login","pass")
data <- sqlFetch(channel, data, errors = TRUE, as = "data frame", nullstring = "sysmis", na.strings = "NA")
..or sqlQuery !
good luck ;-)
regards,christian
Magnus Lie Hetland <magnus at hetland.org> schrieb am 28.08.02 07:44:33:
A friend of mine recently mentioned that he had painlessly imported a data file with 8 columns and 500,000 rows into matlab. When I tried the same thing in R (both Unix and Windows variants) I had little success. The Windows version hung for a very long time, until I eventually more or less ran out of virtual memory; I tried to set the proper memory allocations for the Unix version, but it never seemed satisfied :] I used read.table -- should I have used something else? Is it even possible to work with this large files? I assume a memory-mapped binary file would have been quite efficient (as opposed to an in-memory parsed text file) -- is something like that even possible in R? -- Magnus Lie Hetland The Anygui Project http://hetland.org http://anygui.org -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272860 (secr) Oxford OX1 3TG, UK Fax: +44 1865 272595 -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._