SQLite: When reading a table, a "\r" is padded onto the last column. Why?
ronggui <ronggui.huang at gmail.com> writes:
On 1/6/07, Seth Falcon <sfalcon at fhcrc.org> wrote:
I wonder if things have changed at all in that regard. I would expect read.table to be slower, but, if called carefully, not _extremely_ slower. We should do some timings...
Yeah, Things change a lot. I use system.time to do the timings on a large csv file (365M, with sep="\t", quote="") The result is as follow: 1, use read.table and then dbWriteTable in chunck.
t1
[1] 700.22 16.06 768.01 NA NA
Can you provide the exact call to read.table you used? To give it a fair chance, you need to use the colClasses arg in read.table and choose a reasonable batch size (I think the buffer hard-coded in the RSQLite C code is 1024, so to compare you should choose a number of rows that is ~1MB). + seth