Message-ID: <08BBFB7E-00CE-41A0-A4F4-7D4DE07C7757@dcn.davis.ca.us>
Date: 2021-10-06T19:11:00Z
From: Jeff Newmiller
Subject: RSQLite slowness
In-Reply-To: <YV3v0y6SEJdLyrOb@posteo.no>
Since the sqlite package is contributed, it is NOT related to "core R", and is in fact technically off-topic on this list.
FWIW all SQL implementations work better with indexes, but AFAIK the R data frame support does nothing with indexes. This may be related to your question, or not. I am not a regular sqlite user.
As for fast reading of tsv files, I think arrow, readr, and data.table packages all offer high-performance import functions that could be relevant.
On October 6, 2021 11:49:55 AM PDT, Rasmus Liland <jral at posteo.no> wrote:
>Thank you Bert, I set up a new thread on
>BioStars [1]. So far, I'm a bit
>unfamilliar with Bioconductor (but will
>hopefully attend a course about it in
>November, which I'm kinda hyped about),
>other than installing and updating R
>packages using BiocManager .... Did you
>think of something else than
>BioStars.org when saying ?Bioconductor??
>
>The question could be viewed as gene
>related, but I think it is really about
>how can one easier than with sqlite
>handle large tsv files, and why is that
>parser thing so slow ... I think this
>is more like a core R thing than gene
>related question ...
>
>[1] https://www.biostars.org/p/9492486/
--
Sent from my phone. Please excuse my brevity.