RSQLite slowness
Since the sqlite package is contributed, it is NOT related to "core R", and is in fact technically off-topic on this list. FWIW all SQL implementations work better with indexes, but AFAIK the R data frame support does nothing with indexes. This may be related to your question, or not. I am not a regular sqlite user. As for fast reading of tsv files, I think arrow, readr, and data.table packages all offer high-performance import functions that could be relevant.
On October 6, 2021 11:49:55 AM PDT, Rasmus Liland <jral at posteo.no> wrote:
Thank you Bert, I set up a new thread on BioStars [1]. So far, I'm a bit unfamilliar with Bioconductor (but will hopefully attend a course about it in November, which I'm kinda hyped about), other than installing and updating R packages using BiocManager .... Did you think of something else than BioStars.org when saying ?Bioconductor?? The question could be viewed as gene related, but I think it is really about how can one easier than with sqlite handle large tsv files, and why is that parser thing so slow ... I think this is more like a core R thing than gene related question ... [1] https://www.biostars.org/p/9492486/
Sent from my phone. Please excuse my brevity.