Skip to content

[Bioc-devel] ShortReadQ class and memory usage

2 messages · Mike Smith, Martin Morgan

Original
Mike Smith
# 
Hi all,

I was wondering if anyone could explain the memory usage behaviour I see
when subsetting a ShortReadQ object.

If I create a ShortReadQ with 500 reads, each of length 200bp it uses
~230Kb of memory.  If I do the same, but store 50,000 reads the memory
usage goes up to ~21.5Mb - this seems entirely reasonable.  However, if I
then create a third object by subsetting the first 500 reads from the
larger ShortReadQ it comes in at ~19.5Mb, which seems less understandable.
I was hoping for some insight into why this might be.

I've put some code in a Gist here that can run this example:
https://gist.github.com/grimbough/ca9e668a5772dc00cafb

Thanks,

Mike
Martin Morgan
#
On 03/30/2015 09:04 AM, Mike Smith wrote:
It's actually inherited behavior from XStringSet -- the XStringSet is kind of 
like a 'view' on a single larger object, subsetting the object creates a 
different view but on the same large object. `compact()` actually realizes the 
view as a new instance

   > print(object.size(fq3), units = "auto") ## 19.6 Mb
   19.6 Mb
   > print(object.size(compact(fq3)), units = "auto")
   229.3 Kb

Martin