Memory Leak

Tue, Feb 1, 2011 9:00 PM

Hello again,

Thanks, that explanation helps me understand the issue more. My platform is
Platform: x86_64-unknown-linux-gnu (64-bit) (Ubuntu 10.04 to be more precise).

- Dario.

---- Original message ----

Date: Tue, 1 Feb 2011 21:29:38 -0500
From: Simon Urbanek <simon.urbanek at r-project.org>  
Subject: Re: [Rd] Memory Leak  
To: D.Strbenac at garvan.org.au
Cc: r-devel at r-project.org


On Feb 1, 2011, at 9:00 PM, Dario Strbenac wrote:

Hello,

I'm trying to track down the cause of some extreme memory usage and I've been using Dirk Eddelbuettel's lsos() function he posted on stack overflow. There is a large difference between R's RAM usage :

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
6637 darstr    20   0 30.0g  29g 4712 S    0 63.2  10:34.43 R 

and what objects I have loaded in memory :

lsos()

          Type      Size PrettySize     Rows Columns
A          list 552387720   526.8 Mb        2      NA
B   GRangesList 552376408   526.8 Mb        4      NA
C SimpleRleList 353421896     337 Mb       24      NA
D       GRanges 236410608   225.5 Mb 15272853      NA
E    data.frame   6981952     6.7 Mb    24966      14
F    data.frame   6782136     6.5 Mb    24966      13
G          list   4393704     4.2 Mb    24964      NA
H        matrix   3195760       3 Mb    24964      16
I          list   1798752     1.7 Mb    24964      NA
J       GRanges    312656   305.3 Kb    24964      NA

(The total looks like about 1.5 GB)

I haven't got any calls to external C code in my R script, although the Bioconductor packages I am using do. How can I regain those missing Gigabytes ?

There is no reason why the two numbers should have anything in common. The OS (assuming the above is supposed to be ps output) reports memory that the OS reserved (it is not necessarily what the application is currently using) and lsos (assuming you meant to point to this: http://stackoverflow.com/questions/1358003/tricks-to-manage-the-available-memory-in-an-r-session ) simply shows approximate size estimates of objects in your workspace. So you can expect the former to be larger than the latter (often but not necessarily always) but that's about all you can say.

If you want to know what R objects are allocated, just look at gc() - it has the actual numbers. However, that only includes transient allocations using the R GC, any memory reserved directly from the OS will not be included. Furthermore, many OSes don't release memory from processes for performance reasons so even if you had exact allocation numbers they would not match what you see in ps.

I you want to lean more, you'll have to grab the tools available for your platform to find out who's allocating what. Since you didn't even mention your platform I can't really help you with the specifics, though.

Cheers,
Simon