Possible memory leak for serialize?
Clearly no memory leak as such: if you rm(out) and gc() then OS usage drops down and repeating your example/rm(out) stays stable. It does seem that 'out' is holding on to more memory than it should; I'll try to have a look when I get a chance. Best, luke
On Wed, 5 Sep 2012, Hao Yu wrote:
Hi. serialize may have memory leak issue. When I run the following codes x=split(1:40000, rep(1:8, 5000)) fun=function(x)lapply(x, serialize, NULL) out =replicate(5000, fun(x)) After using gc() severally times, R reports about 800MB memory usage (objects fun, out and x) but the real memory usage is 2.4GB (reported from OS). After deleting all objects, R reports about 12MB usage but real memory usage is reduced to 1.7GB and will stay there. Don't try with big size. I used up all my 16GB ram on my windows PC and caused other programs to crash. I can reproduce this behave on both Windows and Debian with R 2.15.1 64bit. If this memory leak does exist, it may have impact on jobs with lots of R objects exchanges among nodes, in particularly master node. Hao PS: Changing fun to fun2=function(x)lapply(1:8, function(i)serialize(x[[i]],NULL)) has no effect.
Luke Tierney
Chair, Statistics and Actuarial Science
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa Phone: 319-335-3386
Department of Statistics and Fax: 319-335-3017
Actuarial Science
241 Schaeffer Hall email: luke-tierney at uiowa.edu
Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu