Assigning NULL to large variables is much faster than rm() - any reason why I should still use rm()?

Hi,

in my packages/functions/code I tend to remove large temporary
variables as soon as possible, e.g. large intermediate vectors used in
iterations.  I sometimes also have the habit of doing this to make it
explicit in the source code when a temporary object is no longer
needed.  However, I did notice that this can add a noticeable overhead
when the rest of the iteration step does not take that much time.

Trying to speed this up, I first noticed that rm(list="a") is much
faster than rm(a).  While at it, I realized that for the purpose of
keeping the memory footprint small, I can equally well reassign the
variable the value of a small object (e.g. a <- NULL), which is
significantly faster than using rm().

Yes, as you probably noticed rm() is a quite complex function because it has to deal with different ways to specify input etc. 
When you remove that overhead (by calling .Internal(remove("a", parent.frame(), FALSE))), you get the same performance as the assignment.
If you really want to go overboard, you can define your own function:

SEXP rm(SEXP x, SEXP rho) { setVar(x, R_UnboundValue, rho); return R_NilValue; }
poof <- function(x) .Call(rm_C, substitute(x), parent.frame())

That will be faster than anything else (mainly because it avoids the trip through strings as it can use the symbol directly).

But as Bill noted - it practice I'd recommend using either local() or functions to control the scope - using rm() or assignments seems too error-prone to me.

Cheers,
Simon
SOME BENCHMARKS:
A toy example imitating an iterative algorithm with "large" temporary objects.

x <- matrix(rnorm(100e6), ncol=10e3)

t1 <- system.time(for (k in 1:ncol(x)) {
 a <- x[,k]
 colSum <- sum(a)
 rm(a) # Not needed anymore
 b <- x[k,]
 rowSum <- sum(b)
 rm(b) # Not needed anymore
})

t2 <- system.time(for (k in 1:ncol(x)) {
 a <- x[,k]
 colSum <- sum(a)
 rm(list="a") # Not needed anymore
 b <- x[k,]
 rowSum <- sum(b)
 rm(list="b") # Not needed anymore
})

t3 <- system.time(for (k in 1:ncol(x)) {
 a <- x[,k]
 colSum <- sum(a)
 a <- NULL # Not needed anymore
 b <- x[k,]
 rowSum <- sum(b)
 b <- NULL # Not needed anymore
})

t1
  user  system elapsed
  8.03    0.00    8.08
t1/t2
   user   system  elapsed
1.322900 0.000000 1.320261
t1/t3
   user   system  elapsed
1.715812 0.000000 1.662551

Is there a reason why I shouldn't assign NULL instead of using rm()?
As far as I understand it, the garbage collector will be equally
efficient cleaning out the previous object when using rm(a) or a <-
NULL.  Is there anything else I'm overlooking?  Am I adding overhead
somewhere else?

/Henrik

PS. With the above toy example one can obviously be a bit smarter by using:

t4 <- system.time({for (k in 1:ncol(x)) {
 a <- x[,k]
 colSum <- sum(a)
 a <- x[k,]
 rowSum <- sum(a)
}
rm(list="a")
})

but that's not my point.

______________________________________________
R-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Assigning NULL to large variables is much faster than rm() - any reason why I should still use rm()?

Thread (4 messages)