lapply (and friends) with data.frames are slow
On Sat, Jan 5, 2013 at 7:38 PM, Kevin Ushey <kevinushey at gmail.com> wrote:
Hey guys,
I noticed something curious in the lapply call. I'll copy+paste the
function call here because it's short enough:
lapply <- function (X, FUN, ...)
{
FUN <- match.fun(FUN)
if (!is.vector(X) || is.object(X))
X <- as.list(X)
.Internal(lapply(X, FUN))
}
Notice that lapply coerces X to a list if the !is.vector || is.object(X)
check passes.
Curiously, data.frames fail the test (is.vector(data.frame()) returns
FALSE); but it seems that coercion of a data.frame
to a list would be unnecessary for the *apply family of functions.
Is there a reason why we must coerce data.frames to list for these
functions? I thought data.frames were essentially just 'structured lists'?
I ask because it is generally quite slow coercing a (large) data.frame to a
list, and it seems like this could be avoided for data.frames.
Note sure it's a huge deal, but
It does seem to be an avoidable function call with something like this:
lapply1 <- function (X, FUN, ...)
{
FUN <- match.fun(FUN)
if (!(is.vector(X) && is.object(X) || is.data.frame(X)))
X <- as.list(X)
.Internal(lapply(X, FUN))
}
On a microbenchmark:
xx <- data.frame(rnorm(5e7), rexp(5e7), runif(5e7))
xx <- cbind(xx, xx, xx, xx, xx)
system.time(lapply(x, range))
system.time(lapply1(x, range))
It saves me about 50% of the time -- that's of course only using a
relatively cheap FUN argument.
Others will hopefully comment more
M