Running slow

Patrick is right, most of the time is probably taken up for the
reasons documented in the (masterful) R Inferno, namely the rbind()
calls.

There is another problem though and it gets at the very core of R, and
for that matter, all interpreted languages that I'm familiar with.
I'll give a fairly elementary explanation and gloss over many of the
subtleties that R core worries about so we mere mortals don't have to.

At the end of the day, everything is looped, there's no way to get
around it. However, from a code perspective we have a choice of
looping in C or R. Whenever possible it is better to loop in C than R
and most of the key built-in functions, like unique(), are designed to
do just that. The reason for it is pretty straightforward: consider
what has to happen to run a loop in R:

Iterator is defined: a sequence of C calls start this
first line of loop is hit -> interpreted by R -> sent to C code ->
executed -> changed back into an R result -> passed to the next line
of the loop
iterator is increased: C again
second line of loop is hit -> interpreted by R -> sent to C code ->
executed -> changed back into an R result -> passed to the next line
of the loop
etc.

Complicated and/or multiple lines of code only compound the problem
because you have to go up and down multiple times at each iteration.

Looping on the C level gets rid of all those "translations" between
C/R, save 2, and thereby mightily increases efficiency. Hence, even if
you are using the same (or heaven forbid a faster!) algorithm on the R
level, it can look super slow because of all the moving up and down
the ladder; I don't know how unique.C is implemented, but my guess is
it's more or less like what you have now, with more efficient memory
usage/preallocation, it just looks *much* faster because of the C
architecture.

DISCLAIMER: there are quite a few inaccuracies, most small, maybe a
few large, in here, and I probably only am aware of a small fraction
thereof, but this wasn't intended to be a super accurate explanation.

On another note, I should explain my solution a little more clearly.

A straight call to unique() would check for unique ROWS not values of
x. I take x, make a copy so as not to harm the original object, strip
if of its dimensionality (thereby converting it to a vector
efficiently), and then apply unique() which will now find unique
values. It's not a huge thing, but not immediately apparent from what
I did.

Hope this helps,

Michael

Running *slow*

Thread (8 messages)

Running slow