Skip to content

How to extract x rows to get x pvalues using t.test

2 messages · Liaw, Andy, Thomas Lumley

Original
Liaw, Andy
# 
That's not true.  In R, there's a for loop hidden inside apply() (just look
at the source).  In S-PLUS, C level looping is done in some situations, and
for others lapply() is used.

Andy
Thomas Lumley
#
On Tue, 15 Mar 2005, Liaw, Andy wrote:

            

      
      
      
    

        
It's slightly more complicated than this.  lapply() really is a C-level 
loop and apply() eventually calls it.

Now, whatever happends inside apply(), it still true that t.test() has to 
be called 56,000 times, providing a lower bound on the time apply() can 
take. In this case I would be very surprised if apply() saved any time. 
What would save time is writing a stripped-down t-test function, 
especially as only the p-value is being used.

The real problem with apply is that when the objects involved are large, 
apply() can be substantially slower because of greater memory use.  As a 
concrete example, an apply() on a 10000x757 set of replicate weights in 
the survey package used half as much memory when turned into a for() loop. 
As a result it ran several times faster on my laptop (where it was paging 
heavily) and slightly faster on my desktop (which has rather more memory).


 	-thomas