Skip to content

implementing Grubbs outlier test on a large dataframe

3 messages · John Malone, David Winsemius, Frank E Harrell Jr

#
Sending each row of a datatframe, dfm,  as a vector to a function,  
fcn, is as simple as;

apply(dfm, 1, fcn)

e.g.:

 > dfm <- data.frame(x=rnorm(10), y=rnorm(10), z=rnorm(10))
 >
 > apply(dfm, 1, sum)
  [1]  0.7385838 -3.1819193  0.3415670 -0.6552601 -1.3470174  
-0.6446259 -0.6544967
  [8]  0.1778169 -0.3330527  0.6246071

And with the second argument set to 2, you would get a columnwise  
application of the function.

You need to show us what your function looks like to go any further. I  
am unclear how one could get a function that only operates on a single  
row to yield an outlier classification.
#
John Malone wrote:
John - you would be making a strong normality assumption.  You might 
reject H0 using Grubbs' test just because of non-normality, or you might 
fail to reject it just because of non-normality.  Is it really this 
straitforward to declare something an outlier?  What does outlier really 
mean?

The following is must reading.

@Article{fin06cal,
   author =               {Finney, David J.},
   title =                {Calibration guidelines challenge outlier 
practices},
   journal =      The American Statistician,
   year =                 2006,
   volume =               60,
   pages =                {309-313},
   annote =               {anticoagulant
therapy;bias;causation;ethics;objectivity;outliers;guidelines for
treatment of outliers;overview of types of outliers;letter to the editor 
and reply 61:187 May 2007}