Skip to content

aov() and anova() making faulty F-tests

4 messages · Rolf Turner, Peter Dalgaard, PatGauthier

Original
PatGauthier
# 
Dear useRs, 

I've just encountered a serious problem involving the F-test being carried
out in aov() and anova(). In the provided example, aov() is not making the
correct F-test for an hypothesis involving the expected mean square (EMS) of
a factor divided by the EMS of another factor (i.e., instead of the error
EMS). 

Here is the example:


                  Expected Mean Square            df
Mi                     ?2+18?2M                              1
Ij                  ?2+6?2MI+12?(I)                      2
MIij                   ?2+6?2MI                              2
?(ijk)l                        ?2                                   30

The clear test for Ij is EMS(I) / EMS(MI) -  F(2,2)

However, observe the following example carried out in R, 

M <- rep(c("M1", "M2"), each = 18)
I <- as.ordered(rep(rep(c(5,10,15), each = 6), 2))
y <-
c(44,39,48,40,43,41,27,20,25,21,28,22,35,30,29,34,31,38,12,7,6,11,7,12,15,10,12,17,11,13,22,15,27,22,21,19)
dat <- data.frame(M, I, y)
summary(aov(y~M*I, data = dat))
                       Df    Sum Sq       Mean Sq         F value       
Pr(>F)    
m                     1     3136.0           3136.0            295.85      <
2e-16 ***
i                        2      513.7              256.9              24.23     
5.45e-07 ***
m:i                   2      969.5              484.7              45.73     
7.77e-10 ***
Residuals   30       318.0                10.6                     
---

In this example aov has taken the F-ratio of MS(I) / MS(?) -  F(2,30) =
24.23 with F-crit = qf(0.95,2,3) = 9.55 -- significant

However, as stated above,  the correct F-ratio is MS(I) / MS(MI) -  F(2,2) =
0.53 with F-crit = qf(0.95,2,2) = 19 -- non-significant

Why is aov() miscalculating the F-ratio, and is there a way to fix this
without prior knowledge of the appropriate test (e.g., EMS(I)/EMS(MI)?

Thanks for your help, 

Patrick





--
View this message in context: http://r.789695.n4.nabble.com/aov-and-anova-making-faulty-F-tests-tp4660407.html
Sent from the R help mailing list archive at Nabble.com.
Rolf Turner
# 
Your subject line is patent nonsense.  The aov() and anova() functions
have been around for decades.  If they were doing something wrong
it would have been noticed long since.

You should realize that the fault is in your understanding, not in these
functions.

I cannot really follow your convoluted and messy code, but it would
appear that you want to consider "M" and "I" to be random effects.

Where have you informed aov() as to the presence of these
random effects?

     cheers,

         Rolf Turner
On 03/06/2013 03:36 PM, PatGauthier wrote:
Peter Dalgaard
#
On Mar 6, 2013, at 03:56 , Rolf Turner wrote:

            

      
      
      
    

        
Only M and M:I, AFAICT.  And, yes, it is messy; in particular, I refuse to believe that y~M*I has generated output with lowercase m and i!

            

      
      
      
    

        
To be specific, try y~I + Error(M + M:I). Without the random effects, aov() is just telling you that there is a highly significant interaction between M and I, and beyond that, no sensible comparisons can be made.