Skip to content
Prev 310 / 885 Next

prop.test in R

Exactly - elementary texts and methods books recommend the welch test  
for the reason you mention.  Curiously, those same texts recommend  
using anova and regression without automatically correcting for the  
possibility of non-constant variance.  Why is the case of comparing  
two means different from 3?  Those same books will tell you that anova  
is pretty robust to non-constant variance.  well, the two sample  
t-test is anova.

I don't use the welch test except as a conscious decision: ie I really  
want to compare the means while suspecting that the variances differ.   
Generally people are using the t test to certify that two populations  
are different.  If the variances are wildly different, that may be  
much more important than a difference in means.  in fact, to test for  
a difference in means when the variances are wildly different is  
almost always substantively silly.   There was a great example a few  
years ago from a psychiatric journal, comparing two medications, where  
the investigators did a t-test for the means when one distribution was  
unimodal and the other was bi-modal; there was no statistically  
significant difference in the means, but there was a really important  
difference in the distributions.  The automatic use of the welch test  
makes you feel that you are protected against Bad Things, when you  
aren't.

albyn

Quoting Ian Fellows <ian.fellows at stat.ucla.edu>: