Multiple comparison correction?
Hello, I have a question related to mixed effect modeling and how to do multiple comparisons. We have a longitudinal study with different groups and many dependent variables such as of brain cortical volume in different areas, etc. I am using lme, and remember reading somewhere that multiple comparison corrections do not actually apply to linear mixed effects models, due to the statistics involved.
YIKES! I hope that advice isn't in print, like in a textbook! There are times when you should use corrections, and times when it is acceptable not to. But that doesn't depend on the modeling method, it depends on the character of the inferences needed.
For example, if I run the same model on 100 dependent variables, traditionally I would need to correct for multiple comparisons by dividing the alpha level (0.05) by 100 to get the proper criterion of 0.0005, adjusting for the increased likelihood of getting type I errors. I am wondering however, if this process is the same, or even necessary at all for lme models?
What's true is that the usual Tukey critical values and such become unreliable when you have vastly different standard errors. But the multcomp package (see especially the glht function) is built to handle that, and it can support lme objects. When several factors are involved, it may be easier to construct them using the lsmeans function (lsmeans package), then use as.glht and you can summarize them like you would the glht results. Russell V. Lenth - Professor Emeritus Department of Statistics and Actuarial Science The University of Iowa - Iowa City, IA 52242 USA Voice (319)335-0712 (Dept. office) - FAX (319)335-3017