GLMM, LRTs and post-hocs
1) The LRT is a more reliable guide than what I assume are the modified Wald statistics that you are using for the post-hoc comparisons. The Wald statistics involve approximations that can play havoc with the distributional assumptions that are commonly use. 2) Even if both tests are pretty much beyond reproach, as in some normal theory contexts, different testing procedures may give different answers. The LRT is an overall test, automatically taking a/c of multiplicity. If you believe in adjustments for multiplicity, or (a better reason, I consider) think that such an adjustment accords with the aims of the project, you might go with the LRT. ----------------------------------------------------------------------------------------------- Now for what may seem something of a hobby-horse. I wish we could move away from this heavy reliance on p-values. Often, it will do much better justice to the data to say: a) These (. . . .) are the effects stand out clearly, pretty much irrespective of the twists and turns of the way that the results might be interpreted . . . b) Next note effects that are borderline . . . Remember that there can be model uncertainty as well as the statistical uncertainty that a particular form of analysis identifies. With GLMMs that have a fixed scale factor, there is the issue of whether observation level random effects are required to adequately account for the variation. If there are such effects, then unless they are very small and/or there are many observations, think carefully before taking too much notice of p-values that may be given for the Wald statistics. c) Whichever way one shakes it, remaining effect estimates are consistent with statistical variation. This way of presenting results would, often, give a clearer and more honest account of the data and of model uncertainty. John Maindonald email: john.maindonald at anu.edu.au phone : +61 2 (6125)3473 fax : +61 2(6125)5549 Centre for Mathematics & Its Applications, Room 1194, John Dedman Mathematical Sciences Building (Building 27) Australian National University, Canberra ACT 0200. http://www.maths.anu.edu.au/~johnm
On 26/08/2010, at 2:53 AM, Kay Cecil Cichini wrote:
hello everyone, i did a GLMM with two fixed crossed (2 levels each) and one random factor. modelselection by LRTs gives no significance for the one fixed factor i'm mainly interested in, neither for the interaction with the second factor. however when i do post-hoc tests on the first factor within each level of the second i get significant effects at one of the two levels of the second factor, saying factor one is significant but only within level X of the factor two. now i'm clueless how to proceed with this result, because LRTs tell me to dismiss the factor of interest but by pos-hocs i know in fact the one factor has an effect, however restricted on one level of the second factor... any comments on this would be greatly appreciated, kay
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models