Testing Random Effects--On the Boundary

Hi all,

I've read multiple times that using LRT to test the significance of random
effects terms in mixed models yields conservative p-values and that one way
to correct this is to divide the p value in half.  Is this a hard fast rule
or is there a script for R that gives an actual corrected value?

Thank you,
Jacob

It's not hard and fast.  Fabian Scheipl's RLRsim package gives a fast
stochastic algorithm for getting the correct null distribution in these
cases, but it only works for a subset of models.  I *believe* (but may
misremember) that for the simple case of a single, scalar random effect
(i.e. a single blocking factor with an intercept effect only, ~ ... +
(1|block)) that the null distribution is provably 0.5*chi^2(0) +
0.5*chi^2(1) (i.e., you should divide by 2), but (1) this might only
hold for LMMs (not GLMMs) and (2) it might only hold asymptotically and
(3) it definitely doesn't hold for more complex random-effects models.
Pinheiro and Bates 2000 discuss this (as referenced in
http://glmm.wikidot.com/faq#random-sig ; I believe the ur-reference is
Stram and Lee (1994), it's also discussed briefly in Bolker (2008) p 250:

http://ms.mcmaster.ca/~bolker/misc/Bolker_2008_p250.pdf

P&B incorporated simulation machinery in nlme (?simulate.lme -- note
that simulate.lme *predates* the more general simulate() accessor in
base R, and works differently); this sort of functionality can be
replicated pretty easily with lme4, but it will be slow.


Stram, Daniel O, and Jae Won Lee. 1994. ?Variance Components Testing in
the Longitudinal Fixed E?ects Model.? Biometrics 50 (4): 1171?1177.
http://links.jstor.org/sici?sici=0006-341X%28199412%2950%3A4%3C1171%3AVCTITL%3E2.0.CO%3B2-H.

I believe Ben is referring to Self and Liang (1987) results. Alternatively, there is a score-type test (Biometrika, 2003, 90, pp 73-84) which performs quite well in LMMs. I applied it to semiparametric models (Statistics in Medicine, 2008, 27, pp 2902-2921). Let me know if that is something you want to try out. I have the code but will have to dig it out from an untidy collection of functions.

best wishes

Marco

-----Original Message-----
From: r-sig-mixed-models-bounces at r-project.org [mailto:r-sig-mixed-models-bounces at r-project.org] On Behalf Of Ben Bolker
Sent: 21 November 2013 20:11
To: AvianResearchDivision; r-sig-mixed-models at r-project.org
Subject: Re: [R-sig-ME] Testing Random Effects--On the Boundary

Hi all,

I've read multiple times that using LRT to test the significance of 
random effects terms in mixed models yields conservative p-values and 
that one way to correct this is to divide the p value in half.  Is 
this a hard fast rule or is there a script for R that gives an actual corrected value?

Thank you,
Jacob

It's not hard and fast.  Fabian Scheipl's RLRsim package gives a fast stochastic algorithm for getting the correct null distribution in these cases, but it only works for a subset of models.  I *believe* (but may
misremember) that for the simple case of a single, scalar random effect (i.e. a single blocking factor with an intercept effect only, ~ ... +
(1|block)) that the null distribution is provably 0.5*chi^2(0) +
0.5*chi^2(1) (i.e., you should divide by 2), but (1) this might only hold for LMMs (not GLMMs) and (2) it might only hold asymptotically and
(3) it definitely doesn't hold for more complex random-effects models.
Pinheiro and Bates 2000 discuss this (as referenced in http://glmm.wikidot.com/faq#random-sig ; I believe the ur-reference is Stram and Lee (1994), it's also discussed briefly in Bolker (2008) p 250:

http://ms.mcmaster.ca/~bolker/misc/Bolker_2008_p250.pdf

P&B incorporated simulation machinery in nlme (?simulate.lme -- note that simulate.lme *predates* the more general simulate() accessor in base R, and works differently); this sort of functionality can be replicated pretty easily with lme4, but it will be slow.


Stram, Daniel O, and Jae Won Lee. 1994. ?Variance Components Testing in the Longitudinal Fixed E?ects Model.? Biometrics 50 (4): 1171?1177.
http://links.jstor.org/sici?sici=0006-341X%28199412%2950%3A4%3C1171%3AVCTITL%3E2.0.CO%3B2-H.

_______________________________________________
R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models