Wald F tests
I think it is going too far to say that one should not be testing hypotheses (the implication of that "is soooo 20th C"?). But the place of that activity is much more limited than is commonly recognized. Basically, I do not like the range of options that this (half-serious?) survey has on offer, and I'd need to write half a page or more to explain why. Democracy maybe, but (as I suppose is always the case in the political democracies that are on offer) the choices are severely constrained. Where such a hypothesis testing perspective may be appropriate, the preferred starting point is almost always a confidence interval. Why not ask the comparable questions arise for estimation? There's an editorial in Volume 72(5) (pp.1057-1058) of the Journal of Wildlife Management with which I pretty much agree: "... understand that the average reader of the Journal is interested in the biological questions addressed with your work. The analytical framework and resulting results should support those questions and flow from them, not overwhelm them." But I guess that Ben would like us to assume that the proper support framework is in place! Note also, on pages 1272-1278 of the same issue: "Suggestions for Basic Graph Use When Reporting Wildlife Research Results", by Brett Collier. John Maindonald email: john.maindonald at anu.edu.au phone : +61 2 (6125)3473 fax : +61 2(6125)5549 Centre for Mathematics & Its Applications, Room 1194, John Dedman Mathematical Sciences Building (Building 27) Australian National University, Canberra ACT 0200.
On 11/10/2008, at 6:47 AM, Ben Bolker wrote:
Does anyone out there have opinions on this subject? How should one test hypotheses about fixed effects in (G)LMMs, especially for small to moderate sample sizes? (Please ignore issues of _estimation_ (PQL vs Laplace vs AGQ vs ...) Should it amuse you to do so, you can vote at: http://www.surveymonkey.com/s.aspx?sm=yLyfrV_2ftw6WGx2dEFLWnIw_3d_3d (since we all know that scientific questions are settled by a democratic process) a hypothesis testing is soooo 20th century, don't bother b likelihood ratio tests [ignore known anticonservatism] c F tests (LMM) or Wald tests (GLMM) [ignore mismatch with hypothesized null distributions] d bootstrapped confidence intervals e [mcmcsamp confidence intervals -- if available] f randomization/simulation tests of nested null hypotheses g AIC comparisons [ignore that prediction != hypothesis testing] Note that Wald Z tests [option c] are more or less what you're doing, implicitly, if you just eyeball the estimated parameter values and their standard errors. cheers Ben Bolker -------- Original Message -------- Subject: [Fwd: Re: [R-sig-ME] Wald F tests] Date: Tue, 07 Oct 2008 17:51:01 -0400 From: Ben Bolker <bolker at ufl.edu> To: R Mixed Models <r-sig-mixed-models at r-project.org> But ... LRTs are non-recommended (anticonservative) for comparing fixed effects of LMMs hence (presumably) for GLMMs, unless sample size (# blocks/"residual" total sample size) is large, no? I just got through telling readers of a forthcoming TREE (Trends in Ecology and Evolution) article that they should use Wald Z, chi^2, t, or F (depending on whether testing a single or multiple parameters, and whether there is overdispersion or not), in preference to LRTs, for testing fixed effects ... ? Or do you consider LRT better than Wald in this case (in which case as far as we know _nothing_ works very well for GLMMs, and I might just start to cry ...) Or perhaps I have to get busy running some simulations ... Where would _you_ go to find advice on inference (as opposed to estimation) on estimated GLMM parameters? cheers Ben Bolker Douglas Bates wrote:
If I were using glmer to fit a generalized linear mixed model I would use likelihood ratio tests rather than Wald tests. That is, I would fit a model including a particular term then fit it again without that term and calculate the difference in the deviance values, comparing that to a chi-square. I'm not sure how one would do this using the results from glmmPQL. On Fri, Oct 3, 2008 at 3:37 PM, Ben Bolker <bolker at ufl.edu> wrote:
[forwarding to R-sig-mixed, where it is likely to get more
responses]
Mark Fowler wrote:
Hello,
Might anyone know how to conduct Wald-type F-tests of the fixed
effects estimated by glmmPQL? I see this implemented in SAS
(GLIMMIX),
and have seen it recommended in user group discussions, but
haven't come
across any code to accomplish it. I understand the anova function
treats
a glmmPQL fit as an lme fit, with the test assumptions based on
maximum
likelihood, which is inappropriate for PQL. I'm using S-Plus 7. I
also
have R 2.7 and S-Plus 8 if necessary.
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models