mixed model testing

Dear John,

Thank you for your reply. Well then I agree, a random effect should be added if one suspects that it should be there. Neat example with pirates!

Pernicious is a too strong word, but dangerous. Since, as I said, many of the results found in the literature are based on screening for covariates of interest, including only those that have a p-value less than 0.30 (say) in a forward selection model. Using simple linear, or at best, (fractional) polynomial bases to represent covariates, and almost always ignoring interactions (difficult to present). I suppose that it would be OK to be inspired by them, but with a good amount of distrust (if Harrell is right, which seems to be the case). This distrust could perhaps be lessened when the literature concerns randomised trials although covariates may find their way into this area to (and still no interactions...).

Best regards,

Fredrik

-----Ursprungligt meddelande-----
Fr?n: John Maindonald [mailto:John.Maindonald at anu.edu.au] 
Skickat: den 8 november 2007 12:17
Till: Nilsson Fredrik X
Kopia: r-sig-mixed-models at r-project.org
?mne: Re: SV: [R-sig-ME] mixed model testing

For simplicity, I limited attention to a rather small class
of models.  I assumed that the only fixed effect that the
data would support is a linear term, and I do not mind
adding an intercept.  That is realistic, I believe, for data
of this type.

One should not limit oneself to a single random effect
if indeed the data are sampled in a manner (e.g., lawns
within soil types) that makes it natural to expect some
further random effect.  In my example as stated, the data
have no such structure. (NB, the random sample comment,
meaning simple random sample).

I agree that I have used Box's example outside of the
context in which he used it.  A better example might be
use of a reconnaissance that has only a 20% chance
of detecting such pirates as may be present on the high
seas, before deciding whether a valuable cargo that will
venture into those seas should have an escort.

Removing insignificant terms can help understanding
and interpretation.  But if one wants to make anything
of the coefficients, it is necessary to check that the
remaining coefficients have not changed substantially.
If there are more than a few terms to consider, and data
are not from a designed experiment, and attempt to
interpret coefficients is likely to be hazardous. You
mentioned the Harrell book. Rosenbaum's "Observational
Studies" (2edn, Springer, 2002) merits careful attention.

Why do you think Harrell's suggestion pernicious?  Models
that are in the literature can be a good starting point, and
the accompanying discussion an aid to understanding
the science. They may turn out to be more or less right
(as far as one can tell), or to require modification, or the
data may demolish them.  But at least one has a starting
point, rather than an almost unlimited choice of models.

John Maindonald             email: john.maindonald at anu.edu.au
phone : +61 2 (6125)3473    fax  : +61 2(6125)5549
Centre for Mathematics & Its Applications, Room 1194,
John Dedman Mathematical Sciences Building (Building 27)
Australian National University, Canberra ACT 0200.

mixed model testing

Thread (5 messages)