Interpretation of hypothesis tests for mixed models

Thanks for setting me straight about the model

fm1 <- lme(y ~ Trt, random = list(Subj = pdCompSymm(~ Trt - 1)))

being the one that is equivalent to

fm2 <- lme(y ~ Trt, random = ~ 1 | Subj/Trt)

It seems that denDF of a fixed effect test for treatment should also be
the same for fm1 and fm2. Is it possible to modify the method of
computing denDF in nlme to achive this? 

Meanwhile, my understanding is that fm2 is to be preferred over fm1. I
did simulations for fm1/fm2 with no true (fixed) difference between
treatments, which seemed to show that a test with the fm1 formulation
can sometimes produce considerably more statistical significances than
would be warranted. 

I then have another question. How should I go about formulating a model
corresponding to the nesting in fm2 if instead of a treatment factor I
have a covariate? Since in my example Trt was a two-level factor, one
could for instance let the levels be zero and one and regard the
treatment as a covariate. If I express the treatment as a covariate x
and fit

fm4 <- lme(y ~ x, random = ~ 1 | Subj/x)

I get the same denDF as for fm2, but for a general covariate (with more
than two values) denDF depends on the number of distinct values taken by
the covariate (but it should not, should it?). It seems that random = ~
1 | Subj/x treats x as a a factor. Is there another model formulation
that takes care of this problem? 

More generally, if I have complex terms, like a treatment by covariate
interaction, for which I suspect random subject components, how can I
formulate a mixed model so that denDF properly takes into account the
nested random effects?

Olof Leimar, Professor
Department of Zoology
Stockholm University
SE-106 91 Stockholm
Sweden

Olof.Leimar at zoologi.su.se