Skip to content

Modeling correlation structure in mixed models

8 messages · Phillip Chapman, Robert A LaBudde, milton ruser +4 more

Original
Phillip Chapman
# 
Hi All,

I have been trying to learn mixed models in R by reading the books by 
Pinheiro and Bates; Faraway (both linear models books); and Crawley (R 
Book), but I would appreciate some guidance from the more experience R 
users. (I have a fair amount of experience with mixed models in SAS.)

1. Is there another (other than the above) suggested reference for 
understanding the workings of the nlme and lme4 libraries?

2. Is it the case that lme accepts correlated structures ONLY in the 
error term? I have problems in which I would like model random effects 
(such as year) using a random term with an autocorrelated structure. In 
SAS I use options to the ?repeated? statement to add correlation 
structure to the error term, and I use options to the ?random? statement 
to give correlation structure to the other random effects. I haven?t 
found anything in lme or lmer that allows me to specify correlated 
random effects. gee only allows correlation structure in the error term 
and does not allow random effects.

3. All of the examples of random effects in lme seem to have nested 
error structures. Is it the case that lme does not allow crossed random 
effects? lmer allows much more flexible specification of random effects, 
but I don?t see anything that allows correlated error structures.

Thanks in advance,

Phil Chapman
Robert A LaBudde
# 
West et al. Linear Mixed Models. Chapman & Hall.

This text uses SAS, SPSS, HLM and R to set up and solve a variety of 
hierarchical mixed models. The text uses 'nlme' exclusively, but the 
related website has 'lme4' equivalent scripts.
At 06:42 PM 6/26/2009, Phillip Chapman wrote:

            

      
      
      
    

        

      
      
    

        
================================================================
Robert A. LaBudde, PhD, PAS, Dpl. ACAFS  e-mail: ral at lcfltd.com
Least Cost Formulations, Ltd.            URL: http://lcfltd.com/
824 Timberlake Drive                     Tel: 757-467-0954
Virginia Beach, VA 23464-3239            Fax: 757-467-2947

"Vere scire est per causas scire"

  
  


  


      

    

    

      
      

        
      

    

    

      
      

        


  

        

      Kingsford Jones
    

    

      
      #
    

  


  

      
Hi Phillip,

Welcome.  Although I'm a fan of PROC MIXED, I think you'll find doing
your mixed modeling in R a relative joy.  Unfortunately, to experience
the joy one must learn to navigate the byzantine labyrinth of
documentation that has grown from this community effort.  A few leads
are offered below...


On Fri, Jun 26, 2009 at 4:42 PM, Phillip

        
Chapman<pchapman at stat.colostate.edu> wrote:

            

      
      
      
    

        
P&B is of course the authoritative reference for the nlme package, and
Doug has mentioned on this list that in his (limited) spare time he is
working on a book to accompany lme4.  The lme4 package does come with
several vignettes that can be accessed from R by a call to the
vignette function or by simply opening the pdfs in
yourRlibrary/lme4/doc/.  There is also a vignette in the SASmixed
package called 'lmer for SAS PROC MIXED Users'.  Other helpful
references can be found on the CRAN contributed documentation section,
such as the Mixed Models Web Appendix to John Fox's book.  I haven't
read Gelman and Hill's Data Analysis and Regression using
Multilevel/Hierarchical models, but as I understand it they user lmer
extensively, with wrappers for Bayesian inferences.  Also, Harald
Baayen has a freely available draft of a book on analyzing linguistic
data that includes many lmer examples:
http://www.ualberta.ca/~baayen/publications.html

Googling the following may also be useful:
lmer filetype:pdf

Here are some of Doug's documents that show up:

www.stat.wisc.edu/~bates/reports/MixedComp.pdf
user2007.org/program/presentations/bates.pdf
http://www.jstatsoft.org/v20/i02
www.stat.wisc.edu/~bates/IMPS2008/lme4D.pdf

            

      
      
      
    

        
This is something that I have wondered about as well -- as far as I
know one can only specify a correlation structure for the error
covariance matrix, and only using the nlme package (not lme4).
However, given that there are thousands of R packages available I
would not be surprised if someone's already coded up a way to do this
(perhaps in one of the spatial packages using a Bayesian approach,
such as spBayes or geoRglm?)

            

      
      
      
    

        
Although nlme is designed for nested data, crossed random effects can
be specified using a combination of pdBlocked and pdIdent objects (see
page 163 of P&B).  However it's an awkward specification and the
fitting can be slow (IIRC). On the other hand lmer offers elegant
methods of specifying crossed models and speedy methods for fitting
them.

            

      
      
      
    

        
You're welcome -- hope it helped,

Kingsford Jones

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Gabor Grothendieck
    

    

      
      #
    

  


  

      
In addition to the sources already mentioned there are chapters on
mixed models in the books associated with the mgcv and DAAG
R packages.

On Fri, Jun 26, 2009 at 10:48 PM, Kingsford

        
Jones<kingsfordjones at gmail.com> wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Gregor Gorjanc
    

    

      
      #
    

  


  

      
Hi,

Here are my 2 cents.

As far as I know, there is no way in lmer to specify anything but a simple
random effect (though there is ZStar package on R-forge that can be used to
infiltrate some nifty covar. structures as has been for example been done in
pedigreemm package). However, this does not mean that only residual has
a covariance matrix, which is actually a identity matrix times a residual
variance.

Take for example a model with three effects: a, b, and c. Both b and c are
to be modelled as 'random' effect. Then the lmer code is something like

lmer(y ~ a + (1 | b) + (1 | c))

This assumes the following model

y_ijk = a_i + b_j + c_k + e_ijk

note that a, b, and c need not be nested! The variance assumptions are

Var(y) = \sigma^2_b + \sigma^2_c + \sigma^2_e

Now for two records that come from the same b_j but not the same c_k, we have

Cov(y_1, y_2) = \sigma^2_b + \sigma^2_e

etc. However, there is no way to say that there is correlation between
different levels of b or c as can bee done in SAS (PROC MIXED) with a
variety of covariance matrices - full, ... That is at least from what I
know - Douglas can "pop in" here if I am wrong.

Additional thing that is possible in lmer is to use

lmer(y ~ a + x + (1 + x | b) + (1 | c))

which is the so called "random regression" model or "varying intercept and
varying slope" model or ... Here we have a 2x2 covariance matrix for the
effect of b.

Finally, my experience is that PROC MIXED my choke on large datasets, while
lmer handles them due to the use of sparse matrix techniques[1], though
there is some progress on this matter at SAS[2].

Regards, gg

[1]http://ggorjan.blogspot.com/2008/07/proc-mixed-vs-lmer.html
[2]http://support.sas.com/documentation/cdl/en/statug/59654/HTML/default/statug_hpmixed_sect001.htm

  
  


  


      

    

    
1 day later

    

      
      

        


  

        

      ONKELINX, Thierry
    

    

      
      #
    

  


  

      
Dear all,

A small remark on Gregor comment about the correlation structure in
lme4. It can handle two kind of correlation structures within the random
effects: a symmetric variance-covariance matrix (unstructured in SAS
terminology) and independent random effects.

lmer(y ~ a + (1 + x|b))
#symmetric variance-covariance matrix
z11 z12
Z12 z22
#equivalent to the default in nlme (pdSymm)

lmer(y ~ a + (1|b) + (x - 1|b))
#indepent random effects
z11 0
0   z22
#equivalent to pdDiag in nlme

nlme allows for several other correlation structures. Have a look at
?pdClasses. If you can't find the structure you need / like, then you
can allow code your own pdClass. But if you have a limited number of
years and a lot of data, then I would stick to an existing structure
like pdSymm. It will be less efficient than the 'true' structure in your
data, but you need much less assumptions on your model.

HTH,

Thierry


------------------------------------------------------------------------
----
ir. Thierry Onkelinx
Instituut voor natuur- en bosonderzoek / Research Institute for Nature
and Forest
Cel biometrie, methodologie en kwaliteitszorg / Section biometrics,
methodology and quality assurance
Gaverstraat 4
9500 Geraardsbergen
Belgium
tel. + 32 54/436 185
Thierry.Onkelinx at inbo.be
www.inbo.be

To call in the statistician after the experiment is done may be no more
than asking him to perform a post-mortem examination: he may be able to
say what the experiment died of.
~ Sir Ronald Aylmer Fisher

The plural of anecdote is not data.
~ Roger Brinner

The combination of some data and an aching desire for an answer does not
ensure that a reasonable answer can be extracted from a given body of
data.
~ John Tukey

-----Oorspronkelijk bericht-----
Van: r-sig-mixed-models-bounces at r-project.org
[mailto:r-sig-mixed-models-bounces at r-project.org] Namens Gregor Gorjanc
Verzonden: zaterdag 27 juni 2009 15:10
Aan: r-sig-mixed-models at r-project.org
Onderwerp: Re: [R-sig-ME] Modeling correlation structure in mixed models

Hi,

Here are my 2 cents.

As far as I know, there is no way in lmer to specify anything but a
simple random effect (though there is ZStar package on R-forge that can
be used to infiltrate some nifty covar. structures as has been for
example been done in pedigreemm package). However, this does not mean
that only residual has a covariance matrix, which is actually a identity
matrix times a residual variance.

Take for example a model with three effects: a, b, and c. Both b and c
are to be modelled as 'random' effect. Then the lmer code is something
like

lmer(y ~ a + (1 | b) + (1 | c))

This assumes the following model

y_ijk = a_i + b_j + c_k + e_ijk

note that a, b, and c need not be nested! The variance assumptions are

Var(y) = \sigma^2_b + \sigma^2_c + \sigma^2_e

Now for two records that come from the same b_j but not the same c_k, we
have

Cov(y_1, y_2) = \sigma^2_b + \sigma^2_e

etc. However, there is no way to say that there is correlation between
different levels of b or c as can bee done in SAS (PROC MIXED) with a
variety of covariance matrices - full, ... That is at least from what I
know - Douglas can "pop in" here if I am wrong.

Additional thing that is possible in lmer is to use

lmer(y ~ a + x + (1 + x | b) + (1 | c))

which is the so called "random regression" model or "varying intercept
and varying slope" model or ... Here we have a 2x2 covariance matrix for
the effect of b.

Finally, my experience is that PROC MIXED my choke on large datasets,
while lmer handles them due to the use of sparse matrix techniques[1],
though there is some progress on this matter at SAS[2].

Regards, gg

[1]http://ggorjan.blogspot.com/2008/07/proc-mixed-vs-lmer.html
[2]http://support.sas.com/documentation/cdl/en/statug/59654/HTML/default
/statug_hpmixed_sect001.htm

_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

Dit bericht en eventuele bijlagen geven enkel de visie van de schrijver weer 
en binden het INBO onder geen enkel beding, zolang dit bericht niet bevestigd is
door een geldig ondertekend document. The views expressed in  this message 
and any annex are purely those of the writer and may not be regarded as stating 
an official position of INBO, as long as the message is not confirmed by a duly 
signed document.

  
  


  


      

    

    

      
      

        


  

        

      Phillip Chapman
    

    

      
      #
    

  


  

      
Dear r-sig-mixed models board participants:

Rather than clutter the board with individual responses, I will just 
send one note thanking all those who made helpful responses to my 
inquiry about understanding mixed models in nlme and lme4.  Your info 
has been a great help.

1. Robert LaBudde:  I have the West, et al. Linear Mixed Models book, 
but wasn't aware that there was lme4 script on its web site.

2.  Kingsford Jones:  It is great news that Doug Bates is working on an 
companion book for lme4.  Perhaps some advance material will be 
available when it is closer to completion.  I will keep an eye on his 
web site for additional references.  Thanks for the other references and 
for calling my attention to the example on page 164.  In an agricultural 
context it would be called a "split block design."

3. Milton RUser:  I will buy a copy of the "Mixed Effects Models and 
Extensions in Ecology with R".  I had seen it on Amazon was close to 
buying it.  I think I will go ahead.

4. Gabor Grothendieck:  I'll look at the mgcv and DAAG packages.

5.  Alain Zuur;  I'll look forward to your sequel.  Ntzoufras looks 
useful, I'll take a closer look.

6.  Theirry Onkelinx:  I was aware that by listing both effects in the 
same specification that I could get an unstructured covariance.  The 
application that I am currently looking has 30 year, for which I would 
like to fit AR1 structure.

Thanks again,
Phil Chapman