Skip to content

ICC confidence intervals and power analysis for random effects in lmer?

4 messages · Bradley Carlson, Mollie Brooks, Bob OHara +1 more

Original
Mollie Brooks
# 
Hi Brad,
I'm also interested in getting reliable CI on random effects because I want to know how the within treatment variability differs among treatments. The only ways I know to get CI on random effects are to either do likelihood profiling, or (preferably) MCMC sampling. If your desired distribution and link function are built into either glmmADMB or MCMCglmm, then go for those. I have been using ADMB via R2admb recently to do my own profiling and MCMC sampling of semi-non-standard GLMMs. I could try to be more specific and helpful if you have a more specific model formulation in mind, but other people might have better ideas, especially if you wrote to the mixed models list.
best,
Mollie

Mollie Brooks
Ph.D. Candidate
NSF IGERT Fellow
Biology Department
University of Florida
mbrooks at ufl.edu
http://people.biology.ufl.edu/mbrooks
On 11 Apr 2012, at 9:50 PM, Bradley Carlson wrote:

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Bob OHara
    

    

      
      #
    

  


  

      
On 04/12/2012 03:50 AM, Bradley Carlson wrote:

            

      
      
      
    

        
MCMC has already been mentioned and lme4 still has its mcmcsamp() 
function. Failing that, you could try a parametric bootstrap, which 
requires a little bit of coding but simulate() makes it much easier.

            

      
      
      
    

        
Report the random effect and confidence intervals. Retrospective power 
analyses are pretty pointless (e.g. see 
http://beheco.oxfordjournals.org/content/14/3/446.full), unless you're 
planning to repeat the experiment.

Bob

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Tom A. Langen - tlangen
    

    

      
      #
    

  


  

          

      
      
      
    

        
The reviewer was requesting a post-hoc power analysis,  either an 'observed power' or 'detectable effect size' analysis. Hoenig and Heisey (2001) provide the definitive paper showing that the use of post hoc power analysis is fallacious, and that confidence intervals provide a more informative way to evaluate inferences about not rejecting the null statistical hypotheses. There are other good recent papers making the same point, but Hoenig and Heisey (2001) is definitive in my opinion. 

See: Hoenig, J. M., and D. M. Heisey. 2001. The abuse of power: The pervasive fallacy of power calculations for data analysis. Am. Stat. 55: 1-6.

Note that this only applies to post hoc power analysis. Use of power analysis as an aid to planning experiments is an essential tool for experimental design.

Tom Langen
?
Associate Professor 
Departments?of?Biology?&?Psychology 
Clarkson?University 

Box?5805,?Clarkson?U.,?Potsdam?NY?13699-5805 
Phone:?315?268?7933,?Fax:?315?268?7118 
www.clarkson.edu/~tlangen?? 


-----Original Message-----
From: r-sig-ecology-bounces at r-project.org [mailto:r-sig-ecology-bounces at r-project.org] On Behalf Of Bradley Carlson
Sent: Wednesday, April 11, 2012 9:51 PM
To: r-sig-ecology
Subject: [R-sig-eco] ICC confidence intervals and power analysis for random effects in lmer?

I'm performing an analysis of behavioral variation among individual
tadpoles, using individual ID as a random effect and time as a continuous
fixed covariate in the lmer function in lmer4 package. I'm really
interested in making inferences about the random effect (i.e. the extent of
variation among individuals). I'd like to do two things that I can't seem
to find straightforward answers about and I'm hoping someone can help or
point me to a good resource.

1) The intraclass correlation coefficient is of particular interest to me,
as it describes the proportion of variation that occurs among individuals.
Ideally I'd like to report a confidence interval of the ICC but I can't
find any way to calculate one, other than a function in the psychometric
package that appears to only work when there are no covariates in the model
(random effect only).

2) A reviewer requested a power analysis of the ability to detect a
significant random effect. Any tips on how to approach that?

Thanks for any help,

Brad