Skip to content

The effects of adding by-subject or by-item random intercepts

5 messages · Antoine Tremblay, John Maindonald, Daniel Ezra Johnson

Original
Antoine Tremblay
# 
Dear all,

This question is about the effects of adding by-subject or by-item
random intercepts to a model.

If we are contrasting a single condition between two subject groups,
say ReactionTime ~ Sex,
is it warranted (or necessary or ill-advised) to include by-subjects
random intercepts,
since this could (if I'm understanding it correctly) adjust the mean
reaction time for each subject (and thus for
each condition) towards the grand mean, thus reducing or
eliminating the difference in the condition between subjects? And
similarly if we are contrasting a
single condition between two sets of items, say ReactionTime ~ Frequency?

I believe that the addition of the random effect may reduce the effect
of the fixed effect, but should
not remove it entirely. Is this right?

The question would then become: Why would the addition of say by-item
random intercepts to a model
take away an effect that was present in a model without by-item random
intercepts?

Thank you again, your help is well appreciated.
John Maindonald
# 
Think of the intercept (probably at the mean of x)  and slope 
of the fitted lines as statistics that you want to compare between 
groups.  You might use a t-test the test for no significant 
difference in intercept between the two groups. Similarly
for the slope, if you look at that on its own.

If the intercepts do differ between subjects, then the difference
in intercepts must, to be real, be greater than can be explained
by within group variation in intercepts.  Similarly for the slopes.

John Maindonald             email: john.maindonald at anu.edu.au
phone : +61 2 (6125)3473    fax  : +61 2(6125)5549
Centre for Mathematics & Its Applications, Room 1194,
John Dedman Mathematical Sciences Building (Building 27)
Australian National University, Canberra ACT 0200.
http://www.maths.anu.edu.au/~johnm
On 08/12/2009, at 6:15 PM, Antoine Tremblay wrote:

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Daniel Ezra Johnson
    

    

      
      #
    

  


  

      
Shrinkage is not the main issue, as I see it here. When the predictor  
of interest is Sex you should include by-subject random effect(s),  
when it's Frequency you should include by-item. Probably you should  
include both in both cases. You can't do accurate hypothesis testing  
on Sex and Frequency if you ignore the variation among Subjects and  
Items.

        
On Dec 8, 2009, at 2:15 AM, Antoine Tremblay <trea26 at gmail.com> wrote:

        

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Antoine Tremblay
    

    

      
      #
    

  


  

      
Thank you for your replies.
:-)

Let's see if I understood well. Say I test for a frequency effect
(high vs low) ReactionTime ~ Frequency without the by-items random
intercepts and find a difference.

Now in a model where I do include by-items random intercepts
ReactionTime ~ Frequency + (1|Item) the frequency effect disappears.

Then the frequency effect found in the model without by-item random
intercepts was spurious, i.e., was due only to within group
variability and not to a true population effect. Is that right?

Now say I have created artificial items, which I used in my
experiment. I thus have all the whole population of items. Should I
still include Items as a random effect? If it is the case, then,
including or not a random effect is not only a matter of wanting to
generalize over subjects or items, but rather a matter of getting rid
of, so to speak, within-group variability, which, if uncontrolled for,
may lead to spurious effects. Is that right?

Thank you again for your help.
Sincerely,

Antoine

On Tue, Dec 8, 2009 at 7:38 AM, Daniel Ezra Johnson

        
<danielezrajohnson at gmail.com> wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Daniel Ezra Johnson
    

    

      
      #
    

  


  

      
The choice is in theory between treating Item as a random effect or a  
fixed effect. Not in my view omitting it totally.

In practice you have to use a random effect or a singularity (non- 
estimable model) results.

Dan

        
Tremblay <trea26 at gmail.com> wrote: