Use LRT to assess the effect of lower order terms in the presence of a significant higher order term (e.g., interaction)
On Sat, 19 May 2012, Xiao He wrote:
I have a couple of questions regrading how to assess the effects of lower order terms when a higher order term is significant using likelihood ratio tests. I am interested to examine how two independent variables (iv1 and iv2) affect subjects' log-transformed reaction time (logRT). In a series of responses that Dr. Bates made in this thread: https://stat.ethz.ch/pipermail/r-sig-mixed-models/2011q1/005399.html, he recommended that non-significant fixed effects be removed from a model before further llkelihood ratio tests.
[snip]
#Models: #model.main: logRT ~ iv1 + iv2 + (1 + iv2 | subject) + (1 | item) #model.inter: logRT ~ iv1 * iv2 + (1 + iv2 | subject) + (1 | item) # Df AIC BIC logLik Chisq Chi Df Pr(>Chisq) #model.main 8 236.31 271.43 -110.15 #model.inter 9 229.10 268.61 -105.55 9.2097 1 0.002407 ** (1). My first question is as follows: For example, in the case above, the interaction is shown as significant. I know that when an interaction is present, main effects are often not interpretable, so probably that makes it unnecessary to even assess main effects? But say, if for whatever reason, I need to assess lower order terms - for example iv2, should I compare the first pair of models below, or the second pair of models below? It seems to me that comparing the 2nd pair makes more sense because the significant interaction probably shouldn't be removed from the model. First pair: model.main: logRT ~ iv1 + iv2 + (1 + iv2 | subject) + (1 | item) model.iv1: logRT ~ iv1 + (1 + iv2 | subject) + (1 | item) Second pair model.main: logRT ~ iv1 + iv2 + iv1:iv2 (1 + iv2 | subject) + (1 | item) model: logRT ~ iv1 + iv1:iv2 + (1 + iv2 | subject) + (1 | item)
Did you actually try the second one?
(2). My second question has to do with random effect specification. As you can see, in all the models I showed above, I have iv2 as the random slope - which I decided based upon comparing models that have the same fixed effects ( iv1 * iv2) but have different nested random effect specifications. While including iv2 makes sense for models that have iv2 as a fixed effect (e.g., model.main, model.inter), its inclusion doesn't seem to make sense in models where iv2 is not included as a fixed effect - for example, the null model and model.iv1. However, it seems necessary to include it when I compare the models below. I wonder if someone can tell me what is the proper way of dealing with random effects in this kind of scenario and the reasons behind. model.main: logRT ~ iv1 + iv2 + (1 + iv2 | subject) + (1 | item) model.iv1: logRT ~ iv1 + (1 + iv2 | subject) + (1 | item)
I think all this requires you to consult locally about your exact problem domain. Would you expact iv1 and iv2 to interact? Would you expect a subject specific slope on iv2? If iv2, why not iv1? If you choose a different transformation of RT, some of these complications might disappear, so that the interpretation will be cleaner. If iv2 is a well known important covariate of RT from the literature *or* subject specific slopes are needed, iv2 should be in as a main effect. The next questions are whether you also need iv1 in the random effects part of the model (given you find an iv1:iv2 interaction), and whether you have enough data to support estimating all these effects. Cheers, David Duffy,