Skip to content

Collinearity diagnostics for (mixed) multinomial models

3 messages · John Fox, Juho Kristian Ruohonen

Original
John Fox
# 
Dear Phillip (and Juho),

You raise a reasonable point but, unfortunately, one that isn't really 
relevant to the problem at hand.

Applied to a linear model, which is the context in which generalized 
variance inflation was originally defined in the paper by me and Georges 
Monette cited in ?car::vif, the GVIF *is* invariant with respect to 
inessential changes to the model such as centering regressors or any 
change in the bases for the regressor subspaces of terms in the model. 
The GVIF compares the size of the joint confidence region for the set of 
coefficients for a term in the model to its size in a utopian situation 
in which the subspace for the term is orthogonal to the subspaces of the 
other terms, and reduces to the usual VIF when the term in one-dimensional.

Generalized variance inflation has subsequently been extended to some 
other regression models, such as generalized linear models, and it 
retains these essential invariances (although interpretation isn't as 
straightforward).

In response to Juho's original question, I conjectured an extension to 
multinomial logit models, tested some of its invariance properties, but 
unfortunately didn't test sufficiently extensively. (I did suggest 
additional tests that I didn't perform.) It's clear from Juho's example 
that my conjecture was wrong.

The reason that I hadn't yet responded to Juho's recent question is that 
Georges and I are still trying to understand why my proposed definition 
fails for multinomial logit models. It appears to work, for example, for 
multivariate linear models. Neither of us, at this point, has a solution 
to Juho's problem, and it's possible that there isn't one. We're 
continuing to discuss the problem, and one of us will post an update to 
the list if we come up with either a solution or a clear explanation of 
why my proposal failed.

Thank you for prompting me to reply, if only in a preliminary manner.

Best,
  John
1 day later
Juho Kristian Ruohonen
# 
Many thanks to John and Philip (and Georges behind the scenes). I'll keep
an eye on this thread. If there's a solution before my thesis goes to
print, I'll certainly adopt it posthaste.

Otherwise, I might just end up reporting all C-1 GVIF^(1/(2*DF)) statistics
for each binary subregression for each predictor, as calculated by
car::vif(). It's a bit messy, but then multinomial models themselves are
messy with their C-1 sets of coefficients, and this would be no different.
At least it's maximally transparent.

That said, I hope our experts reach a breakthrough.

Best,

Juho





ke 1. helmik. 2023 klo 18.19 John Fox (jfox at mcmaster.ca) kirjoitti:

  
  


  


      

    

    

      
      

        


  

        

      John Fox
    

    

      
      #
    

  


  

      
Dear Juho,

        
On 2023-02-02 4:08 p.m., Juho Kristian Ruohonen wrote:

            

      
      
      
    

        
Georges made a similar suggestion in our discussions, but there's a 
problem: the multinomial logit model logits are for each other level of 
the response (say B, C, D) vs the baseline level (A), while the 
individual binary logits would be for, say, A vs. {B, C, D}, B vs. {A, 
C, D}, etc. One could fit binary logit models to subsets of the data, B 
vs. A for those for whom the response is A or B, etc., but I believe the 
coefficients would differ from those for the multinomial logit model.

            

      
      
      
    

        
You're more optimistic than I am. Georges has a good explanation for why 
the GVIF works with the multivariate linear model, but the necessary 
properties aren't shared by the multinomial logit model.

Best,
  John