Skip to content
Prev 19699 / 20628 Next
Original Raw

Collinearity diagnostics for (mixed) multinomial models

Sorkin, John 
Juko,
It is my understanding, perhaps incorrect understanding, that collinearity of the independent variables is accessed independent of the nature of the dependent variable. Collinearity leads to inferential problems, i.e. determining if the predictors variables are independent predictors of the dependent variable. When the independent variables are collinear, the shared variance among the independent variables makes it difficult, or impossible, to determine (1) if the predictors variables are independent predictors of the dependent variable and (2) the magnitude of the contribution each independent variable makes to the prediction of the dependent variable?s value, regardless of the class of the dependent variable (continuous, binary, multinomial, etc.). On the other hand collinearity does not generally effect prediction. A set of collinear independent variables can predict a dependent variable accurately even if one can not separate the contribution of each of the collinear independent variables to the prediction of the dependent variable. I do not see why collinearity among independent variables would have a different effect on, or be looked for in multinomial models any differently than one would look for collinearity in the ?usual? linear regression where the dependent variable is continuous.  I would be happy to be disabused of an error in my understanding of collinearity.
John

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows

From: Juho Kristian Ruohonen<mailto:juho.kristian.ruohonen at gmail.com>
Sent: Friday, February 25, 2022 8:50 AM
To: Sorkin, John<mailto:jsorkin at som.umaryland.edu>
Cc: stevedrd at yahoo.com<mailto:stevedrd at yahoo.com>; John Fox<mailto:jfox at mcmaster.ca>; r-sig-mixed-models at r-project.org<mailto:r-sig-mixed-models at r-project.org>
Subject: Re: [R-sig-ME] Collinearity diagnostics for (mixed) multinomial models

I am indeed talking about collinearity of the predictors, not the response. A multinomial model consists of C-1 binary submodels, so it arguably doesn't make sense to measure collinearity in the entire dataset at once but, rather, it should be measured separately in the C-1 subdatasets to which the C-1 submodels are fit. My question is whether the way I propose to do this (in the original post) is sensible.

Best,

Juho

pe 25. helmik. 2022 klo 15.19 Sorkin, John (jsorkin at som.umaryland.edu<mailto:jsorkin at som.umaryland.edu>) kirjoitti:
I would agree with Steven. Collinearity is problem with the predictor variables, not the outcome variable. Given a multinomial model y = f(x1, x2, x3, . . . xn), one could run a simple linear regression x1 = f(x2,x3, . . .,xn) and look at vif to determine if x2 . . . xn are colinear and perhaps an additional regression x2=f(x1,x3, . . .xn) to determine if x1, x3, . . . xn are colinear. If I am missing something, I hope someone will correct me.
John (but not John Fox)

Sent from Mail<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgo.microsoft.com%2Ffwlink%2F%3FLinkId%3D550986&data=04%7C01%7Cjsorkin%40som.umaryland.edu%7C0f6e1d2cffb8442c60f308d9f865ba36%7C717009a620de461a88940312a395cac9%7C0%7C0%7C637813938549646032%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=YOdb6JICQeEleTBmEc8uU0GR6n5Hsq5TEf8Zd8FAQYQ%3D&reserved=0> for Windows

From: stevedrd--- via R-sig-mixed-models<mailto:r-sig-mixed-models at r-project.org>
Sent: Friday, February 25, 2022 8:07 AM
To: John Fox<mailto:jfox at mcmaster.ca>; Juho Kristian Ruohonen<mailto:juho.kristian.ruohonen at gmail.com>
Cc: r-sig-mixed-models at r-project.org<mailto:r-sig-mixed-models at r-project.org>
Subject: Re: [R-sig-ME] Collinearity diagnostics for (mixed) multinomial models

This seems odd to me, but then I don't usually analyze multinomial models.  Is there an issue with collinearity in the response variable in a multinomial model?  I would think that the levels are collinear by definition.  So then the issue, it seems to me, is whether there is collinearity in the fixed effects - and that should be independent of the response variables.  Could you use the vif() function with a standard response (say = 1) to check collinearity in the fixed effects?  I would think that your method on the sub datasets may not capture all of the collinearity in the full model.
But I could be waaaaaaay off base on this.
SteveDenham
On Friday, February 25, 2022, 03:24:15 AM EST, Juho Kristian Ruohonen <juho.kristian.ruohonen at gmail.com<mailto:juho.kristian.ruohonen at gmail.com>> wrote:
Dear John (and anyone else qualified to comment),

I fit lots of mixed-effects multinomial models in my research, and I would
like to see some (multi)collinearity diagnostics on the fixed effects, of
which there are over 30. My models are fit using the Bayesian *brms*
package because I know of no frequentist packages with multinomial GLMM
compatibility.

With continuous or dichotomous outcomes, my go-to function for calculating
multicollinearity diagnostics is of course *vif()* from the *car* package.
As expected, however, this function does not report sensible diagnostics
for multinomial models -- not even for standard ones fit by the *nnet*
package's *multinom()* function. The reason, I presume, is because a
multinomial model is not really one but C-1 regression models  (where C is
the number of response categories) and the *vif()* function is not designed
to deal with this scenario.

Therefore, in order to obtain meaningful collinearity metrics, my present
plan is to write a simple helper function that uses *vif() *to calculate
and present (generalized) variance inflation metrics for the C-1
sub-datasets to which the C-1 component binomial models of the overall
multinomial model are fit. In other words, it will partition the data into
those C-1 subsets, and then apply *vif()* to as many linear regressions
using a made-up continuous response and the fixed effects of interest.

Does this seem like a sensible approach?

Best,

Juho




ma 27. syysk. 2021 klo 19.26 John Fox (jfox at mcmaster.ca<mailto:jfox at mcmaster.ca>) kirjoitti:
_______________________________________________
R-sig-mixed-models at r-project.org<mailto:R-sig-mixed-models at r-project.org> mailing list
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-mixed-models&amp;data=04%7C01%7Cjsorkin%40som.umaryland.edu%7C5fb7bcf6b8824a3109f708d9f85fa6f1%7C717009a620de461a88940312a395cac9%7C0%7C0%7C637813912584894963%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0&amp;sdata=VXBlGoxZ5iq3OWpGhpxjVbAn9w4OUUTtSp8BARHFQW0%3D&amp;reserved=0<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-mixed-models&data=04%7C01%7Cjsorkin%40som.umaryland.edu%7C0f6e1d2cffb8442c60f308d9f865ba36%7C717009a620de461a88940312a395cac9%7C0%7C0%7C637813938549646032%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=rwAL1mmsx9aToNsg67XbvZf4G8FnPt21I29JnrBdiww%3D&reserved=0>


_______________________________________________
R-sig-mixed-models at r-project.org<mailto:R-sig-mixed-models at r-project.org> mailing list
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-mixed-models&amp;data=04%7C01%7Cjsorkin%40som.umaryland.edu%7C5fb7bcf6b8824a3109f708d9f85fa6f1%7C717009a620de461a88940312a395cac9%7C0%7C0%7C637813912584894963%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0&amp;sdata=VXBlGoxZ5iq3OWpGhpxjVbAn9w4OUUTtSp8BARHFQW0%3D&amp;reserved=0<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-mixed-models&data=04%7C01%7Cjsorkin%40som.umaryland.edu%7C0f6e1d2cffb8442c60f308d9f865ba36%7C717009a620de461a88940312a395cac9%7C0%7C0%7C637813938549646032%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=rwAL1mmsx9aToNsg67XbvZf4G8FnPt21I29JnrBdiww%3D&reserved=0>
Message-ID: <MN2PR03MB5167467057A768AEC62EDD40E23E9@MN2PR03MB5167.namprd03.prod.outlook.com>
In-Reply-To: <CAG_dBVdJy=gK6xDdh8pB5jMxtOaZ=rbca0tN2p-bDKvdWsDeVg@mail.gmail.com>
Back to archive Original archive Thread
5 / 5

Thread (5 messages)

Juho Kristian Ruohonen Collinearity diagnostics for (mixed) multinomial models Feb 25 stevedrd m@iii@g oii y@hoo@com Collinearity diagnostics for (mixed) multinomial models Feb 25 Sorkin, John Collinearity diagnostics for (mixed) multinomial models Feb 25 Juho Kristian Ruohonen Collinearity diagnostics for (mixed) multinomial models Feb 25 Sorkin, John Collinearity diagnostics for (mixed) multinomial models Feb 25