Different LLRs on multinomial logit models in R and SPSS

Date: Fri, 7 Jan 2011 11:23:04 -0500
From: David Winsemius <dwinsemius at comcast.net>
To: sovo0815 at gmail.com
Cc: r-help at r-project.org
Subject: Re: [R] Different LLRs on multinomial logit models in R and SPSS


On Jan 7, 2011, at 8:26 AM, sovo0815 at gmail.com wrote:

On Thu, 6 Jan 2011, David Winsemius wrote:

On Jan 6, 2011, at 11:23 AM, S?ren Vogel wrote:

Thanks for your replies. I am no mathematician or statistician by far,
however, it appears to me that the actual value of any of the two LLs
is indeed important when it comes to calculation of
Pseudo-R-Squared-s. If Rnagel devides by (some transformation of) the
actiual value of llnull then any calculation of Rnagel should differ.
How come? Or is my function wrong? And if my function is right, how
can I calculate a R-Squared independent from the software used?

You have two models in that function, the null one with ".~ 1" and the 
origianl one and you are getting a ratio on the likelihood scale (which is 
a difference on the log-likelihood or deviance scale).

If this is the case, calculating 'fit' indices for those models must end up 
in different fit indices depending on software:

n <- 143
ll1 <- 135.02
ll2 <- 129.8
# Rcs
(Rcs <- 1 - exp( (ll2 - ll1) / n ))
# Rnagel
Rcs / (1 - exp(-ll1/n))
ll3 <- 204.2904
ll4 <- 199.0659
# Rcs
(Rcs <- 1 - exp( (ll4 - ll3) / n ))
# Rnagel
Rcs / (1 - exp(-ll3/n))

The Rcs' are equal, however, the Rnagel's are not. Of course, this is no 
question, but I am rather confused. When publishing results I am required 
to use fit indices and editors would complain that they differ.

It is well known that editors are sometimes confused about statistics, and if 
an editor is insistent on publishing indices that are in fact arbitrary then 
that is a problem. I would hope that the editor were open to education. (And 
often there is a statistical associate editor who will be more likely to have 
a solid grounding and to whom one can appeal in situations of initial 
obstinancy.)  Perhaps you will be doing the world of science a favor by 
suggesting that said editor first review a first-year calculus text regarding 
the fact that indefinite integrals are only calculated up to a arbitrary 
constant and that one can only use the results in a practical setting by 
specifying the limits of integration. So it is with likelihoods. They are 
only meaningful when comparing two nested models. Sometimes the software 
obscures this fact, but it remains a statistical _fact_.

Whether you code is correct (and whether the Nagelkerke "R^2" remain 
invariant with respect to such transformations) I cannot say. (I suspect that 
it would be, but I have never liked the NagelR2 as a measure, and didn't 
really like R^2 as a measure in linear regression for that matter, either.) I 
focus on fitting functions to trends, examining predictions, and assessing 
confidence intervals for parameter estimates. The notion that model fit is 
well-summarized in a single number blinds one to other critical issues such 
as the linearity and monotonicity assumptions implicit in much of regression 
(mal-)practice.

So, if someone who is more enamored of (or even more knowledgeably scornful 
of)  the Nagelkerke R^2 measure wants to take over here, I will read what 
they say with interest and appreciation.

S?ren

David Winsemius, MD
West Hartford, CT