eta-squared in lmer

You may want to report an adjusted R-square value and add a footnote
that this is an area of active development. For example, in Oberauer &
Kliegl (2006, p. 605, Eq. 2,. Journal of Memory and Language, p. ), we
used a formula from (McElree & Dosher, 1989, Journal of Experimental
Psychology: General) and added the following footnote:

"Footnote 3. The development of R2 statistics or equivalent indicators
of goodness of fit for multilevel regression models is an active field
of research (e.g., Roberts & Monaco, 2006). For example, one problem
with the formula employed in this article is that the inclusion of
level-2 predictors (i.e., subject-level predictors) could
theoretically render the adjusted R2 statistic negative. We included
this statistic only for reasons of rough comparability of these
results with related earlier research and as an additional index for
the comparison of nested models involving different level-1 predictors
(i.e., item-level predictors)."

There is probably more recent work on this. At a general level, if
your theory expects a small effect, you should not be forced to
document a large one. Indeed, that might suggest that something went
wrong. Large effects are very desirable in applied settings (where you
do not care why an instrument works), but I do not think it is a
useful general criterion in the context of theory-guided research.

Reinhold Kliegl

eta-squared in lmer

Thread (4 messages)