[FORGED] Re: Using variance components of lmer for ICC computation in reliability study
Thanks all, My original question appears to be now two (1) the distribution of my DV (hence what models to use; (2) specification of my lmer model to parse out variance components. Topic (1): DV distribution Yes, my measure is a sliding rule between 1-10 of subjective pain, so any number up to a single decimal is plausible. Is a linear model automatically excluded, or can (a) do a fitted/residual plot for checking; (b) log transform the dv if (a) shows evidence of non-normality. Going back to Rolf's point of social science, you are right. But realistically, many measures in biomechanics (which I am in), are analyzed using linear models, even though they are bounded. Example, a simple scalar height is bounded to a lower limit of zero, and an upper limit of what ever instrument is created. Joint angles are bounded physiologically too. So when are measures really -inf/inf? Topic (2): lmer Assuming my DV is appropriate for lmer, base on the experimental design used, I hope to receive some feedback on my fixed and random effects specification still ? Thanks again all, for the kind response Bernard -----Original Message----- From: bbolker at gmail.com <bbolker at gmail.com> Sent: Friday, June 15, 2018 2:28 AM To: Doran, Harold <HDoran at air.org> Cc: Rolf Turner <r.turner at auckland.ac.nz>; r-sig-mixed-models at r-project.org; Bernard Liew <B.Liew at bham.ac.uk> Subject: Re: [R-sig-ME] [FORGED] Re: Using variance components of lmer for ICC computation in reliability study More generally, the best way to fit this kind of model is to use an *ordinal* model, which assumes the responses are in increasing sequence but does not assume the distance between levels (e.g. 1 vs 2, 2 vs 3 ...) is uniform. However, I'm not sure how one would go about computing an ICC from ordinal data ... (the 'ordinal' package is the place to look for the model-fitting procedures). Googling it finds some stuff, but it seems that it doesn't necessarily apply to complex designs ... https://stats.stackexchange.com/questions/3539/inter-rater-reliability-for-ordinal-or-interval-data https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3402032/
On Thu, Jun 14, 2018 at 6:58 PM, Doran, Harold <HDoran at air.org<mailto:HDoran at air.org>> wrote:
That?s a helpful clarification, Rolf. However, with gaussian normal
errors in the linear model, we can?t *really* assume they would
asymptote at 1 or 10. My suspicion is that these are likert-style
ordered counts of some form, although the OP should clarify. In which
case, the 1 or 10 are limits with censoring, as true values for some
measured trait could exist outside those boundaries (and I suspect the
model is forming predicted values outside of 1 or 10).
On 6/14/18, 6:33 PM, "Rolf Turner" <r.turner at auckland.ac.nz<mailto:r.turner at auckland.ac.nz>> wrote:
On 15/06/18 05:35, Doran, Harold wrote:
Well no, you?re specification is not right because your variable is
not continuous as you note. Continuous means it is a real number
between -Inf/Inf and you have boundaries between 1 and 10. So, you
should not be using a linear model assuming the outcome is continuous.
I think that the foregoing is a bit misleading. For a variable to be
continuous it is not necessary for it to have a range from -infinity
to infinity.
The OP says that dv "is a continuous variable (scale 1-10)". It is
not clear to me what this means. The "obvious"/usual meaning or
interpretation would be that dv can take (only) the (positive integer)
values 1, 2, ..., 10. If this is so, then a continuous model is not
appropriate. (It should be noted however that people in the social
sciences do this sort of thing --- i.e. treat discrete variables as
continuous --- all the time.)
It is *possible* that dv can take values in the real interval [1,10],
in which case it *is* continuous, and a "continuous model" is indeed
appropriate.
The OP should clarify what the situation actually is.
cheers,
Rolf Turner
--
Technical Editor ANZJS
Department of Statistics
University of Auckland
Phone: +64-9-373-7599 ext. 88276
On 6/14/18, 11:16 AM, "Bernard Liew" <B.Liew at bham.ac.uk<mailto:B.Liew at bham.ac.uk>> wrote:
Dear Community,
I am doing a reliability study, using the methods of
https://www.ncbi.nlm.nih.gov/pubmed/28505546. I have a question on
the lmer formulation and the use of the variance components.
Background: I have 20 subjects, 2 fixed raters, 2 testing sessions,
and
10 trials per sessions. my dependent variable is a continuous
variable (scale 1-10). Sessions are nested within each
subject-assessor combination. I desire a ICC (3) formulation of
inter-rater and inter-session reliability from the variance components.
My lmer model is:
lmer (dv ~ rater + (1|subj) + (1|subj:session), data = df)
Question:
1. is the model formulation right? and is my interpretation of
the variance components for ICC below right?
2. inter-rater ICC = var (subj) / (var(subj) + var (residual)) #
I read that the variation of raters will be lumped with the residual
3. inter-session ICC =( var (subj) + var (residual)) /( var
(subj) + var (subj:session) + var (residual)) some simulated data:
df = expand.grid(subj = c(1:20), rater = c(1:2), session = c(1:2),
trial = c(1:10)) df$vas = rnorm (nrow (df_sim), mean = 3, sd =
1.5)
I appreciate the kind response.
_______________________________________________
R-sig-mixed-models at r-project.org<mailto:R-sig-mixed-models at r-project.org> mailing list