[FORGED] Re: Using variance components of lmer for ICC computation in reliability study
That?s a helpful clarification, Rolf. However, with gaussian normal errors in the linear model, we can?t *really* assume they would asymptote at 1 or 10. My suspicion is that these are likert-style ordered counts of some form, although the OP should clarify. In which case, the 1 or 10 are limits with censoring, as true values for some measured trait could exist outside those boundaries (and I suspect the model is forming predicted values outside of 1 or 10).
On 6/14/18, 6:33 PM, "Rolf Turner" <r.turner at auckland.ac.nz> wrote:
On 15/06/18 05:35, Doran, Harold wrote:
Well no, you?re specification is not right because your variable is not continuous as you note. Continuous means it is a real number between -Inf/Inf and you have boundaries between 1 and 10. So, you should not be using a linear model assuming the outcome is continuous.
I think that the foregoing is a bit misleading. For a variable to be continuous it is not necessary for it to have a range from -infinity to infinity. The OP says that dv "is a continuous variable (scale 1-10)". It is not clear to me what this means. The "obvious"/usual meaning or interpretation would be that dv can take (only) the (positive integer) values 1, 2, ..., 10. If this is so, then a continuous model is not appropriate. (It should be noted however that people in the social sciences do this sort of thing --- i.e. treat discrete variables as continuous --- all the time.) It is *possible* that dv can take values in the real interval [1,10], in which case it *is* continuous, and a "continuous model" is indeed appropriate. The OP should clarify what the situation actually is. cheers, Rolf Turner -- Technical Editor ANZJS Department of Statistics University of Auckland Phone: +64-9-373-7599 ext. 88276
On 6/14/18, 11:16 AM, "Bernard Liew" <B.Liew at bham.ac.uk> wrote:
Dear Community, I am doing a reliability study, using the methods of https://www.ncbi.nlm.nih.gov/pubmed/28505546. I have a question on the lmer formulation and the use of the variance components. Background: I have 20 subjects, 2 fixed raters, 2 testing sessions, and 10 trials per sessions. my dependent variable is a continuous variable (scale 1-10). Sessions are nested within each subject-assessor combination. I desire a ICC (3) formulation of inter-rater and inter-session reliability from the variance components. My lmer model is: lmer (dv ~ rater + (1|subj) + (1|subj:session), data = df) Question: 1. is the model formulation right? and is my interpretation of the variance components for ICC below right? 2. inter-rater ICC = var (subj) / (var(subj) + var (residual)) # I read that the variation of raters will be lumped with the residual 3. inter-session ICC =( var (subj) + var (residual)) /( var (subj) + var (subj:session) + var (residual)) some simulated data: df = expand.grid(subj = c(1:20), rater = c(1:2), session = c(1:2), trial = c(1:10)) df$vas = rnorm (nrow (df_sim), mean = 3, sd = 1.5) I appreciate the kind response.